ROUGH EDITED COPY

The Computer ate my VCR: Creating a digital media lab for use in Instruction

June 27, 2005

1:00 P.M.

CAPTIONING PROVIDED BY: CAPTION FIRST, INC.

PO BOX 1924

LOMBARD, IL 60148

* * * * *

This is being provided in a rough-draft format. Communication Access Realtime Translation (CART) is provided in order to facilitate communication accessibility and may not be a totally verbatim record of the proceedings.

* * * *

>> good afternoon, my name is Delbert DAEGLE.

I'm going to introduce our presenters this afternoon.

First, I would like to recognize our interpreter Christina,

our two presenters this afternoon, the first Gary Abernathy.

He is the Director of Technology for the national terrorism preparedness institute.

And our second presenter is Beth Carlson.

Beth teaches English and reading and sign language at St. Petersburg college.

And she has a Ph.D. in second language acquisition and information technology.

Instructional technology.

Their topic is the "computer ate my V.C.R.: Creating a digital media lab for use in instruction."

Thank you.

>>[INAUDIBLE]

>> there's no audio.

>> okay.

I'm supposed to give you 30 seconds and I didn't do that.

I told you you were going to have to remind me.

Rewind, that sounds better, huh? should I start again or just keep going?

Anyway, he has been now at that location but really still is very, very involved and interested in delivery systems for even in my area teaching English and reading to students who are deaf and in the interpreter training program.

So we're lucky that he has this interest because basically it's on his free time.

Okay.

Don't look so serious.

I think we -- we've all were happy to say that now we're beginning to see that we're teaching ASL in the sign language programs as a foreign language, as a second language, but I don't know how much really yet we're actually looking at the research on second language acquisition and how much are we applying that into teaching methodology.

I know we're doing a lot, it's in the beginning phases, it's really been in the last ten years that we start to look at a lot of. This so a few things we find in our teaching methods in ASL I, II, III and the interpreter training programs, some of these issues come up again and again.

In terms of exposure to languages, we know that you have to have exposure.

You have to have a lot of exposure and then how much of that exposure is sufficient for learning and how much of it needs to be focused on specific items, forms, structures, areas and how much can just be sort of acquired through negotiations, interactions, so on and so forth.

Unfortunately as an instructor you have to grade these students.

You have to have some method of assessment, so you can't really do it on the spot with sign language so much.

So of course the way the teachers tend to teach is that in class, you know, there's a lot of dialogue, you know, no voice, and then maybe give a little feedback during a sign error, or, you know, or a hand shake, okay, whatnot, but how much of that really translates into how the students are doing?

It's really not that certain.

We do know that there has to be exposure, there has to be attention to form and structure and the learner has to be -- has to be manipulated, targeted with the task at what you're trying to have develop.

Okay.

These are -- these are common sense notions, really.

And so but it's tricky in an ASL class.

You can present the grammar and the vocabulary.

You can present the interactions, the dialogues, but it's not like in, say, for instance, a Spanish class where you have recordings, you can listen, you can go check, you can go through dialogues.

Video technology has improved a lot.

Okay, how much we can archive and view.

But the input has to be structured an manipulated to make the forms really salient and designed also so that learners can attend to those forms.

Explicit instruction can facilitate acquisition.

So that is kind of a little bit -- Gary and I were talking about as we got together on this.

Here is your part, here is my part, and I-- as we talk about these things and I think more and more now we all need to become focussed if we teach ASL or as we teach deaf students English and reading, same concept, it's English now.

These are some of the issues, what are the feedback techniques and how are error correction methods used?

And is it delayed versus immediate?

Is it explicit or implicit?

Are you focusing on form or the function?

And is it holistic or analytic?

This is what the conference of interpreter trainers says that anyone who is teaching interpreter trainers or sign language as -- or ASL should be thinking about.

Appropriate recording facilities facilitate and play back equipment must be made available with freeze frame slow motion capabilities so that student consist see their errors an facilitate self construction production.

All grading should be done on videotape to provide full and appropriate feedback and does include the time we sit down with each student.

If you think about the video, the way we traditionally looked at, you know, the teacher takes the video, used to be, now it's CD's, right?

But what Gary is going to talk about is even better, you would have these videotapes in your bag and you would carry your fifteen videotapes off and sit down for hours and hours and hours and, you know, stop, right, stop, right, stop, right, and we went to CD.

Now, we're at digital.

What are we doing?

We're carrying the CD's around.

Okay.

Same concept, really.

Stop, right, stop, right.

So, yeah, we've made improvements, but we're not quite where we need to be yet.

I know there's a lot of instruments that people use for measuring assessment, whatnot.

This just happens to be one simple one that we have at St. Pete college and if you remember the old ABC of ASL book, in it's what 25th printing or something, it has the dialogues in it, and this was dialogue 3.

And so this is the instrument we use when we look at now what we open up on the computer, laptop, whatever, you know, stop or pause, feedback, pause, feedback, pause, so we're still doing that, and what are we looking at?

The aspects of language.

But if you think about it, when you look at that one little five-minute clip, how do you adequately address every one of these major areas?

So you tend to focus on what are the biggest problem areas.

And I -- how many of you are involved in at all teaching sign language or working with deaf students?

I mean, okay.

Okay.

So this will be familiar to you.

This is an ASL I class.

These happen to be two of my students.

I have four to show you.

And I want you to raise your hand when you see an error.

Okay?

I'm going to try to stop it, okay? Hopefully I can do this right.

Where is the mouse -- I mean -- there it is.

>> when you see an error.

Same dialogue.

Raise your hand.

Oh, come on, now.

Okay.

Keep going.

They're afraid to raise their hands.

See an error?

Raise your hand.

Good.

Good.

Okay.

So has it finished yet?

Okay.

So what do you think the average was, about every two seconds you saw something, right?

How facilitative is that kind of feedback?

And how would you go about doing that?

Sitting down with a student?

Well, I've got to tell you, how realistic is that?

It would be great, but if you teach five classes of ASL and you have fifteen students in each class, how much time are you going to have to sit down with each student and if you have three assessment, those three dialogue, you have a mid term and a final.

Okay.

It's not that effective to do it that way.

It's best for the student, but in terms of time, it's consuming.

I was watching Catherine and -- cat here, and about every three seconds they raised their hands.

What were most of the errors?

Hand shape.

>> it was related to the hand shapes and the movement.

And the facial expressions.

I need to repeat that...

>> oh, most...

>> they were related to the hand shape and the body posture and the facial expressions.

>> okay.

Now, it's easy if you're sitting down with that student to stop them and say, whoops, make a correction, but if you're not, it's not that easy, but, yeah, most of the errors tend to be movement, nonmanuals and they're actually a couple location errors but a hand shape phonology really is the biggest.

So we also know 150 hand shapes, so what I'm trying to get to you picture is how big is this job really that we're giving feedback on.

If you truly want those students to learn the language, you need to be thinking about all of these things.

And then, you know, there's so many differences in variation where it's okay if you do it this way, and, you know, you're going to have the influences of the language in another sentence.

So there's a lot to look at.

If you think about the locations of signs, you know, these are all of the types of locations that you can have for placement.

And in movement of the signs, you know, you're looking at these kind of sequences, these are articulatory bundles, do they have that, and force if it's good idea, movement, how do you communicate those kinds of things to the student who is ASL I, didn't like Spanish, too hard, want ASL, easy.

Okay, but really if you look at your classes, ASL I, II, there's their four years of foreign language a lot of them.

Now the interpreting students, it's easier because they get this basis and they're at the next level but those beginning students, you know, there's a lot to the language that I think they don't assume.

It's gesture, it's all pictures.

It's all iconic, all the signs -- you can see them.

Well, we know that is not true.

And then, you know, in situations like where deaf is okay this way, but the next time you see it, it's this way.

They need to learn that, that those factors that affect how the sign is made.

Palm orientation, we all know, you know, when you go from something like sit to name, you can change the meaning, and then, you know, with movement, where you have this, all those things that become involved.

Where did my video go after that?

Did we lose it? I hope not.

Okay.

No, it comes after this, I think.

How about, you know, in terms of nonmanuals, you have those students in ASL I and II and they come in and they're like... they're afraid to move their face, and yet that carries so much grammatical information it can't be ignored.

Like not yet, if your tongue is not out, it's not right.

Home, and then questions and negatives.

Or two different sentences. Or differences in meaning.

And then in terms of discourse, you can also tell whether it's recorded speech, turn taking, topic control.

And I want to sort of emphasize the impact, two of my students will, of nonmanuals and I want you to raise your hand whenever you see kind of a nonmanual error.

Right here, right?

There we go.

>> what is it, about every second?

How do you give feed -- feedback on that?

You can tell we emphasize -- [LAUGHTER]

Look at his expression here.

Now, he did wrinkle his brows for the -- yeah.

And there's a little fuzzy something that happens in the middle of this, but just ignore that.

Okay. Yeah, so once again, you're at about every two seconds you see an error.

Okay.

How much of that would you want to focus on?

If you went back to what we looked at, the major factors, because you can totally overwhelm them, right?

But they need to have the feedback that relates to how well they're learning the language.

And the factors that affect meaning, okay, have to be dealt with.

They have to be dealt with.

Question?

Oh.

Gary, can you...

>> not all areas -- some areas actually cause problems in terms of meaning.

You're saying particular areas, but these are the ones that cause communication break down.

>> right.

You know, I think eventually you have to begin to target most everything.

Not target, but you have to bring it to their attention that here is a mistake.

Now, this one is critical.

You said where's the train?

Big mistake.

Train, no.

Sit, wrong.

Okay.

You've changed -- you've affected the meaning.

The students say when we go through practice sentences, whenever you start affecting the content it's a very big mistake T content and the meaning.

Okay?

We continue to focus on phonology, phone logical processes, but in terms of the meaning you need to really start with that, I think.

This is my own opinion, I guess, because clearly every two to three seconds what are you going to do?

It's just like with deaf students and writing, you don't, you know, the paper because they'll give up and then they will go back to Spanish and say ASL hard, Spanish easy.

Okay.

What?

So again, there's the kind of information we tend to look at and in terms of these are more grammatical.

They don't affect not looking at meaning a lot, so now what we do, we take the CD, we do this, but Gary is going to show us a better way.

The reality of this kind of feedback.

Is it explicit, is it implicit, or is it nonexistent?

Because what do most students do?

They get their CD, they get their paper back, they go home, they put it away, probably.

A lot of them may actually go back and look at them, maybe 30%, I suspect.

And then we don't know really what is the effect if we did it during, what's the difference, because you've got to have a way of recording I.

Implicit, this was at best now we're sort of at an implicit level and it takes a lot of time to get anywhere with that.

Some more really is needed that looks at -- I got louder all of a sudden and moved my lips whatever.

What happens in on online test, when I say "online," I don't mean online, I mean at the point of learning when it can be adjusted.

In content rich language situations.

So now that we have all of this great technology, we want to start looking at ways that we can -- new ways to use it to kind of achieve these tasks that we're trying to do and really go back to what we know are best practices in terms of learning languages.

Okay.

Gary's turn now.

You don't have to wear the funny belt.

>> I might want to.

>> can I go on this side?

Is that okay?

>> can you hear me?

>> can you hear me now?

>> okay.

So my role in this interaction here was to try to come up with a way to hopefully improve instruction or help the instructors with their instruction.

Some of the issues were really evident to me.

I know very little about sign language per se, but I am a teacher, I'm an instructional technologist, and we looked at other video models that were appealing in terms of approving an instruction.

I wanted to synchronize the students' modeling of the assignment or behavior or interpretation of the text or the voicing of the signing in either direction.

I want them to take the media and synchronize that and allow the instructor to stop it so that you could insert comments.

You could give the student feedback at the point where the student needed feedback, not after the fact.

Unfortunately, the world that we were in was the world of VHS.

In reality, the instructors would take a pile of videotapes and the first time I saw this, I was coming from a different world.

The first time I saw an instructor with literally 20 or 30 VHS tapes carrying them to take them home because they couldn't possibly get it home during the course of their day, play them back, play, pause, write a note, equate that to a great, and go on and on, it takes hour to do this.

I'm not sure what the factor is, but multiples of one, so if you're teaching for X number of hours and you start multiplying that by factors, you realize they were getting overwhelmed T other thing I know, I only know this anecdotally, I'm an English teacher gone bad, I would say, turning to technology, I would review papers, give them corrections and the students would focus on one particular piece of information.

The grade.

Didn't matter what I wrote on the paper, I could have said I hate your family, everyone is going to fail, and that didn't matter as long as it was an A or a 95 at the end of the paper.

And I realized that same thing was happening with this instruction.

So I want to make a move from these large bulky VHS tapes away from the cassette decks which I was astounded still using cassette decks and move toward the digital realm.

The digital realm has multiple formats and everyone has their preference, and I don't want to debate that.

The path that we chose was to use window's media, it was so prevalent, virtually every machine runs on windows.

So we want to take the videotape, the VHS tape, the analog tape, and turn it into a digital file, a wmv in this case, but the format is not that important, and we wanted to be able to take signed and digitize that as well and there's a number of different reasons you wanted to do that.

Analog is bid.

Analog is linear, which means you're playing a videotape, you literally go, play, stop, play, stop, and you go on and on.

You can't move around within the video that you're watching.

In reality, we typically communicate nonlinearly.

We move around, we hit tangent, we focus on topics and you might want to go forward an backwards and associate things, look for recurring errors, you can't do that in an analog/linear format.

I hope in the movement to digital we end with the smaller nonlinear files, the files are dynamic, I can change them quickly.

I can reuse them because they're small.

I can move them around via the -- don't have one -- flash drive.

You can move them into a network location, you can save them into a file in a PC or put them on a CD, but the bottom line is they're little, they're fast to deal with, and you don't have to deal with them in a lock step 1, 2, 3 fashion.

So we end up with something that is portable.

Because it's portable and smaller, it's more efficient, you spend less time wrestling with it, you have less space, because it's digital and it's computer based, that means you can actually access files remotely.

You can do the same thing for the students.

You can give them access to the file with the feedback from wherever.

Including online, so you don't carry a pile of tapes, go back to the class, turn them in, get them back and go round and round.

You can also put comments in -- and retain them, create a portfolio in one stored area.

Okay.

So how are we going to do that?

Good question.

When I started this two years ago, we did it with a very expensive tower, it was about $3,500 PC, just a big box, processor, lots of memory, I don't know how technology folks are here, but the fastest machine I could buy with the most memory it would handle with very expensive video capture cards, these were cards that were almost a thousand dollars apiece, I linked the cards to mini DV cameras, the cameras were about a thousand dollars apiece, when all was said and done I had a computer that itself was $4,000, I had a flat screen monitor that was about a thousand dollars, I $42,000 in cameras and $2,000 in video cards.

And it worked.

And all I needed was about another 18 months of design an programming and what not and we got it to work.

Over the course of two years, we kept playing with this, kept tweaking it, and we ended up making our own application.

We made an interface for this thing that allows you to create a structure and you can structure it however you like, a class, a semester, a class, an assignment, portfolios within the assignments, the assessments are imbedded in the students' work, and you had control over the structure of the assignments.

By that I mean I wasn't trying to limit this to just the video.

Wanted to be able to use this tool to help the interpreters so they might be voicing some signed component and it was checking the accuracy of their voice, or go the opposite direction, signing something that is voiced, or have students learning ASL and looking at their technical approach which that is expertise, the hand placement, the facial expression which I can understand a little bit about but I understood the criteria enough to know what we were looking for.

What do we do?

We make this program that does what it described, makes a class, makes a semester, makes an assignment, makes a portfolio.

The teacher dictates all of that.

Call it what you like, describe what you want to work.

What do you want in the assignment?

Do you want just text?

Do you want an existing piece of video?

Do you want a sound file?

Really it's mix an match.

You can put these things together however you like, but the goal, again, is to retain this, make it small, make it digital, make it archivable, make it portable and allow a student to receive feedback from their instructor, so we need this thing.

It outputs WMV files, windows media video, standard little video, that is free as long as you have XP and you bow down to Mr. Gates, which I think about 98% of the world is at this point.

We wanted to put comments back into the video, but they needed to be text based because if we have students that are deaf, they're not going to hear it, obviously, so we put XML driven caption, so it's text, XML markup language so -- not sure not geeky we are, essentially lets you link other things together, have the XML tell the video to be here and the caption goes over here and the whole thing goes over there.

It's a little controller of this stuff.

I need to put it into something that was user friendly.

Cool, got all of this whiz BANG deep stuff but how are we going to use it?

I put it in HTML.

It's got buttons on it driven by java script.

More geek talk.

Programming language that allows you to control the video player.

Remember, we have videos.

So we make this thing that lets you do what you're going to see.

The resulting format, it puts it into a folder, the folder contains the video, either one, two or the component that you use in your assignment.

Feedback from the instructor if you chose to give them feedback, it has the XML that controls all the things and how they interact, it controls the HTML, ends up with a window that has play buttons that you're familiar with, that play videos, the video, if we're using two videos, are synchronized, they move in real time, this happens in real time, you go in the classroom, if the technology doesn't fail me, I'll attempt to show you that shortly, here is what the thing looks like.

Yes, we like security.

>> sign language interpretation.

An example could be I could ask my students who is interested in the medical profession or the legal profession.

The legal profession student can go out and they could...

>> now, here I stuck a comment in.

This particular piece was done -- I used the same person so you could see it was done at a different time, but we can do it simultaneously.

This was carol downing, who was one of our instructors at St. Pete's, she's gone on to another teaching position.

She was her own interpreter.

Down below I inserted the comments.

This is the output file the student would receive back.

I forgot what I put --

>> said don't move so far... but he doesn't remember, he doesn't need to -- No.

>> yeah, carol might have given me that, I'm not sure.

Yeah.

And the point was to show you these comments could be whatever you want.

We make a list of comments, you can identify the criteria you want to reference and type of comment you want to use so there's a Pat drop down, you pick the comment and insert it or you can make your own, so whatever is on the list doesn't work, you can add new ones or put in your own created on the spot comments.

>> get themselves set up so they can interpret, say, court TV.

They would have to know of course all the jargon and all the lingo.

They would then put that in their CD and give it back to me.

I could then sit down and grade their presentation organize accordingly.

A student interested in the medical field could find a presentation on retina STIGMATOSIS.

I could review their success in having interpreted that situation.

It allows for interpreting students to get real life experience in an environment that enhances individual development.

So as you can see, for teaching the art and science of sign language interpretation, or any language, the DML has truly proven to be unequalled in its effectiveness for both the teacher and the student.

The applications are endless.

>> and of course there's controls so we go backwards and forwards, you can move through this at whatever speed you want, if it were a longer piece you wouldn't have to watch the whole thing if you're focusing on some particular type of instruction.

Now the trick is to see if I can get back to the Powerpoint from here.

Okay.

So the thing that you just saw ends up being browser based, we tend to be in windows an internet explorer, it's browser based, it's HTML, these happen to be WMV's.

We started this thing working with a very expensive tower, very expensive video card, relatively expensive video cameras and it now works on my laptop which is about a year old with web cams, no card, just a built-in board-based piece and if it doesn't blow up the projector, we'll get to show that to you shortly.

Okay.

>> just, again, to tie it back a little bit about in terms of its usefulness in language -- can I just hold that, because I'll give it back to you in a minute.

In terms of the usefulness of such a tool, rather than delayed feedback for students, it's -- it's, you know, occurs during the interaction and it kind of addresses the targeted information at the point that maybe errors occur, and in some ways it can be much more effective because you know that they are exposed to it, in fact, if they have to wait until the end of the video to even see their grade, they do have to run through it, it's not that they can click open, see their grade, close, they have to start at the beginning, view all the feedback, and then they get their grade so... okay.

Gary is going to show you more of it.

Are we there yet?

We're not.

>> no, that's okay.

As I said we started out, we had a relatively expensive proposition.

It will now run on a laptop, these cameras when I bought them were about 179.

They're two USB web cams, cheapie cameras, plug them in and away you go.

The program that we have, college gave us permission to sell these to other schools if you're interested in it, 19.99 for one and let you put it wherever you want for $5,000.

Now, references, should you need them, this will be on a presentation which we will upload for you.

And go ahead.

>> I was wondering, you know, when you showed [INAUDIBLE]

Talking on the left side and on the right side she's interpreting [INAUDIBLE]

Can you do it where you would have the -- especially when you're trying to correct Nan manuals or hand shapes or movement, body shiftings, things like that, could you have the student enterprising the segment they did on the left and have a deaf person or instructor signing on the right with comments underneath so they can really go back and forth.

>> yes.

>> that would be great.

>> you can use any combination that you want of media.

I used the dual video because for most people that is the most sophisticated approve.

I'll show you momentarily that we can use a text window, I can use a sound file, I can use an existing media, someone else's video or you can use real time dual video simultaneous input and that was the hardest one to accomplish, to have two cameras running concurrently, synchronized timewise and embed text into them and have them archived.

InitialLy that required an extremely powerful machine, sophisticated cards and it's now down to a laptop.

Any other specific questions?

I'll show you the whiz bang here in the last two minutes, if I can.

I can't get out of here because I can't see anything.

Yeah, couldn't see what I typed.

Now the keyboard is not working for some reason.

>> and I just made a nonsenseical task here, here is a class, a bunch of example, the last one is an assignment for RIT... is this working?

You see the encoder should -- the encoders are mad.

Yeah, we had issues with the projector freaking this out.

Of course it worked in the showings without you folks here.

As is always the case.

>> what it can do is pick up each camera shot and put it into the two [INAUDIBLE]

>> yeah, it wouldn't launch the second driver.

It's something in the card or... won't pick up the second driver.

I changed the preference for the projector an unfortunately it won't see both drivers.

After the fact, once I disconnect it from this projector I will be happy to show it to you.

I'm stimied, technology has turned against me, unless you have a quick solution.

Questions?

>> if you want to talk to Gary a little bit afterwards, he has this all set up and you can see how it worked a little bit better.

We have to share one.

Are there any more questions?

Nope?

Everyone... nothing?

Okay.

Thank you.

>> it's cooler when it call works in front of you.

[ APPLAUSE ]

>> there's information on the -- the question was where would you go to buy this?

And the -- actually I'm the con tact at the moment.

I had to convince folks at St. Pete college that this might be worth something to other people.

We spent thousands of hours on this and it runs on a pretty inexpensive platform, and it will actually work.

I'll be happy to show you individually.

I'm on this presentation or you can go to DML the digital media lab support and the e-mail forward to myself, andy the interface for me and a young man I work with is just brilliant, did most of the programming, and he and I can actually accommodate almost any eventuallity summing you're on a USB platform and you have a USB based video or a fire wire, I'm sure folks want to use an inexpensive environment, you can configured it to work with a higher power tower.

The better the camera, the better the image quality.

I think they're good enough to see what it is you're looking at.

You know, I'm not an expert in that area, but, yes, question?

>> you have this SLIP program in Spanish?

The only portion of it that is really language specific, this thing is actually designed to work in any language, the original intention was to apply this to sign language and to interpreter training, but the reality is those comments are simply lines of text.

You could type in whatever you want T sound files that you make, whether it be Spanish, Russian, Chinese, Japanese, it really wouldn't matter, that is your input, and the video, of course, whatever language your students are using would be the appropriate one, so it's not English based.

Any other questions?

Yes.

>> I'm not very high Tech as you'll be able to tell by my question. The part that you said that you can purchase, the part that you're selling is a software that would be necessary and you can -- up to $5,000 and put it in as many places as possible.

>> yes.

>> so that as many people around the state could have --

>> should qualify that within a site.

I'm sure that if you wanted to buy it for a whole state, it took me a year and a half to convince the college that this might be something other people would like to buy.

And I had a really hard time doing it, and quite frankly they had never dealt with any kind of IP intellectual property right and I signed away my rights, I don't make any money for this, I thought it was something worth while, they wanted to build in enough money for Tech support, and basically make it functional unto itself, so they're letting me sell this per site, if you want to put them in a bunch of stations for student, multiple students could use it within a given campus, I would suspect if you wanted to negotiate statewide...

>> there's 8 coordinators around the state that do assessment.

>> we could probably work that out.

I don't think anyone is going to get rich on this.

That wasn't the intent.

Yes?

>>[INAUDIBLE]

>> we had it on our computers for last semester and the previous semester and our -- one of our technologists not on the campus deleted everything.

So it's been off the computers for awhile.

We've played with it.

I haven't gotten to use it in my classes.

Oh, yeah.

>>[INAUDIBLE]

>> oh, yeah, I do, I haven't quite -- and that sign language interpretation program is getting moved into the communications building and where the deaf students are, so once that happens we're going to get a whole -- how did you interpret that (indicating) we're going to get ahold of it in our computers where it needs to be, as well.

So.

Thank you, everyone.

* * * * *

This is being provided in a rough-draft format. Communication Access Realtime Translation (CART) is provided in order to facilitate communication accessibility and may not be a totally verbatim record of the proceedings

* * * *