My colleague Helmut Baehring and I are working on the development of a graphic input, output tablet at Fern Universitaet Hagen. My English is not that good so I'll try to make a little joke - I'm a little bit jealous about these gels. Sounds very good, if this price of $1000 is realistic after all the managers have done their work. I think we would be out of this race. But okay. We will try what we can.
The contents of my talk include why tactile graphic input/output tablets are needed. Then I would like to define the properties of a nearly ideal tablet and deal with it's effects and why an ideal tablet isn't yet available. Then I'll explain the construction of an electrorheological tablet driven by an electroheological fluid, outlining some advantages and disadvantages of such tablets. And, I have a little bit about the graphical user interface problem which may be skipped in my talk.
Applications would be reading text illustrated by drawings or text mixed with symbols like in dots plus, reading mobility maps, using electronic communication media, creating drawings. This is an input tablet also, so you can feel immediately what you have done or can work with objects you're feeling. We would use things like oscilloscopes or cameras in education. And, last but not least, using graphical user interfaces. For example, in comparing static presentations of maps or something like that with doing it with dynamic tablets, there would be the advantage that you do not have to use arrows or special markings for the connection between graphical items and explaining text. You just have to touch a graphical item and if you have a split screen you would get the text for this item in the text area or, the other way around, if you are reading about something in a text, touching this text could in some way enhance the presentation of the appropriate graphical item.
So, now to the ideal tactile tablet. We have heard about this 20 or 30 dots per inch and, okay, we depend on the same technology. Because you want to display braille you do have not the whole scale between 20 and 30 dpi, but 20 or 30 because distance between the braille dots is a tenth of an inch. With 30 dpi you would have a little bit better graphical presentation and you would have larger braille dots because each braille dot would consist of four pixels of the tablet and that's more like paper braille. Contrary to this, with the 20 dpi you would have one braille dot presented by one pixel and that would mean you would have the same distances between the braille dots as usual but the dots would feel smaller. I had some examples on swell paper and I showed it to a blind colleague. He said at once, "okay, I can read it, but it feels unfamiliar because the characters are smaller than normal." I disagreed, but he was right when you are looking for the whole size of the characters because the dots are smaller.
The area of the whole tablet should be large enough to display a lot of information but not so large that the user has to search around in order to find the objects. And so, we think it would be nice to have an area that can be covered by two hands. And, that would be between 16 by 12 inches to 20 by 15 inches. And, with 20 or 30 dpi, it would mean for the whole tablet between 80,000 and 270,000 pixels--that's a lot. Each pixel should be set or reset without influencing the other pixels. If you want, for example, to move the cursor over the tablet, you do not want to clear the whole screen and bring up the new picture, being just the same picture with another cursor position.
Then, the tablet should be an input tablet as well, of course. Because, for example, if you could feel with one hand some objects on the tablet and have to use a mouse or cursor keys or something like that to move a cursor over the tablet and to follow with your hands across, it would be an inconvenient process. So, if you touch an object and you want to shift it, delete it, or something like that, it should be possible to do it immediately on the tablet surface.
Another demand would be, of course, rapid changing of the whole display. Of course, it will take some amount of time in every case but it should not be such a long time that the user has a feeling he has to wait for the tablet to become ready.
It would also be fine if this tablet was flat and light weight because then you could use it with portable computers. For example, some of you know the e-mail mailing list "blind-L". In the last weeks there was a discussion about aids to help you to go to an unknown environment, using mobility aids. One point was that it would be fine if there would be mobility aids for sighted persons, too. (laughter) It will not be practical, was the point, to set up a whole system for blind people alone, but if all people were using such systems, blind people could use it too if they had some tactile display showing a piece of a virtual map and so on. So perhaps it could be an application. And, of course, then this tablet must be very light weight to be useful. So, those are the demands, now the problems.
You have 80,000 to 270,000 display elements and you cannot build a controller with 80,000 outputs and control each element separately. The display elements have to have some kind of memory. Then you can address one display element and tell them you have to be settled or you have to be lifted up and then address another. If this addressing process is working very fast, it seems as if you have control over all elements immediately. Now if you use normal electromagnetic or electrostrictive display elements, they have no memory effect. Of course you might have some kind of multiplexing but with such a large amount of display elements this would be very hard to realize.
The next point is that the diameter of the single display element should not exceed the diameter of the tactile element, in this case 1/20 or 1/30 of an inch. Otherwise you have to mount the display elements in different levels and that would mean it would be very expensive to mount it and to fix it if there is something wrong. Then the price of the single element should not exceed something like, perhaps, ten cents or so. With ten cents you have $8,000 for this tablet. The next problem is that mounting and electrically connecting 80,000 or more elements would be very expensive. Even if you get the electromagnets or something like that for free, it would be very expensive to screw them together and to solder all those wires and so. So that leads us to the idea that there must be in the fabrication process some kind of integration. You all know this from the electronic industry where you can buy one transistor for some cents but if you would buy a million transistors and all these transistors are glued together into a device, it would take about $100,000 or more. But, if you buy 486 processors and I don't know the exact number, but you have about one million transistors, it takes about $500 or so. That's due to the fact that the transistors are not fabricated piece by piece, cut and then mounted together and connected but instead some steps of the fabrication of all the transistors is done in parallel. We should to try to use this principle with the fabrication of this tablet too.
So, before explaining how it will work I should explain what an electroheological fluid is. It's a little bit like this gel. It's not a solution, but I don't know the right word. You have a fluid and solid particles in it - a dispersion - oh that's the right word - a dispersion of solid particles in a fluid. And the fluid without an electric field would have a certain viscosity. There's a certain range of viscosity you can choose. When you apply an electric field to this fluid, there are built up chains of solid particles and fluid particles on the microscopic level, and the viscosity of the fluid is raised. But, it's not like a high viscosity fluid which would mean that with certain pressure you have a certain velocity unequal zero. But with ERFs, you have a threshold of the pressure gradient below that there's no velocity. So, you can build very simple valves by just taking a tube and having two strips of metal inside the tube. If you apply a voltage to the electrodes, if the pressure gradient inside the tube is not too high there's just no velocity of the fluid.
That leads us to the construction of the tablet. We have a board of a nonconductive material. We will use just the same material as for PCB's - that's epoxy resin. This board is the body of the tablet and inside the board you have vertical holes, in the first approach with a diameter of about 0.8 mm. The distance between two display elements will be 1.27 mm resulting in 20 dpi. Inside these holes there are two strips of copper. In the industry of producing PCB's, the process of bringing copper inside the holes of a PCB is common. Our problem is just how to etch a part of this copper inside the holes away. The top of the board is covered by a membrane, an elastic membrane, which is preformed so that you have a little pimple over each channel. And the channels and the space under the board is filled with ER fluid. Of course, there must be a pump for exerting some low pressure or high pressure on the fluid. Then we need collecting lines, connecting all these electrodes together and these collecting lines will be in horizontal and vertical directions.
To control these valves, first we must have such values of the low and high pressure and of the controlling voltage, that with half the maximum voltage, the valves are closed. Then, we can use two voltage sources. One is connected to the vertical collection lines and one to the horizontal collection lines. On each collection line is, of course, an electronic switch. Here on the drawing it's just like a light switch. These switches can bring the voltage of one source to this collection line or can switch the collection line to go low. If you are in the resting state, you have all these lines switched to voltage. If you denominate one voltage of a voltage source with u, you have two times u between each pair of electrodes - all valves are closed.
To present a picture on the tablet, you would normally switch first all lines to go low, that means all valves are open. Then you would exert low pressure on the fluid, all pimples are settled down. Then switch all lines to the voltage so all lines are closed. And then, you can build up the picture line by line or column by column by switching one horizontal collection line to low. Now all valves are closed because one times u is enough to keep the valve closed. If you then switch one vertical line, then the dots in this line are lifted. The input facility is done by just employing the noise, the electrical noise you have by this switching. If the user is touching the tablet with one stylus or with his finger, this noise is coupled capacitively to the finger or to the stylus and you can compute the place where the stylus or where the finger is.
One advantage of this ER tablet is, first, a display element having some inherent memory. For example, for this 80,000 point tablet, you need just about 320 plus 240 switches and the controller to have 560 outputs. That's practical. Then, using the display elements with 1/20 or 1/30 inch is practical. This idea of integration of the display elements in the whole tablet is realized here.
Now, some disadvantages. It takes some seconds to change the whole screen; we expect this time to be in the range of about ten seconds. And the first tablets will be too heavy and too large to be portable. I think it will be a board of about 30 millimeters and with this area it will have a weight of some kilograms; then you have the pump for the fluid and you have a voltage generator. And there's another fact, the voltage is relatively high. You are working with two voltage sources of 1,000 volts. Of course the user will not come in contact with this. It's like a sighted computer user using a normal screen, he sits in front of 20 kilovolts or so. That's no safety problem, but it takes some money to have this voltage sources.
The technical problems are that the solid particles in the fluid tend to settle down. If you have the fluid inside a bottle and let it rest for some weeks, then the solid part is on the bottom and the rest of the fluid is over this. If you exert shearing forces like some pumps do, it will have the same effect, but faster. And, if the electric fields are very strong and inhomogeneous, it will cause this separation too.
I think I can skip this graphical user interface part and perhaps let me say some words about some other work we are doing. As recreation from all this fluid stuff, we are doing something else with conventional technology. The basic question we are concerned with is how to provide to one finger, a picture of a virtual screen. This was caused by the fact that there are some very nice devices dealing with virtual screens. But they are using just the Optacon display, and there are some approaches using braille cells and I find that both are not very good. Braille cells give you a different impression than moving dots when scanning over paper braille, and the Optacon display has the disadvantage of large training time before you can really use it with success. So, we try to develop a nonvibratory device intended to present braille characters whose display geometry looks a little bit like an Optacon display turned 90 degrees. Instead of five columns, we have four rows. I will speak about this at the conference, and have it on the exhibition if I succeed in finding some software bugs in the next day. And, I would be curious to see such a display on a trace puck instead of an Optacon display. I would expect that the user could do the same as with the Optacon, but without training, or nearly without training.
Thank you very much.
Q: Josh from BSI. Earlier I asked about the contacts in the gel and that was with the previous speaker, but it seems like you have an awful lot of stuff going on in the background of your display, so to speak. And, while you talked about an individual element cost, it seems to me like you're not really manufacturing individual elements. It seems like if, as the earlier example went, John poked his pencil through the thing, would you be able to replace the individual pixels , or would you need to replace the entire thing?
If you damage the display can you replace individual units or do you have to replace the whole display?
A: You can just replace layers of this board, for example the membran e plus a three millimeter carrier layer. If someone is jamming it with a hammer or so, you can forget it. There is no way to replace a piece o f it, going through all levels. But, I think normally, just the membran e would be damaged and then you can replace the membrane.
Q: Emerson Foulke, Braille Research Center. Now, if you pump fluid up in a column it pushes against the pimple that's at the top of the channel and the pimple is inverted forming a dot at that location on the display surface. First of all, how high is that dot? Can you tell me something about how large the dot is?
A: It will be about 0.5 mm.
Q: .5 mm, and then when some of the fluid is pumped out of channel again, pressure is reduced and the dot inverts again, I guess, doesn' t it? Why does it invert? Is it just vacuum, or what causes the dot to invert so that it disappears from the display surface?
A: It's not real vacuum, but about half the pressure of the surrounding air.
Q: Less pressure underneath the membrane than on top of it.
A: Yes.
Q: And so it inverts again. Is this membrane with the pimples, is thi s reliable, I mean will they invert and invert again a few million times?
A: We will use plastic material which is very reliable, but we are still carrying out some experiments. We are just working on the machine forming and welding this plastic. Oh, that's -- I don't know how to --as to the name in English, but in German it would be Polyurethan. Audience: Polyurethane
Q: Can somebody remind me, what is the height, I know braille varies all the time, but what's the standard height of braille?
A. About 15/1000.
Q: Greg Vanderheiden. You had talked earlier about one of the important things of an ideal tablet is that you can detect a person's position on the tablet. Do you have any mechanism so that you could, for example, use it with computer enhancements? Do you have any plans or how are you trying to detect where the person is feeling on the tablet.
A: You mean as some guiding aids where objects are?
Q: Do you have any mechanism so that as the person is feeling this image, a computer could figure out where they were?
A: Yes, this is done by looking at the voltage on the body of the use r or on the stylus which the user is using. You can imagine when scanning from column to column or row to row then you will get the greatest voltage when the column or row under the finger or under the stylus is switched. And so, you can do it in a vertical and horizonta l direction. Of course there is some problem if the whole hand would be laying on the tablet, to determine which is the forefinger or so, but I think it is no basic problem.
Q: The other thing you were talking about, the Optacon training time. Most of the training time involved with the Optacon is when you're trying to recognize letters, but if you are only using it to feel shapes, people are usually able to pick it up. They get better with time, but there isn't the training time, as you alluded to in your comments.
A: But I think this vibrotactile display isn't very natural.
Q: It's not any where near as nice as having a stable display as the one's you're discussing here.