Computer graphics processing and selective visual display system – Display peripheral interface input device – Touch panel
Reexamination Certificate
2001-03-08
2004-01-06
Shalwala, Bipin (Department: 2673)
Computer graphics processing and selective visual display system
Display peripheral interface input device
Touch panel
C345S001100, C345S001200, C345S087000, C345S178000, C178S018010, C178S018030, C178S018110, C178S019010, C349S005000, C349S006000, C348S744000
Reexamination Certificate
active
06674426
ABSTRACT:
COMPUTER PROGRAM LISTING APPENDIX AND TO OTHER APPENDICES
Appendix A is “Computer Program Listing Appendix” on a CD (Compact Disk). containing the following 17 text files consisting of 18427730 bytes.
File Name
File Size
Creation Date
aaaAgents.uue
200782
03/01/2001
aaaLib.uue
188119
03/01/2001
blank_bmp.uue
135554
03/05/2001
blank_map.uue
56
03/05/2001
bridge.uue
1930063
03/05/2001
coord_rdr.uue
44300
03/05/2001
EggTimer.uue
45179
03/05/2001
GateKeeper.uue
94929
03/01/2001
Gesture.uue
10643440
03/01/2001
mm_ogi_nl.uue
2031646
03/05/2001
multiparse.uue
1980857
03/05/2001
QuickSet.uue
784537
03/01/2001
rasa_simple.uue
26288
03/01/2001
Speech.uue
191979
03/01/2001
startup.uue
500
03/05/2001
TermEx.uue
39179
03/01/2001
tts.uue
90322
03/05/2001
Appendix B is a copy of forty two page co-pending application No. 60/188,524 filed Mar. 10, 2000.
Appendix C is a copy of twelve page co-pending application No. 60/188,553 filed Mar. 10, 2000.
Appendix D is a copy of a seven page document entitled “Unification-based Multi-modal Parsing” by Michael Johnson
Appendix E is a copy of a seven page document entitled “Creating Tangible Interfaces by Augmenting Physical Objects with multi-modal Language” by David McGee and Philip Cohen.
Appendix F is a copy of a nine page document entitled “Something from Nothing: Augmenting a paper-based work practice via multimodal interaction” by David R. McGee, Philip R. Cohen and Lizhong Wu.
All of the above listed appendices A to F are hereby incorporated herein in their entirety by reference.
FIELD OF THE INVENTION
The present invention relates to computer systems and more particularly to a method and system which provides for interaction between a user and a computer system.
BACKGROUND OF THE INVENTION
The prior art provides a number of methods for computationally augmenting natural, non-digital settings. The term augmenting is herein used to mean extending a physical artifact so that it represents or denotes something more or something new, and thereby gains computation meaning and behavior.
A paper, by Underkoffler, J. and Ishii, H., Urp: “A luminous-tangible Workbench for Urban Planning and Design”, published in the
Proceedings of the ACM Conference on Human Factors in Computing Systems
(CHI '99), May 1999, ACM
Press, 386-393 describes a system called “Urp” that planners use to build models, rulers, clocks, and other physical objects. Objects are tagged by patterns of colored dots, and if a pattern is recognized, a vision system sends Urp the associated object's location. With Urp, augmented objects “behave” as you would expect them to: rulers measure distances, clocks mark time, and so on. The object's physical characteristics and the environment it inhabits govern these expectations
A paper by Wellner, P.,“Interacting with paper on the DigitalDesk” published in the Communications of the ACM, 1993. 36(7): 87-96 describes a system that augments office work by introducing paper into a workstation environment. Through computer vision, users can point at numbers on a real piece of paper, in response to which the system performs optical character recognition and pastes the recognized number into the system's calculator. Similarly, regions of real paper, like a sketch on a napkin, can be cut and pasted into a painting program.
A paper by Ishii, H. and Ullmer, B., “Tangible bits: towards seamless interfaces between people, bits and atoms”, published in the
Proceedings of the ACM Conference on Human Factors in Computing Systems
(CHI '97), March 1997, ACM Press, 234-241 describes a system with a shared whiteboard, uses barcode-tagged cards to hold digital ink. However, the ink can only be retrieved when scanned by a barcode reader connected to a desktop computer.
A paper by Coen, M. H., “Design principles for intelligent environments” published in the
Proceedings of the Conference on Artificial Intelligence
(AAAI '98), July 1998, American Association for Artificial Intelligence, 547-554 describes a system that uses Post-it Notes to activate behaviors in the room. Different colored Post-it Notes are used so that they can be easily distinguished from each other and from the supporting table by a vision system. Ink on each note is used only to remind the user of the action to expect, not as input to the system.
A system call “Passage ” is described in a paper by Streitz, N. A., Geibler, J., and Holmer, T., “Roomware for cooperative buildings: integrated design of architectural spaces and information spaces” published in the
Proceedings of the First International Workshop on Cooperative Buildings: Integrating Information, Organization, and Architecture
(CoBuild '98), February 1998, Springer-Verlag, 4-21 and a technique termed RFID (radio frequency identifier) is described in a paper by Want, R., Fishkin, K. P., Gujar, A., and Harrison, B. L., Bridging physical and virtual worlds with electronic tags, in the
Proceedings of the ACM Conference on Human Factors in Computing Systems
(CHI '99), May 1999, ACM Press, 370-377. These systems interpret the human act of augmenting paper in order to create a digital representation and provide some flexibility in changing that data. Within the Passage concept, meaning can be linked graphically to a physical object whenever that object is placed on a “bridge.” In the initial prototype, the bridge is a scale and recognizes objects based on their weight. With the RFID system, tags are hidden in books, documents, watches, etc. As with Passage, associational augmentations can be formed when the tags are first detected. These systems do not yet support a pre-existing work practice, nor can users learn what information is associated with an object unless the users and the object are adjacent to a bridge or detector. More generally, associational augmentation methods like these and others, such as the use of colored dots, glyphs, or bar codes, fail to present the linked digital information to the user without the assistance of technology.
A paper by Moran, T. P., Saund, E., Melle, W. V., Bryll, R., Gujar, A. U., Fishkin, K. P., and Harrison, B. L., “The ins and outs of collaborative walls: Demonstrating the Collaborage concept”, published in the
Proceedings of the ACM Conference on Human Factors in Computing Systems
(CHI '99), May 15-20, 1999, ACM Press, CHI'99 Extended Abstracts, 192-193T herein called the Collaborage concept. The Collaborage concept characterizes augmented systems consisting of a board and various tagged physical information items. Several such prototypes have been built. One of these prototypes is an In/Out board system with glyph-tagged magnetized photos that can be slid from the Out column to the In column and vice-versa. Within seconds, a vision system recognizes the change in location of the glyph and an In/Out web page is updated to reflect the change in status. If the system were to fail, individuals could still check the physical In/Out board, move their picture from one column to the other, add hand-written annotations, and walk away with up-to-date information. Because objects are augmented using glyphs rather than a natural language, users cannot easily add new digital information to the board. For example, a new employee cannot use any magnet and photograph and expect it to work in the Collaborage system.
SUMMARY OF THE PRESENT INVENTION
The present invention provides a method and system designed to augment, rather than replace, the work habits of its users. These work habits include practices such as drawing on secondary physical documents (e.g. Post-it™ notes) using a symbolic language and placing these secondary physical documents on a primary physical document such as a map. With the present invention the system observes and understands the language used on the primary and secondary physical documents and the system assigns meaning simultaneously to objects in both the physical and virtual worlds. Since the data is preserved in physical form, the physical documents serve as a back-up in the case of electronic system failure. With the present
Cohen Philip R.
McGee David R.
Wu Lizhong
Baniak Pine & Gannon
Kovalick Vincent E.
Oregon Health & Science University
Shalwala Bipin
LandOfFree
Augmenting and not replacing paper based work practice via... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Augmenting and not replacing paper based work practice via..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Augmenting and not replacing paper based work practice via... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3251487