Computer programming using tangible user interface where...

Details Computer programming using tangible user interface where... Computer programming using tangible user interface where...

1997-10-30

2001-01-16

Hafiz, Tariq R. (Department: 2762)

Data processing: software development, installation, and managem

Software program development tool

Translation of code

C717S152000, C717S152000, C717S152000

Reexamination Certificate

active

06175954

ABSTRACT:

BACKGROUND OF THE INVENTION
1. Field of Invention
This invention relates generally to tangible computing, and more particularly to the creation of computer programs by manipulating physical objects that represent program language elements of a program or programming language. The basic functions of the tangible computing system call for observation by a computer of a real, three-dimensional work space, where each physical object can be assigned a program language element, and assignment of attributes to, and/or movement of, each object results in a new program component.
2. Description of Related Art
Human computer interface (HCI) has evolved from the historical human interaction with machine by way of levers and buttons to the present-day keyboard, mouse and graphic display. However, interactions between people and computer programs are generally confined to graphical user interface (GUI) applications on desktop computers. The essence of a GUI is the metaphor—or icon—depicting a real world application in a virtual environment. Thus, GUI solutions take advantage of realistic metaphors.
Ubiquitous computing is an approach to computing where many computers are available throughout a physical environment, while being unnoticed to the user. This type of computing, however, is characterized by exporting a GUI-style interaction metaphor to large and small computer terminals situated in the user's physical environment.
“Augmented reality,” or “computer-augmented environments,” is another approach to computing which attempts to integrate real and computational media. For example, physical documents can be moved with digital documents using a video projection of computer display onto a real desk with physical documents.
“Passive real-world interface props” have also been developed, where a user is given physical props as a mechanism for manipulating three-dimensional models within a traditional computer screen.
With the advent of virtual reality, haptic interfaces were developed to allow a user to move his hand and interact with virtual components within a virtual environment.
A newer type of HCI is called “tangible user interfaces” (TUIs), which augments the physical world by coupling digital information to everyday physical objects and environments. For example, a flat-panel display is a physically instantiated window which allows haptic interaction with three-dimensional digital information bound to physical objects. Thus in TUI, physical objects replace virtual icons, and manipulation of the physical objects represent what is occurring in the virtual world.
Other systems using direct manipulation programs include “ToonTalk,” which permits programs to be constructed by user manipulation of graphic representations of physical objects, all within a virtual environment.
SUMMARY OF THE INVENTION
This invention provides a system which overcomes the limitations of the virtual environment when creating and generating programs, discussed above.
This invention further provides a system that allows computer interaction to create programs and digital artifacts that have embedded programs.
In the tangible computing system of this invention, users manipulate selected physical objects to create and control programs and other digital constructions thus creating unrestricted programs for general purposes.
This invention provides a tangible computing system that observes, understands, and reacts to manipulations of a set of objects in a work space by one or more users. The people, places, and things in the work space, or more generally, the place of interest, must be observable at least to the extent that a specific set of motions on a particular set of objects can be communicated to a computer or a network of computers. The minimum level of understanding required of the system for tangible computing may be expressed as the ability to recognize a tangible language and trigger the appropriate action when the observed input is correct or errors are detected. Actions may include the generation of other digital constructions, for example, compilation, interpretation or translation, generation of control events for an active application, for example, a user interface, and generation of error messages or corrective actions for error conditions.
This invention also provides a tangible programming language used to prepare code from the real representations. The tangible programming language of this invention represents data as a set of tangible data objects that can be picked up from a location under observation, dropped, and used to invoke operations on other data objects by gestures made with or applied to the object. Such gestures include, for example, touching objects together.
The tangible programming language of this invention also provides for external operations, including saving programs in a memory and restoring programs from the memory, and inputting and outputting data values.
In the tangible computing system of this invention, the system observes or senses what is being done to objects in the work space, interprets these actions as meaningful steps in the work at hand, and updates the state of the system as a result of these actions. In the tangible computing system of this invention, users are given better control over their computational needs. The concreteness of the tangible objects permits a group of users to interact in a natural way with the tangible programming language elements.
Other aspects of the invention will become apparent from the following more detailed description, taken in conjunction with the accompanying drawings.


REFERENCES:
patent: 5287502 (1994-02-01), Kaneko
patent: 5398313 (1995-03-01), Kojima et al.
patent: 5436639 (1995-07-01), Arai et al.
patent: 5517663 (1996-05-01), Kahn
patent: 5638060 (1997-06-01), Kataoka et al.
patent: 5704836 (1998-01-01), Norton et al.
patent: 5732227 (1998-03-01), Kuzunuki et al.
patent: 5796386 (1998-08-01), Lipscomb et al.
patent: 5822587 (1998-10-01), McDonald et al.
patent: 5917490 (1999-06-01), Kuzunuki et al.
Hiroshi Ishii and Brygg Ullmer, Tangible Bits: Towards Seamless Interfaces Between People, Bits and Atoms, Conference on Human Factors in Computing Systems (CHI 97), Mar. 22-27, 1997, retrieved from Worldwide Web Site http://tangible.www.media.mit.edu.
Ken Kahn, Seeing Systolic Computations in a Video Game World, IEEE Conference on Visual Language, Proceedings, pp. 95-101; Sep. 1996 (Kahn 1).
Ken Kahn, Seeing Systolic Computations in a Video Game World, paper submitted to IEEE Conference on Visual Languages '96 on Feb. 12, 1996, retrieved from Worldwide Web Site http://www.toontalk.com (Kahn 2).
Various pages retrieved on Aug. 21, 1997 from the Toon Talk Worldwide Web site at http://www.toontalk.com/English/.
Various pages retrieved on Aug. 21, 1997 from the MIT Epistemology & Learning Group World Wide Web Site at http://el.www.media.mit.edu/groups/el/projects.
IBM Dictionary of Computing pp. 607,608,670, Dec. 1994.
Ullmer, B.; Ishii, H.; “The MetaDESK; Models and Prototypes for Tangible User Interfaces”; Proceedings of the 10th Annual Symposium on User Interface Software and Technology; pp. 223-232, Oct. 1997.
Rauterberg, M.; Bichsel, M.; Meier, M.; Fjeld, M.; “A Gesture Based Interaction Technique for a Planning Tool for Construction and Design”; Proceedings of the 6th IEEE International Workshop on Robot and Human Communication; pp. 212-217, Sep. 1997.
Fitzmaurice, G.; Buxton, W.;p “An Empirical Evaluation of Graspable User Interfaces: Towards Specialized, Space-Mutliplexed Input”; Proceedings of the 1997 Conference on Human Factors in Computing Systems: pp. 43-50, Mar. 1997.
Rauterberg, M.; Mauch, T.; Stebler, R.; “The Digital Playing Desk: A Case Study for Augmented Reality”; Proceedings of the 5th IEEE International Workshop on Robot and Human Communication; pp. 410-415, Nov. 1996.
Kahn, K.; “Drawing on Napkins, Video-Game Animation, and Other Ways to Program Computers”; Communications of the ACM; vol. 39, No. 8, pp. 49-59, Aug. 1996.
Suzuki, H.; Kato H.; “Interaction-Level Support for Collaborative Learning:

Affiliated with

Also associated with

Rating

Computer programming using tangible user interface where... does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Computer programming using tangible user interface where..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Computer programming using tangible user interface where... will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-2513590