Personal Interaction Panel Two-Handed Interface for Augmented Reality Applications Project duration: 1996-1999 Contact: Zsolt Szalavári
Description	The Personal Interaction Panel (PIP) is a new interaction paradigm that uses a simple two-handed hardware set-up to enable natural and easy-to-learn control over 3D applications. Users can interact with the mixture of 2D and 3D interface elements on the PIP, based on the metaphor of "2D in 3D" rather than "2D and 3D".
Application	The PIP can be used to control virtually any type of augmented reality application. The hardware set-up consisting of a notebook sized panel and a pen is nearly modeless. Appearace and functionality is augmented using three dimensional computer graphics, so the interface is free to incorporate any design issues.
Problems	Immersive 3D applications require not only high display quality but also easy-to-use support for interaction. Many approaches try to introduce new metaphors, which people from different educational backgrounds may find unnatural and hindering. Highly specialized interfaces may support one application excellently but fail on performing general tasks and vice versa.
Approach	To overcome drawbacks of earlier approaches, our PIP interface relies on the everyday tool handling experience of working with pen and paper. We interpret the sheet of paper as a 2D surface in a 3D environment, which can hold information on it's surface. Using 3D computer graphics we extended this observation and create a three dimensional two-handed interface, that can hold traditional GUI elements, as well as widgets and application specific control elements. Our work concentrates on the general design of the PIP as an interface for immersive interaction and on application specific "sheets", which support the needs of scientific visualization in "Studierstube", navigation in virtual environments and manipualtion of objects.
Publications	The Personal Interaction Panel - A Two-Handed Interface for Augmented Reality Zsolt Szalavári, Michael Gervautz appeared in Computer Graphics Forum, 16, 3 (Proceedings of EUROGRAPHICS'97, Budapest, Hungary), pp. 335-346, September 1997 Using the Personal Interaction Panel for 3D Interaction Zsolt Szalavári, Michael Gervautz appeared in the Proceedings of the Conference on Latest Results in Information Technology, Budapest, Hungary, pp. 3-6, May 1997 Das "Personal Interaction Panel" - die Schiefertafel der Jahrtausendwende (in German) Zsolt Szalavári appeared in the Proceedings of the Workshop "Vom Bildschirm zum Handrad", October 1997

	Snapshots showing results
	The hardware setup shown in the left image is composed out of a lightweigth (wooden) panel and an in-house pen. Note the Polhemus Fastrak receiver mounted on the pen!
	Following pictures show only renderings projected over real environment using i-Glasses! see-trough head-mounted displays. Left image shows browsing objects from a palette and drag&drop them to and from the surrounding scene.
	Objects not needed can be thrown into an automatic augmented trashcan standing in the room.
	The Personal Interaction Panel can be used as a virtual photo camera to make snapshots from the virtual scene. To make a 3D image the user simply places objects into the scene, arranges them and shoots a photo from the desired camera position.
	Navigating virtual landscapes becomes an easy task by showing a corresponding map on the surface of the panel. The arrows provided on the PIP allow the scrolling of the landscape beneath the user.
	This montage shows what the user sees using our system. The virtual imagery is presented to him as an overlay to the real environment. The image shows a sheet where a model of a dynamical system is beeing modified and recalculated depending on the selected parameters.
	The virtual part of "Studierstube" shows a representation of a HMD where the users head is positioned. This view can be used for remote observation of the system, while users work in the augmented environment.

	Quicktime videos
	All clips have been recorded in real time on a Silcon Graphics O2 workstation. The O2 was also used to mix a real video source with computer generated images for making these clips. This sequence shows browsing objects from a palette and drag&drop them to and from the surrounding scene. (6.6 MB Quicktime)
	The Personal Interaction Panel can be used as a virtual photo camera to make snapshots from the virtual scene. To make a 3D image the user simply places objects into the scene, arranges them and shoots a photo from the desired camera position. (2.3 MB Quicktime)
	This sequence shows the investigation of a dynamical system. Scaling is done in a local loop, using existing geometry from the model. Parameter modification of the system (width of the streamsurface needs the recalculation of the model in a wide loop. (4.6 MB Quicktime)
	Why not doing nonsense with this interface ... ? (2.0 MB Quicktime)
	This is the last part of the video presented at EUROGRAPHICS '97. As many people use our environment it is important to clean up after yourself! (4.4 MB Quicktime)