SEARIS 2008 1th Workshop on IEEE VR

ABSTRACT

The Software Engineering and Architectures for Realtime Interactive SystemsWorkshop at IEEE VR 2008 gathered specialists in the fields of realtime simulation, computer games, Virtual, Augmented, Virtualized, and in general Mixed Realities, around the central topic of application development in research and production. We decided to use the name Realtime Interactive Systems (RIS) in order to encompass all these types of applications, since we felt their implementations share common principles, and inspiring solutions in one field could benefit the others. The goal of this workshop was to analyze and structure the current state-of-the-art in software engineering and architectures as required for Realtime Interactive Systems. We wanted this workshop to point out paradigms, concepts, methods, and techniques that support technical developments required in our field. A unified presentation of systems will allow us to support research and development in a more efficient way, and will provide a valuable source of information for future developments. This Workshop is our first attempt to address the complex issue of RIS development and to summarize the work our community is doing. The full proceedings are available online at http://www.searis.net

1 STATE OF THE ART

Several conceptual and technical approaches have been developed and utilized in the field of RIS. There are multiple environments with inspiring solutions, but usually they are based on common principles and techniques which are often implemented from scratch. Loosely related workshops have been organized, in areas such as Standards for Immersive VR [2], Specification, Authoring, and Adaptation of Mixed Reality Applications [1], or more recently on Next Generation User Interfaces [3]. However, we want to concentrate on the insights related to the supporting tools used in the implementation of RIS, and the overall architecture of such solutions. This seems to us a daunting task: There are myriads of systems and supporting libraries in each one of the fields inside the RIS concept, each one with differences in approaches, coverage of the field, documentation, and lively community. Just to emphasize this point, the following is an incomplete list by area of interest inside our definition of RIS, with no particular order:

Virtual Reality: VRJuggler, Virtools, OpenSG, OSG, DIVERSE, dVS/dVISE, Performer, Alice, CAVELib, Panda3D, Inventor, MASSIVE 1/2, Virtual Design 2, PowerVR, Avango, LightSpace, StudierStube, MetaVR, VRML, InTml, Lightning, FlowVR, OpenMASK, VISTA, MORGAN, Instant Reality, …
Augmented Reality: ARToolkit, StudierStube Tracker, ARTag, …
Game Development: Torque, CrystalSpace, OGRE, Quake, Unreal2, …
Realtime Simulation: PhysX, RAMSIS, …
Visualization: VTK, ITK, AVS, …
Supporting libraries: OpenGL, DirectX, Chromium, Equalizer, Java3D, COIN, JoGL, OpenRT, VRPN, JInput, …
3D Communities: Second Life, Project Wonderland, …
3D Applications: Google Earth, Google SketchUp, Paraview 3.2, …
These environments are used either as RIS standalone or integrated solutions, so results and scopes widely vary. We would like to extract from this universe useful common practices and guidelines for RIS developers, as well as point out areas of future research. For example, contributions and discussions in the workshop indicate that there are interesting directions of research on the following aspects:
Clustering: Common hardware implementations include clusters, and several active projects are including ways to use these resources.
Multi-rate systems: In order to accelerate user’s feedback, some researchers are looking into ways to create visual output as a composition of a high frame rate interactive part and a slower frame rate background, probably at higher rendering quality.
Architecture: Common approaches use architectures based on dataflows or layers, but there are interesting abstraction mechanisms or novel non-functional requirements which may require novel solutions.
Mobile solutions: Restrictions in some hardware platforms such as cell phones require special solutions and tools. The following sections describe important issues from the accepted contributions, and some of our own conclusions.

2 CONTRIBUTIONS

We divided contributions from authors into four sections:

Systems. These presentations concentrated on an overview of current development environments for RIS.
Abstraction issues. Here we concentrated contributions on the issues of reusable VR platforms and alternatives to scene graphs.
Special issues. This section concentrated novel concepts such as RIS in mobile devices, multi–rate systems, and adaptivity of Mixed Reality Applications.
Semantic and Dynamic Modeling. These contributions describe ways to enrich RIS with semantic and dynamic information.
The following subsections present an overview of the contributions.

2.1 Systems

As we have seen, it is possible to mention more than 50 more or less active tools and support libraries for the development of VR and AR applications, without mentioning the hundreds of game engines and supporting tools included in the RIS development spectrum. As a starting point of the analysis of the RIS development field, we got here contributions that described the common principles and novel ideas in six of these systems: InTml, Lightning, FlowVR, OpenMASK, VISTA, and Morgan.

InTml, presented by Pablo Figueroa, is an academic development that started around 2000 that features a dataflow based architecture, a platform independent language for describing applications and components, a formal description in Z, and two implementations in C++ and Java. One of its strengths is to provide a platform independent description of an application, with clear separations between devices, behavior, and content. This separation allows developers to adapt an application to a particular hardware platform at the architectural level. Future work is concentrated on a library of platform–independent components and on improving tools and documentation for novel users.

Lightning, presented by Matthias Bues, shows an active VR environment that has evolved since 1996 inside Fraunhofer IAO. Applications are created as dataflows of nodes, inspired in the VRML execution model and an abstraction of a scene graph, in which only interesting nodes are seen. The system has also included a scripting interface, a procedural alternative to the dataflow, and a cluster solution. As future work, there are ideas on how to extend the system for multiple users and asynchronous execution.

One of the younger systems in this list is FlowVR, presented by Bruno Raffin. It is a system specially designed to run complex simulations on a cluster. An application is defined as a set of concurrent modules connected in a FlowVR network. There are tools for viewing such a network and traces of execution. One interesting demo allows users to interact with a simulation of a flow or a deformable cloth by video input of their hand movements.

OpenMASK, presented by Bruno Arnaldi, is a system that started its development in 1994. It is a system organized in two layers, a kernel that provides an abstraction of the runtime environment, and the simulated objects layer, which allows a dataflow of distributed objects. There are separated services for user’s feedback and interaction, and a mechanism for adding new behavior to existing components.

Ingo Assenmacher presented VISTA, a C++ based development environment that was originally developed for the WorldToolkit, around 1998. It is designed as a simple library that gives programmers the opportunity to decide issues such as distribution, interaction, or execution model. It defines a set of interfaces that hide interchangeable implementations, for example geometry or window management services.

Jan Ohlenburg presented MORGAN. This novel framework is designed to run in a wide number of platforms, with few changes in an application’s source code. It has an implementation targeted to PDAs, MORGAN Light, which allows programs to efficiently run in such platforms.

2.2 Abstraction Issues

Anthony Steed presented some lessons and insights of 15 years of developments in his lab over several systems and in particular over DIVE. Some of his conclusions from his experience are:

Event systems are easier to understand and program than dataflow systems. It is quite natural for programmers to plug components by subscribing to event providers, since it is a common concept in 2D systems. Interaction details about each hardware platform should be described apart from an application. Inside an application, they have used three abstractions — Vehicle, User, Body — which present a uniform set of events to the application. Portability could be achieved by this mechanism, since details about mapping devices to such events are hidden from applications. Meta-data about the virtual world and its semantics should be separated from the application code. DIVE has a separate file that includes all geometry files and adds the required semantics. Some DIVE characteristics were also described: local and global events in a distributed setup, high-level events for hiding groups of events, and scene state that is visible to just some users.

Florian Mannußpresented an abstraction for entities, which gives structure to his system. Although a scene graph is seen as an important technical solution in any VR solution, it is shown here that it may be too low level as the main structure in an application. They propose a system based on entities, objects in a VR world with a set of modifiable characteristics. Around entities they propose a system with input devices, attendees, and manipulators.

2.3 Special Issues

We classified as special issues the following: AR applications on mobile phones, multi-frame rate systems, configurability and adaptivity of MR applications.

Dieter Schmalstieg addressed the particular issues involved in AR application development in mobile phones. He presented the Studierstube ES platform, which addresses specific characteristics in this platform: small memory footprint for the runtime environment, integration of existing media formats, scripting capabilities, multi–user interaction based on a blackboard in a server, and a camera based 6DOF tracker based on fixed point operations.

Jan Springer presented a short overview of his multi-frame rate system, which has a full description in a paper at VR 2008. Their basic idea is to separately render background objects from the elements in the interface that are part of the user’s feedback. This method guaratees system responsiveness, although it creates issues related to temporal artifacts and output composition.

Finally, Irma Lindt reflected on the several approaches for adapting an application to his environment, such as device abstraction layers, user interface description languages, MR toolkits and frameworks, MR authoring systems, and adaptive MR applications. They argue that device abstraction layers provide a common ground that hides hardware details, although it can be too low level as a configuration provider, and that MR toolkits should provide and enforce mechanisms for adaptation and configuration.

2.4 Semantic and Dynamic Modeling

These contributions addressed ways to explicitly describe semantic information, in order to provide better support for behavior in complex RIS applications. These authors are interested in the semantics associated to dynamic interaction, temporal synchronization, spatial relationships between objects, and information about the application’s state.

Kristopher Blom makes explicit the information about the expected changes in objects in a virtual world, in order to facilitate the creation of RIS. Their concept is to create applications that explicitly take into account time, but leaving to the platform the implementation details, such as time progression and resolution.

Marc Latoschik presents a way to integrate VR systems, actorbased systems, and a knowledge base. In such a system, it is possible to take advantage of the inherent parallelism of the actor paradigm, as well as guaranteeing a consistent world state by means of the semantic information that each actor has about the system’s execution.

Lode Vanacken describes a system that takes into account semantic relations between entities. A system is modeled in terms of states, events, tasks, labels, and transitions. They add Concepts as an element in their designs, in order to allow tasks to make decisions in terms of such values. Concepts could also be related, in order to take into account more complex interactions.

Finally Gerwin de Haan presented a system based on state machines and a scripting language in order to facilitate RIS development.

3 DISCUSSION

SEARIS was a very lively and enriching experience for us, with a wide variety of contributions and points of view. We would like to emphasize the following ideas from papers and presentations at the Workshop.

There are pervasive solutions in the area of RIS, such as scene graphs, device abstraction layers, dataflow architectures, and abstractions for the integration of several libraries. Several solutions add layers on top of scene graphs, in order to hide too many details about geometry or to add semantic information. Other modules in a RIS solution may interact with this high level representation, instead of with the scene graph. Scripting languages may help for rapid prototyping of RIS, although there are some evidences that software evolution of such solutions is rather difficult. The architecture of several systems is based on dynamically loadable components. This mechanism seems to be a powerful way to interchange components in a solution, although it also make more complex the deployment of RIS solutions. Although dataflow architectures are present in several RIS frameworks, it is also stated that in some cases they make development more difficult. It has been proposed a combination of both dataflow and procedural paradigms, although the semantics of a system with parts in both paradigms is still not clear. Mobile phones and PDA may require specific development platforms, due to their own restrictions and features. Some solutions have been developed, which implement a subset of capabilities of full-fledged ones, so applications are more or less portable between platforms. There are several approaches to address application adaptability to its environment (i.e. user characteristics, hardware platform, environment), although there are not clear solutions so far. Semantic information could allow the creation of more complex and generic behavior. It is important to keep such information apart from an application’s code.

4 CONCLUSION

SEARIS was our first attempt to unite a community interested in Software Engineering issues related to the development of RIS, and we are grateful for the acceptance we received at IEEE VR 2008. This is only the beginning, and there are several directions of future work. A more comprehensive description of systems and common ideas could be created, in order to state better our current state of the practice. More novel ideas should also be stated and documented, specially from the area of game development, which was not well represented. Finally, more gatherings are planned in the near future, so we can increase the amount of knowledge and people involved in these issues.

ACKNOWLEDGEMENTS

Thanks to all participants of SEARIS08, in particular to Anthony Steed for his detailed list of related tools and environments.

REFERENCES

[1] R. Dachselt, P. Figueroa, I. Lindt and W. Broll, editors. Mixed Reality User Interfaces: Specification, Authoring, Adapt ation (MRUI’07). Shaker Verlag, 2007.

[2] N. F. Polys, D. Brutzman, A. Steed, and J. Behr. Future standards for immersive VR: Report on the ieee virtual reality 2007 workshop. Computer Graphics and Applications, IEEE, 28(2):94–99, March-April 2008.

[3] O. Shaer, R. Jacob, M. Green, and K. Luyten. User interface description languages for next generation user interfaces. CHI 2008 Workshop. http://www.eecs.tufts.edu/ oshaer/workshop/, 2008

Description:

IEEE VR 2008 WORKSHOP “SEARIS”

SEARIS - Software Engineering and Architectures for Realtime Interactive Systems Several conceptual and technical approaches have been developed and utilized in the field of Realtime Interactive Systems (RIS). Virtual, Augmented, Virtualized, in general Mixed Realities, as well as realtime simulation and computer games led to manifold inspiring solutions for RIS developments in research and production. But they also implement very common principles in the field. However, it is often difficult to identify and separate both novel results and well known solutions in any new system. The goal of this workshop is to analyze and structure the current state-of-the-art in software engineering and architectures as required for Realtime Interactive Systems. We want this workshop to point out paradigms, concepts, methods, and techniques that support technical developments required in our field. A unified presentation of systems will allow us to support research and development in a more efficient way, and will provide a valuable source of information for future developments.

In preceding years, several researchers have organized related workshops and loosely organized themselves as an unofficial interest group during past events and beyond. Their goal is to create a forum where researchers from all directions in the broad field of Virtual and Augmented Reality can contribute and debate their respective technical approaches. The workshop will provide a face-to-face opportunity to further support this emerging project.

ORGANIZATION

Conference website: www.searis.org (online soon) Submission of

a position/experience paper on a specific system or
a theoretical architectural framework/approach.

First Half day:

Introduction by Organizers
Presentation of the submissions in tecnote style (10 min. each)

Second Half day:

Panel/plenum discussion debating the work presented
Identifying general structures, directions, and fields of applications
Formulation of common requirements and possible paradigms, concepts, methods and techniques to solve them.

Summary

Combined presentation of these group discussions
Conclusion and Future Activities
The target audience includes researchers and developers from VR/AR as well as from technically close fields like mixed reality, ambient/pervasive computing and - of course - the computer games community.

The results of the discussions and the workshop will be collected and published after the workshop.

SUBMISSION AND IMPORTANT DATES

Submissions should follow the IEEE Computer Society format, and should be between 3 and 4 pages long. Please include all author and contact information in your submission. Additional materials (text, video, pictures, XML files etc.) are welcomed to support the submission. Unless clearly indicated otherwise, submission of a workshop contribution and associated materials implies permission for a publication on the Web. The submissions will be printed and published by Shaker.

All papers should be between 3-4 pages and must be submitted electronically using an online conference system. Details will be found at www.searis.org.

Submission deadline: January 26th, 2008
Notification of acceptance: February 7th, 2008
Camera-ready submission: February 9th, 2008

ORGANIZERS

Marc Erich Latoschik, AI & VR Lab, Bielefeld University, Bielefeld, Germany
Dirk Reiners, University of Louisiana, Lafayette, USA
Roland Blach, CC Virtual Environments Fraunhofer IAO Stuttgart, Germany
Pablo Alejandro Figueroa Forero, Universidad de los Andes Bogota, Colombia
Raimund Dachselt, Otto-von-Guericke-Universität Magdeburg, Germany