Title

Interactive Applications and Games

Organizers

Florian Metzger, Matthias Hirth, and Steven Schmidt

Motivation and objectives

For many years, users mainly consumed multimedia content passively, e.g., by watching videos or listening to audio content. Therefore, even today most sophisticated user models for audio-visual (A/V) material only consider technical parameters that influence the quality of the content or the playback behavior. However, multimedia systems are rapidly changing, and users develop from passive consumers to active participants.

This active participation is most evident when it comes to gaming and virtual reality (VR) applications. Here, the users’ interactions directly control the A/V stimuli created by the system. The user involvement imposes new challenges to system designers and consequently also on the user models describing those systems. The impact of engagement on a user perceived quality and consequences of novelty effects are not yet well investigated. Furthermore, new relevant QoE aspects evolved while issues like cybersickness have to be avoided. Additionally, technical parameters such as end-to-end delay or methods to control interactive systems need to be considered. However, the interactivity of applications is not only a phenomenon in the entertainment area. More and more commercial applications of VR become mature in the context of medical education, telemedicine, or construction processes.

Going one step further, also the interactivity of Web-based applications creates new challenges that have not been considered in the past. Recent advances in web-technologies enable a plethora of features that can now easily be realized in Web browsers. This fosters the development of rich Web applications replacing traditional applications on the end-user device, starting from text processor to business applications. While this transition makes it easier for the developer to support multiple end-devices and increases the availability to the application for the end-users, new technical aspects like the end-to-end delay of the network connection now also play an essential role in the user’s perception of the application quality.

To address the challenges arising for the interactivity of games, VR applications, and Web-based applications, existing test QoE test methodologies need to be refined, and entirely new techniques need to be developed. While passive A/V tests are mainly performed with pre-recorded stimuli, it remains unclear to which extent this is possible for the evaluation of the perception in interactive applications. On the contrary, it is still an open question, how to design and implement realistic emulations or testbeds for interactive applications that enable user studies at a large scale or collaborative scenarios involving multiple users. Further, existing work already indicates that a user’s familiarity with an interactive application plays an important role in the perception of impairments, however, this is not yet considered in any existing QoE model. From a conception perspective, a clear definition of ‘interactivity’ is required and categorization of interactive applications to enable structured research efforts.

These kinds of novel approaches to (subjective as well as objective) user-centric quality metrics and evaluation methods for interactive applications are the core topics we want to explore in this special session.

Topics of interest

Topics include, but are not limited to:

Interactive applications in general
- Application-specific metrics and frameworks for interactivity
  - Objective metrics (QoS)
  - Derived metrics (KPI, KQI)
  - Mapping to QoE and UX
- Design and repeatability of QoE experiments for interactive applications
- Methods for enabling crowdsourcing-based evaluations for interactive
- systems
- Assessment and modeling of Quality of user experience (QUX) of
- interactive applications
- Inter-/intra-application definitions of interactivity (qualitative and
- quantitative)
- Interactivity determination and quantification
Gaming Quality of Experience
- Metric-based categorization of video games
- Methodologies and challenges of in-situ studies
- Repeatability, proficiency, learning curve and training process of test participants
VR & AR applications
- Interactivity requirements for immersive VR/AR applications
- Quality of interaction and interactivity metrics of haptic, wearable or multi-modal user interfaces
- Metrics and test methodologies for interactive environments