|
|
|
<span style="font-size: small"> [[Home|<< prev]] </span> <span style="font-size: medium"> | ['''1'''] [[General Design Decisions from Consolidation Process|2]] [[UI Framework|3]] [[Common UIs|4]] | </span> <span style="font-size: small"> [[General Design Decisions from Consolidation Process|next >>]] </span>
|
|
|
|
|
|
|
|
<p align="right"> [{{fullurl:UIwiki|action=pdfbook}} Download UIM wiki pages as a PDF book]</p>
|
|
|
|
|
|
|
|
|
|
|
|
<font size="6">Introduction to the User Interaction Management Expert Group </font>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
__TOC__
|
|
|
|
|
|
|
|
==ID Card==
|
|
|
|
{| class="uaal_project_id_card" border="1" style="cellspacing=0; bordercolor=gray; align=left; valign=top;"
|
|
|
|
! align="left" bgcolor="#DDDDDD" colspan="2" | Expert Group: ''User Interaction Management''
|
|
|
|
|-
|
|
|
|
| class="uaal_project_id_card_highlight" bgcolor="#EEEEEE" | Forge Project Home
|
|
|
|
| http://forge.universaal.org/gf/project/uaal_ui/
|
|
|
|
|-
|
|
|
|
| class="uaal_project_id_card_highlight" bgcolor="#EEEEEE" | SVN Root Repository
|
|
|
|
| http://forge.universaal.org/svn/uaal_ui
|
|
|
|
|-
|
|
|
|
| class="uaal_project_id_card_highlight" bgcolor="#EEEEEE" | Wiki Home
|
|
|
|
| http://forge.universaal.org/wiki/Home
|
|
|
|
|-
|
|
|
|
| class="uaal_project_id_card_highlight" bgcolor="#EEEEEE" | Tracker Home
|
|
|
|
| http://forge.universaal.org/gf/project/uaal_ui/tracker/
|
|
|
|
|-
|
|
|
|
| class="uaal_project_id_card_highlight" bgcolor="#EEEEEE" | Forums
|
|
|
|
| http://forge.universaal.org/gf/project/uaal_ui/forum/
|
|
|
|
|-
|
|
|
|
| class="uaal_project_id_card_highlight" bgcolor="#EEEEEE" | Continuous Integration
|
|
|
|
| http://depot.universaal.org/hudson/job/ui/
|
|
|
|
|-
|
|
|
|
| class="uaal_project_id_card_highlight" bgcolor="#EEEEEE" | Javadoc
|
|
|
|
| http://depot.universaal.org/hudson/job/ui/site/ui.pom/apidocs/index.html
|
|
|
|
|-
|
|
|
|
| class="uaal_project_id_card_highlight" bgcolor="#EEEEEE" | Maven Release Repository
|
|
|
|
| http://depot.universaal.org/maven-repo/releases/org/universAAL/ui/
|
|
|
|
|-
|
|
|
|
| class="uaal_project_id_card_highlight" bgcolor="#EEEEEE" | Maven Snapshot Repository
|
|
|
|
| http://depot.universaal.org/maven-repo/snapshots/org/universAAL/ui/
|
|
|
|
|}
|
|
|
|
|
|
|
|
==Definition==
|
|
|
|
|
|
|
|
This group identifies solutions for explicit interaction between a human user and AAL spaces. This goes beyond the traditional human-computer interaction (HCI) where the interaction is usually assumed to be bound to one single computer and its peripherals. We address the shift of paradigm from HCI to ''HEI'' – human-environment interaction[[#Footnotes|<sup>[1]</sup>]].
|
|
|
|
|
|
|
|
HEI, however, includes both the ''implicit'' and the ''explicit'' interaction. In universAAL, the Service Infrastructure and Context Management expert groups deal with the implicit interaction in AAL spaces, and the User Interaction Management (UIM) expert group is responsible for the explicit interaction which stayed in the shadow of implicit interaction for a long time[[#Footnotes|<sup>[2]</sup>]]. With the proliferation of (multi-)touch sensing, big displays as well as small displays embedded in all possible devices, and new interaction forms supported by special devices (e.g. Nintendo WiiMote and Microsoft Kinect) on one side, and progresses in speech recognition, natural language processing, and gesture recognition on the other side, explicit user interaction in smart environments is becoming more and more important.
|
|
|
|
|
|
|
|
==Scope==
|
|
|
|
|
|
|
|
Before enumerating some of the immediate challenges with which UIM must deal, we would like to summarize here the few terms from ''The universAAL Reference Model for AAL'' (see the universAAL deliverable D1.3 Part II) that are most relevant for us:
|
|
|
|
|
|
|
|
;AAL Space
|
|
|
|
: A smart environment centred on its human users in which a set of embedded networked artefacts, both hardware and software, collectively realize the paradigm of Ambient Intelligence, mainly by providing for context-awareness and personalization, adaptive reactivity, and anticipatory pro-activity. Smart homes and cars are examples of AAL Spaces.
|
|
|
|
;Channel
|
|
|
|
: Smart environments need to bridge between the physical world and the virtual realm with the help of certain devices. Channel denotes the bridging passage provided by such devices between the physical world and the virtual realm. Depending on the kind of channel opened, a channel might be called a sensing channel (provided by sensors), an acting channel (provided by actuators), an input channel (provided by microphones, keyboards, etc.), or an output channel (provided by displays, loudspeakers, etc.). The latter two types of channels might be referred to as I/O Channels.
|
|
|
|
;I/O Device
|
|
|
|
: An abbreviation for input and / or output device. A device that provides an input and / or output channel for facilitating explicit interaction between a smart environment and its human users. Input devices, such as a microphone, a keyboard, or a mouse, can capture an instruction or response that is provided by a human user and represent it in terms of data in the virtual realm. Upon receive of data within the virtual realm that is intended to be presented to human users, output devices, such as displays and loudspeakers, can make it perceivable to the addressed humans.
|
|
|
|
;Multimodal Interaction
|
|
|
|
: The adaptive (context-aware and personalized) utilization of the set of I/O channels available in a smart environment for handling explicit interaction with human users in a natural way. "Multimodal" in this context serves as an abbreviated reference to the potential of performing the interaction using multiple channels in parallel, possibly with a hybrid mix supporting different modalities.
|
|
|
|
|
|
|
|
Based on the above understanding, we see the most urgent challenges of the UIM in
|
|
|
|
* Providing for the independence of applications from the concrete I/O infrastructure (set of concrete I/O channels) available in AAL spaces as the latter might differ in its occurrences considerably. This can be achieved by distinguishing between an "application" layer and a "presentation" layer where the UI Framework sits in between. The components on the imaginary presentation layer can then be called I/O channel managers as opposed to applications that reside on the imaginary application layer.
|
|
|
|
* The separation of the management of the concrete I/O infrastructure in an AAL Space from applications necessitates a modeling of user interaction (UI Model) that is sufficient both for describing user interfaces in a modality-neutral manner and for performing personalized and context-aware adaptation, thereby setting the course for awesome user experience in AAL spaces.
|
|
|
|
* Consequently, there is need for intelligent (personalized and context-aware) brokerage between applications and I/O channel managers based on the above framework for adaptation.
|
|
|
|
* Last but not least, UIM must also introduce a framework for modality fusion when capturing user input from different input channels as well as modality fission when using different output channels for presenting system output to human users.
|
|
|
|
|
|
|
|
==Relationship to the Concrete Architecture==
|
|
|
|
|
|
|
|
As a matter of fact, the universAAL expert groups have been formed during the consolidation and generalization of the system architectures of the so-called input projects towards the universAAL concrete architecture. Therefore, there is often a direct relationship between the expert groups and certain architectural artefacts of universAAL in a way that they influence each other bilaterally, especially when it comes to definitions.
|
|
|
|
Next figure shows universAAL concrete architecture with highlighted UI building blocks:
|
|
|
|
|
|
|
|
[[https://raw.githubusercontent.com/wiki/universAAL/ui/ArchUIHighlight.png|800px]]
|
|
|
|
|
|
|
|
In case of the UIM expert group, the following abstract building blocks from the System Decomposition Model in the universAAL deliverable D1.3 Part IV fall into the scope of the UIM expert group:
|
|
|
|
|
|
|
|
[[https://raw.githubusercontent.com/wiki/universAAL/ui/ArchUIBB.png|400px]]
|
|
|
|
|
|
|
|
* The [[UI Framework|UI Framework]] building block as one of the mandatory space-level managers is supposed to address the challenges related to explicit interaction between AAL spaces and its human users. It must provide for independence of applications from the concrete I/O infrastructure (set of concrete I/O channels) available in AAL spaces as the latter might differ in its occurrences considerably. It must define a framework for capturing user input and presenting system output to human users thereby brokering between applications and the concrete setup in the AAL Space at hand. Two major challenges in this conjunction are the support for multimodality and guaranteeing the adaptation of presentation to the user situation and needs. The existence of a unified UI model might facilitate the provision of such a UI framework.
|
|
|
|
* The [[Common UIs|Common UIs]] building block serves as a placeholder for the pickable space-level managers that belong to the "presentation" layer, namely the I/O channel managers abbreviated here as UI handlers. Assuming that the separation between presentation and application tiers is achieved by the UI Manager above, UI handlers can be added, removed or replaced dynamically and freely; hence, they belong to the configuration-specific set of managers. This kind of managers would be responsible for capturing human input (with or without modality fusion) and presenting system output to humans (with or without modality fission) in an application-independent way.
|
|
|
|
|
|
|
|
==Concerns & Requirements==
|
|
|
|
|
|
|
|
'''Concerns/Needs:''' Easy maintenance; Possibility to adapt applications to wide variety of end user needs; Easy learning/adoption; Ease of use
|
|
|
|
|
|
|
|
The following requirements are related to this expert group. More information about the requirements can be found in [http://www.universaal.org/index.php?option=com_content&view=article&id=66&Itemid=15 D1.2-C]. For a more detailled mapping between requirements and building blocks / software artefacts please refer to the page of the individual building block.
|
|
|
|
|
|
|
|
'''RC3_R1''' ''User Interface framework:'' The Reference Architecture shall define a UI framework that simplifies the development of end user appropriate HCI.
|
|
|
|
|
|
|
|
User interfaces are greatly responsible for the users’ perception of the system and therefore they must be able to adequately fulfill their needs and expectations. They shall be intuitive, easy to learn and configure, easy to maintain and easy to adapt. Their usability shall certainly be a top priority for designers.
|
|
|
|
|
|
|
|
'''Technical User Interaction requirements''':
|
|
|
|
|
|
|
|
'''RC3_TR1''' ''Consistent look and feel:'' The UI framework shall support a consistent look and feel between different interfaces and applications. Consistent colours, shapes and symbols should be used together with natural language and vocabulary familiar to the end user. When designing for assisted persons specific usability standards for the appropriate focus group shall be applied.
|
|
|
|
*Rationale: Users have to feel comfortable using applications. By maintaining the same look and feel across the application screens users do not get confused and their interaction with the system becomes much easier and natural. By using different colours and symbols for different types of messages de-signers can emphasize the semantics of the message even more, but this is up to a designer to decide having in mind end-users’ specific needs and possible impairments. It is however very important to keep the amount of information on the screen at minimum in order not to cognitively overload end-users. Interaction flows should also be as simple and error prone as possible.
|
|
|
|
|
|
|
|
'''RC3_TR2''' ''Multi-modal user interaction:'' The UI framework shall support multi-modal user interaction.
|
|
|
|
*Rationale: Support for different modalities is becoming more and more important in user interaction. With the advancement of technology there are different means of interaction that are being utilised. Thus the UI framework must provide adequate support for the fusion of the input modalities as well as for fission of output modalities. This can result in increased usability as the weakness of one modality can be complemented by the strength of another modality. In order to achieve this a common model for representing the user input and system output has to be defined. In many cases different modalities can be used equivalently, however the applications should remain modality-agnostic and unaware of the technology used for I/O.
|
|
|
|
|
|
|
|
'''RC3_TR3''' ''Input fusion and output fission:'' The UI framework shall be able to combine the input from multiple devices into one input representation (input fusion) and to split one output representation into the output to multiple devices (output fission) in a meaningful way.
|
|
|
|
*Rationale: This requirement directly follows from the support of multi-modal interaction (TR2). Input is captured in different modalities and the meaning of an interaction can only be inferred by taking the input of all modalities into consideration. Accordingly, the output can be presented to the user with different modalities.
|
|
|
|
|
|
|
|
'''RC3_TR4''' ''Adaptability and adaptivity:'' The UI framework shall support adaptability (allowing change in order to meet user requirements and preferences prior to the start of the interaction) and adaptivity (allowing the system to alter its interaction during run-time).
|
|
|
|
*Rationale: Adaptation of user interfaces is one of the most important aspects of the user interaction framework that has to be addressed appropriately. Adaptation itself can be done during design time or during run time. Since users’ preferences and requirements are rarely static not all parameters can be defined prior to the start of a user’s interaction with the system. For this reason it is important to have the possibility to dynamically modify user interfaces. UI elements should be handled as independently as possible in order to be more easily rearranged and personalised. Also, the change from one device to another device (e.g. from a big screen to a small screen) could be part of this adaptation. Please note that the term adaptation in this context only addresses adaptation of how information is presented to the user not a user-dependent selection or adaptation of the information content itself.
|
|
|
|
|
|
|
|
'''RC3_TR5''' ''Independence between application and presentation layer:'' The UI framework shall enable independence between application and presentation layer.
|
|
|
|
*Rationale: The UI framework shall enable abstract representation of user input and system output, regardless of the modality. The support of adaption mechanisms (RC3_TR4) to provide the output according to different output devices and for different impairments or preferences, introduces the challenge to present one and the same output in various ways. To prevent application engineers from implementing all these output techniques for all applications and services in the system, the underlying UI framework must realise this. Thus, the output is actually only an abstract representation, and a clear distinction between application and presentation layer must be ensured.
|
|
|
|
|
|
|
|
'''RC3_TR6''' ''Adequate data presentation:'' The UI framework shall be able to provide adequate data presentation based upon the information received from the user's application (such as the target language).
|
|
|
|
*Rationale: Personalisation of user interfaces to users’ specific needs and preferences is very important. Apart from inferring some aspects user interfaces should also be customizable based on explicitly received instructions from end users.
|
|
|
|
|
|
|
|
'''RC3_TR7''' ''Short and understandable feedback:'' User interfaces shall provide short and understandable feedback to the user. Whenever possible, actions performed by user shall be reversible.
|
|
|
|
*Rationale: For high quality user experience, the user should always know what the system is currently doing. This can be achieved by providing short and understandable feedback. For example, long processing phases between interaction phases should be accompanied with appropriate status messages like progress bars.
|
|
|
|
|
|
|
|
'''RC3_TR8''' ''Predictable and easy to learn UIs:'' User interfaces shall be predictable and easy to learn.
|
|
|
|
*Rationale: Assisted persons might be reluctant to adapt to complex user interfaces therefore interaction techniques should be easy to learn. Predictability also supports user acceptance and can be achieved by providing consistency across various dialogs and services. This way, the user can build on prior knowledge (see also TR1).
|
|
|
|
|
|
|
|
'''RC3_TR9''' ''Highly responsive UIs:'' User interfaces shall be highly responsive.
|
|
|
|
*Rationale: User interfaces shall be highly responsive, giving the user, especially elderly persons, the impression that changes are instantaneous. The UI framework and the underlying communication layer shall support this functionality by guaranteeing certain Quality of Service parameters like latency. The application itself should minimise the amount of unnecessary updates and round trips between IO Handler and the application.
|
|
|
|
|
|
|
|
'''RC3_TR10''' ''Pluggable UIs:'' The UI framework shall support pluggable UIs.
|
|
|
|
*Rationale: As an open system the platform – and, thus, the UI framework – shall support the inclusion of new components at any time without the need to restart the system. The system can then be modified to support new devices or to extend to different users and user groups with different impairments or preferences.
|
|
|
|
|
|
|
|
'''RC3_TR11''' ''UI model based on open standards:'' The UI framework shall support UI model based on open standards.
|
|
|
|
*Rationale: Using open standards facilitate the openness of the platform. Well-defined standards ease development by existing tutorials and tools, especially when the developer is already familiar with the technology. Optimally, these standards have already been used and proven valuable in similar application domains.
|
|
|
|
|
|
|
|
== Footnotes ==
|
|
|
|
{|
|
|
|
|
|- valign="top"
|
|
|
|
|[1]||Encarnação J (2007) ''HEI!: the human environment interaction''. In LNCS 4557: Human Interface and the Management of Information. Springer, Berlin / Heidelberg, 623-631.
|
|
|
|
|- valign="top"
|
|
|
|
|[2]||To satisfy the core requirement that intelligent environments should automatically react to the contextual changes as far as reasonable, researchers were mostly focused on capturing changes in the context and providing mechanisms for automatic reaction upon such changes.
|
|
|
|
|}
|
|
|
|
|
|
|
|
== Chapters ==
|
|
|
|
|
|
|
|
*[[Home|1 - Introduction to the User Interaction management Expert Group]]
|
|
|
|
*[[General Design Decisions from Consolidation Process|2 - General Design Decisions from Consolidation Process]]
|
|
|
|
*[[UI Framework|3 - The universAAL UI Framework]]
|
|
|
|
*[[Common UIs|4 - Example UI Handlers in universAAL]] |
