... | ... | @@ -300,6 +300,122 @@ The project board of universAAL is considering high-level interactions with open |
|
|
== Adoption of technologies ==
|
|
|
According to the design decisions and the analysis of the input projects, it was decided to use the PERSONA approach as base in UIM.
|
|
|
|
|
|
== UI Framework ==
|
|
|
|
|
|
===Black box description===
|
|
|
The UI Framework as a building block in the universAAL reference architecture is the only abstract building block in the '''core''' of the universAAL runtime support for uSpaces that is related to the explicit interaction between human users and uSpaces. Hence, this building block must cover all the requirements that are considered as general and must realize shared mechanisms needed for facilitating the development of I/O channel managers and applications.
|
|
|
|
|
|
<u>Note:</u> before going further it is important to mention that there is a special chapter on how to use UI Framework in the [https://github.com/universAAL/platform/wiki/UI-Bus-%28Quick%29 Reference Documentation].
|
|
|
|
|
|
The selected PERSONA solution covered most of these requirements by building on top of the middleware and the framework for supporting context-awareness (which is also adopted by the Context Expert Group in universAAL).
|
|
|
|
|
|
[[UI_Fw.png|400px|center]]
|
|
|
|
|
|
===Design decisions===
|
|
|
|
|
|
The following figure illustrates separation of application and presentation layers and the overall workflow of the UIM framework from the application inception of the abstract description of the user interface till the delivery of this output to the end user and then collecting of possible input from the user and delivering it back to the application.
|
|
|
|
|
|
[[UIMfw_overall.png|700px|center|UIM Fw workflow]]
|
|
|
|
|
|
Note: content in this section will be added in next version of the deliverable
|
|
|
|
|
|
====Modality- and layout-neutral description of dialogs====
|
|
|
|
|
|
Dialog descriptions in universAAL enable composing abstract representation of the user interfaces preserving modality and layout neutrality. They are based on [http://www.w3.org/MarkUp/Forms/ XForms] which is an XML format and W3C specification of a data processing model for XML data and user interface(s), such as web forms.
|
|
|
In universAAL own java implementation of XForms specification is used for RDF-based data model in order not to add additional complexity with respect to the decisions made in middleware Expert group.
|
|
|
Apart from being based on open standard it enables separation of application and presentation layers which are two identified requirements for this Expert Group.
|
|
|
'''Dialog''' in universAAL UIM Framework is a „Form” containing both form data and form controls.
|
|
|
Next figure shows 3 groups in a "Form"
|
|
|
[[form.png|600px|center]]
|
|
|
In universAAL we distinguish 4 types of a dialog that are as follows
|
|
|
[[dialogs.png|600px|center]]
|
|
|
|
|
|
====Adaptation parameters====
|
|
|
|
|
|
<u>Application responsibility</u>
|
|
|
|
|
|
Applications just need to provide the dialog's Form object along with the addressed user, the priority of the dialog, and the language and privacy level of the content.
|
|
|
|
|
|
Optionally Applications may provide hits of how to represent components in certain modalities, to help the Handlers make better decisions, yet these recommendations may or may not be applied by the handlers. A specially delicate modality is the graphical modality so recommendations on layout, size, alignment can be added to each element.
|
|
|
|
|
|
<u>Framework responsibility</u>
|
|
|
|
|
|
Framework responsibility is to add the current adaptation parameters (specified within UI Preferences subprofile of the User) to the call (by asking the Dialog Manager) before selecting the appropriate UI handler and forwarding the call to it.
|
|
|
This call can be local or remote depending on the fact if the called UI Handler is residing on the same node as the application or not. This process is completely seamless to the application which does not have to anything after forwarding the UIRequest to the UI Bus expect.
|
|
|
|
|
|
<u>The notion of an UI Request</u>
|
|
|
|
|
|
When considering UI Framework from application point of view, the application is sending the request to the UI Fw that some dialog (screen if we talk about GUI) should be presented to the end user. When user is finished with inputing the data that the application asked for he submits the dialog and the response carrying his input is wrapped within UIResponse returning to the calling application.
|
|
|
|
|
|
[[UIReq1.png|400px|center]]
|
|
|
|
|
|
The message carrying the abstract representation of the user interface is traveling as a payload within UIRequest. Apart from the abstract representation of the user interface the UIRequest is carrying some additional information send by the application, but also acts as a container for adding UI Preference or adaptation parameters that are added by the Dialog Manager and user by the UI Bus (more concretely UI Bus Strategy) for determining the UI Handler that should be used for interaction with the end user (defined by the application).
|
|
|
|
|
|
[[UIRequest.png|400px|center]]
|
|
|
|
|
|
<u>The notion of an UI Handler Profile</u>
|
|
|
|
|
|
UI Handlers provide at the registration time info about themselves. This way the UI Framework knows which UI Handler has which capabilities and can use this information when deciding best UI Handler for delivering UIRequest (coming from the applications) to designated user.
|
|
|
|
|
|
[[UIHandlerProfile.png|400px|center]]
|
|
|
|
|
|
====User-centred dialog management====
|
|
|
|
|
|
<u>Detailed design decisions</u>
|
|
|
|
|
|
Personas (as fictional and super-typical characterization of user created to represent user group) have become an increasingly popular way to customize, incorporate and share the research about users [[#Footnotes|<sup>[1]</sup>]].
|
|
|
Note: more about Personas can also be found here: [http://en.wikipedia.org/wiki/Persona_%28marketing%29 Persona on Wikipedia]
|
|
|
|
|
|
In universAAL deliverable D1.1. (Reference Use Cases for AAL) that can be found here: [http://www.universaal.org/index.php?option=com_content&view=article&id=66&Itemid=15 universAAL Deliverables] Persona ''Peter'' is used to address Assisted Person, and Persona ''Jack'' is used to denote his son.
|
|
|
A short excerpt of ''Scenario from the point of view of Elderly person'' from D1.1:
|
|
|
<pre>Peter, 77 is widower and since his wife died one year ago his health has deteriorated a lot. His son
|
|
|
Jack is worried because his father does not follow a healthy diet after the death of his wife. Besides,
|
|
|
he has hypertension and cholesterol, which are both a common illness at this age.</pre>
|
|
|
|
|
|
During development phase Persona '''Saied''' will be used within UIM EG components to denote Assisted Person where they will be associated with specific abilities/disabilities, impairments and other envisioned characteristics.
|
|
|
|
|
|
Dialog management is very important in controlling interaction with users since it represents the center of mediating messages between components used for user interaction. A module for dialog management controls output generation
|
|
|
In voice based systems dialog managements comes after natural language understanding phase when meaning is extracted and actions/prompts follow.
|
|
|
|
|
|
In general a dialog consists of a series of information exchange events between a user and a system. The initiative of the dialog progression falls into one of the following:
|
|
|
* system initiative
|
|
|
* user initiative
|
|
|
* mixed initiative
|
|
|
|
|
|
In universAAL most initiatives fall into user-initiative stage, buth nevertheless all stages are possible depending on the different scenarios and applications residing in the system.
|
|
|
|
|
|
====Refinement in terms of software artefacts====
|
|
|
Following software artefacts comprising this building block are:
|
|
|
* UI Bus
|
|
|
* Dialog Manager
|
|
|
|
|
|
==Common UIs==
|
|
|
Common UIs building block is a collection of UI handlers and Supporting UI services.
|
|
|
|
|
|
[[Common_UIs_building_block.png|550px|center]]
|
|
|
|
|
|
Components comprising this building block are meant to be numerous since they represent the set of available plugg-in UI Handlers.
|
|
|
In scope of universAAL only development of example UI handlers is planned. However later development of more Handlers will enrich the overall offer when deploying the UI Fw in the homes of the end users since they will have a broader range of UI Handlers available to choose from.
|
|
|
|
|
|
A UI Handler is a component which accepts UI Requests coming from the applications and adds the user's input as a payload that goes into UI Responses which are forwarded to the applications.
|
|
|
|
|
|
<u>Simple UI Handler</u>
|
|
|
Simple UI Handler is the most basic form of UI handler that acts as a subscriber of the UI Bus and accepts the UI Requests that it can adequately present to the user. In this respect it publishes its capabilities in terms of which specific user impairments and needs it addresses. It additionally uses adaptation parameters coming from the user interaction profile data so that it enriches and personalizes the final outcome according to the user preferences. User interaction preferences are firstly defined by an Assisted Person when initializing the system and later used by Dialog Manager in the sense they are added to the UI Request coming from the application layer toward the presentation layer.
|
|
|
|
|
|
[[simple_handler.png|thumb|center|350px|Simple UI Handler concept.]]
|
|
|
|
|
|
<u>Context Aware Handler</u>
|
|
|
A handler which (additionally in respect to the Simple UI Handler) adapts to the context is called a context aware handler. The means by which this handler gets context information is the context bus. So within its components it needs a context subscriber as well as a set of rules to adapt the configuration of the translator package.
|
|
|
|
|
|
[[contex_aware_handler.png|thumb|center|450px|Context aware UI Handler concept.]]
|
|
|
|
|
|
Example UI Handlers that are planned in scope of universAAL are:
|
|
|
* [[UI Handler GUI Swing|GUI based UI Handler]]
|
|
|
* [[UI Handler WEB|Web UI Handler]]
|
|
|
* [[UI Handler SMS|SMS UI Handler]]
|
|
|
* [[UI Handler Kinect|Gesture UI Handler]]
|
|
|
|
|
|
|
|
|
== Footnotes ==
|
|
|
{|
|
|
|
|- valign="top"
|
... | ... | |