4.8 Expand Dialog Description (SCXML)

Expand Dialog Description (SCXML)

As described in section Interaction Manager (IM), MMIR defines the dialog flow of an application by specifying transitions between application states using SCXML. The description file is located at www/config/statedef/dialogDescriptionSCXML.xml. This description is automatically loaded and utilized via the DialogManager: The dialog engine handles events and dispatches them to controllers' actions and/or renders controllers' views based on transition specified in dialog description.

Besides the dialog description, there is also www/config/statedef/inputDescriptionSCXML.xml which is utilized by the InputManager. The InputManager is meant to receive the "raw" input events from different modalities/input devices (e.g. touch input, speech input, gesture input), and then relaying "meaningful application events" to the DialogManager (e.g. by modality fusion). For instance, the InputManager could process the speech event show me information and the touch event select(obj1), and map dispatch an application event show information about obj1.

It is important to note, that the SCXML files are application-specific. The SCXML files in the StarterKit are only an example of how to use these, specifically in context of this example application (i.e. the StarerKit).

In the StarterKit, touch input events of buttons will first raise an event on the InputManager that signal the modality (touch_input_event), and then raise the specific event with additional data: which object/element was touched and other data if necessary (see function executeAfterEachPageIsLoaded in app.js).

For speech input events in the StarterKit, the modality and data of the event is signaled to the InuputEngine within one single event using speech_input_event.

Figure 15 shows a visual representation the dialog description of the StarterKit. This description is depicted in Figure 17 as a finite state automaton (FAS). The start is the initial state of our dialog. The init transition (the transition from start state to the main_state) means that after raising the init event which will be raised in app.js of the StarterKit after loading all required components of the MMIR and some application-specific initialization. The init event triggers the transition from the start state to the main_state, which, on entering the new state, will render the login view of the Application controller (see onentry annotation: render('Application','login')). For more information on SCXML, we recommend reading the W3C SCXML standard.

<scxml xmlns="http://www.w3.org/2005/07/scxml" version="1.0"
        profile="ecmascript" id="scxmlRoot" initial="start">

  <state id="start">
    <transition event="init" target="main_state" />
  </state>

  <state id="main_state">
    <onentry>
      <script>
        mmir.DialogManager.render('Application', 'login');
      </script>
    </onentry>

    <transition event="click_on_login_btn" target="login_user" />
    <transition event="click_on_sign_up_btn" 
                target="registration_form" />
  </state>

  <state id="registration_form">
    <onentry>
      <script>
        mmir.DialogManager.render('Application', 'registration');
      </script>
    </onentry>

    <transition event="click_on_register_btn" 
                target="try_to_register_new_user" />
    <transition event="back" target="main_state" />
  </state>

  <state id="try_to_register_new_user">
    <onentry>
      <script>
        mmir.DialogManager.perform(
          'Application', 'register'
        );
      </script>
    </onentry>

    <transition cond="mmir.User.getInstance() == null" 
                target="main_state" />
    <transition cond="mmir.User.getInstance() != null" 
                target="logged_in" />
  </state>

  <state id="login_user">
    <onentry>
      <script>
        mmir.DialogManager.perform('Application','login');
      </script>
    </onentry>
    
    <transition event="login_failed" target="main_state" />
    <transition event="user_logged_in" target="logged_in" 
                cond="mmir.User.getInstance() != null" />
    <transition event="user_logged_in" target="main_state" 
                cond="mmir.User.getInstance() == null" />
  </state>

  <state id="logged_in">
    <onentry>
      <script>
        mmir.DialogManager.render('Application', 'welcome');
      </script>
    </onentry>
    <transition event="back" target="main_state" />
  </state>

</scxml>

Figure 15: Dialog Description of the StarterKit

The name attributes in state-, transition- etc. tags are non-functional, i.e. can be safely omitted. However, if you use Eclipse, its XML editor will use these name attributes in the outline view (usually displayed on the right), which can proof useful when navigating within the file.

Figure 16: Outline view for SCXML file with name attributes in STATE and TRANSITION tags

NOTE that Eclipse may have several XML editors installed. If there is no outline view available when you open an SCXML file, you may need to select a specific XML editor, e.g. by opening the context menu for the file, and then Open With → Android Common XML Editor (or Open With → Other... and select an appropriate editor in the Editor Selection dialog that opens).

Figure 17: Dialog Description of the StaterKit as FSA

Expanding the StarterKit with implementations (files) of models, views, or controllers does not extend the behavior of your application. For this, you also have to extend the dialog description of the StaterKit example. Lets start by adding some modifications in the welcome view of the application controller, that will create the GUI element. As you have seen in Figure 14, after logging in, there is no more interaction available to the user. Now we want to allow the user to create a new appointment. To do this, we add a new button to the welcome view. The new welcome view of your application should look as follows:

@contentFor("header"){
   <h1>
      @localize("welcome_header")
   </h1>
}@
@contentFor("content"){
   <h2>
      @localize("welcome_text")
   </h2>
   <button id="appointmentButton" name="appointment_btn"
           data-inline="true">
     @localize("create_appointment")
   </button>
}@

Each HTML button element in the StarterKit view descriptions must have a unique name.

This is due to the fact, that the button's name is used for automatically generating touch events (see the executeAfterEachPageIsLoaded function in app.js).  

For HTML button elements, the StaterKit example automatically raises the following events (synchronously)

 

 

• touch_start_on_name
• touch_end_on_name
• click_on_name

where name is the name of pressed button. For example by pressing the appointment button in the welcome view, MMIR would first raise touch_start_on_appointment_btn, then touch_end_on_appointment_btn, and lastly click_on_appointment_btn. This mechanism can be specified by setting a callback function via the setOnPageRenderedHandler function in DialogManager.

In the StarterKit example, this callback function is specified by the function exectueAfterEachPageIsLoaded in app.js (which is passed into DialogManager.setOnPageRenderedHandler() during the initialization process).

Next, we need to extend the config/statedef/inputDescriptionSCXML.xml file, so that it can handle the touch event of the newly created button. Note that by default the added button will trigger 2 events on the InputManager for each 'touch_start', 'touch_end', and 'click'-event: first an event touch_input_event, and then the corresponding event for the button, i.e. in this case touch_start_on_appointment_btn, touch_end_on_appointment_btn, and click_on_appointment_btn.

When we look at the inputDescriptionSCXML.xml we can see, that the first touch_input_event will bring the InputManager's state machine into the touch_input state, which in turn automatically redirects to its inner state start_touch:

  <transition event="touch_input_event" traget="touch_input" />
  
  <!-- ... --> 

  <state id="touch_input" initial="start_touch" name="touch_input">
    <state id="start_touch" name="start_touch">
    <!-- ... --> 

For extending the inputDescription to handle the button's event, we first add a transition for the start_touch state, that will be triggered when the click-event off button is fired:

<state id="start_touch" name="start_touch">

<!-- ... -->

  <transition event="click_on_save_appointment_btn" 
    target="save_appointment"></transition>

Then we create the transition's target state appointment, within the touch_input state, which finally handles the event: on entering the appointment state, the "application event" for the button will be raised on the DialogManager:

<state id="touch_input" initial="start_touch" name="touch_input">
  <state id="start_touch" name="start_touch">
  
  <!-- ... -->

</state>

<!-- ... -->

<state id="appointment" name="appointment">
  <onentry>
    <script>
      mmir.DialogManager.raise(
        'click_on_appointment_btn'
      ); 
    </script>
  </onentry>
</state>

Now we have to handle the new event in the dialogDescription. For this, we add a transition in our dialog description which will be triggered after raising the click_on_appointment_btn event. To do this, we first open the config/statedef/dialogDescriptionSCXML.xml file and find the logged_in state, and the add the following transition to it:

<transition event="click_on_appointment_btn"
            traget="create_appointment" />

We now need to add the new create_appointment state. The following code snippet contains the definition of our new state.

<state id="create_appointment">
  <initial>
    <onentry>
      <script>
        mmir.DialogManager.render(
          'Calendar', 'create_appointment'
        );
      </script>
    </onentry>
  </initial>

  <transition event="click_on_save_appointment_btn" 
              target="save_appointment" />
  <transition event="click_on_discard_appointment_btn" 
              target="logged_in" />
  <transition event="back" target="logged_in" />

  <state id="save_appointment">
    <onentry>
      <script>
        var data = jQuery.parseJSON(
          '{"container_id":"create_appointment"}'
        );
        var result = mmir.DialogManager.perform(
          'Calendar','create_appointment', data
        );
      </script>
    </onentry>
    <transition target="logged_in" />
  </state>

</state>

In the StarterKit the event flow for user input does not go directly to DialogManager (i.e the dialog dialog description), but is handled through the InputManager. Generally, first an event is raised on the InputManger signaling the modality: For touch events, the touch_input_event is raised. Then, the actual "meaningful" event is raised on the InputManager, e.g. click_on_login_btn.

Analogously for a speech input, a speech_input_event would be raised first, followed by an event submitting the ASR (Automatic Speech Recognition) result.

See also config/statedef/inputDescriptionSCXML.xml for the state chart specification of the StarterKit.

After adding the new state to our dialog description, we are can start the application and inspect our newly added modifications. Figure 18 shows the new screens of calendar application.

Figure 18

In order to test, if the SCXML files have valid syntax, we can explicitly compile the SCXML files (previous versions of MMIR required the resulting JavaScript files of the compilation, but the current version interprets the SCXML files directly using SCION).

For this compilation we use the common-scxml-js tool from the Apache SCXML's sandbox projects.

[For reference: the Apache sandbox project common-scxml-js is not online anymore; it was located here https://www.google.com/search?q=common-scxml-js ]

You can explicitly compile the SCXML files using the ANT build.xml task with the targets generateDialogEngine and generateInputEngine (or when generating all resources, use compileAllNodeJs or compileAllNodeRhino).

---

< previous: "Expanding Application Dictionaries"