Learning Analytics Model (LAM) and meta-LAM

The Learning Analytics Model (LAM) is responsible for describing which data is to be collected for a particular educational videogame and how this data should be processed, reported, and interpreted. The LAM is an integral part of building usable dashboards for the different stakeholders, and is a key component of Game Learning Analytics.

For a complete definition of a LAM the following steps need to be previously defined:

1. Learning goals (e.g. specific knowledge, procedures, tasks) to be achieved in the game.
2. Game goals (e.g. tasks, levels) that reflect the learning goals in the game.
3. Traces to be sent by the game for its later analysis and visualization, containing the learning information and following the xAPI-SG Model.
4. Analysis model to extract the meaning (related to the learning goals) from the xAPI traces to determine if the game is actually meeting is expected goals or not
5. Visualizations to adequately represent the traces and information already analyzed to help stakeholders understand their meaning.
6. Personalized alerts and warnings if needed can be added to the dashboard by specifying what are the conditions under they shall trigger.

For BEACONING, a general meta-LAM is also required detailing how analytics for a single game or mini-game should be aggregated for game-plots built of multiple games and mini-games.

LAM example: First Aid Game

First Aid Game is a game designed for players between 12 and 14 years old to teach first aid techniques in three particular situations: chest pain, unconsciousness and choking. Each situation appears as a different game level where the player may interact with the main victim or a mobile phone. By answering textual or visual questions, the player learns if the decisions made are correct or not. After completing each level, a mark from 1 to 10 appears according to the errors made and their importance. The complete description of the game can be found in [1].

For the First Aid Game, a complete Learning Analytics Model was defined including the game’s learning goals, game goals, traces to be sent, analysis model, visualizations and personalized alerts and warnings. This LAM was used in the experiments that validated the Analytics Technology for BEACONING.

The defined LAM for First Aid Game is the following:

1. Learning goals:
   1. Learn how to react in an first aid emergency situation.
   2. Learn first aid techniques for chest pain situation.
   3. Learn first aid techniques for unconsciousness situation.
   4. Learn first aid techniques for choking situation.
   5. Learn the emergency telephone number (112).
   6. Learn the security position.
   7. Learn the CPR procedure.
2. Game goals:
   1. To satisfy LG1: Successfully complete all three levels of the game.
   2. To satisfy LG2: Successfully complete chest pain level.
   3. To satisfy LG3: Successfully complete unconsciousness level.
   4. To satisfy LG4: Successfully complete choking level.
   5. To satisfy LG5: Successfully answer the specific question about the emergency number.
   6. To satisfy LG6: Successfully answer the specific question about the security position and watch the explanation video.
   7. To satisfy LG7: Successfully answer the questions about: How to open the airway, how to determine if the patient is breathing, correct position for doing chest compressions, rhythm of the chest compressions. Also, watch the video about the entire procedure.
3. Traces to be sent:
   1. Progress in each level: Initialized trace at the beginning of each level; Progressed trace with each progress in the level with progress from 0 to 1; Completed trace when level is completed and extension with level score.
   2. Score in each level: sent as an extension in the completed trace of the level.
   3. For each question, answer selected by the student and if the answer is correct or incorrect: Selected trace of type alternative with response the selected options and success true if the option is correct or false if it is not.
   4. Scenes and Cutscenes (Accessible): Accessed trace that is sent when the player enters a scene or cutscene; Skipped trace that is sent when the player decides to skip a cutscene (video).
   5. Interaction with game elements (e.g. game main character or mobile phone): Interacted trace with object the element interacted with.
4. Analysis model: The analysis model should specify how the traces are going to be analyzed to extract the relevant information to be displayed in dashboards and to derive the previously defined learning goals (Traces are in xAPI format but this is not relevant here. The important is how to interpret that traces). The analysis model for this example includes:
   1. Progress of the whole game increases in ⅓ with each completed level.
   2. For each level, the completed trace and its score will determine if the learning goal (LG2, LG3 and LG4 for the three levels) will be achieved or not. For this purpose, a threshold for the score needs to be defined: for instance, we set the threshold to 5 out of 10, therefore, if the analysis receives a completed trace for level chest pain with score higher than or equal to 5, we conclude that GG2 has been achieved, and therefore, from its definition, we conclude that LG2 has been achieved too.
5. Visualizations: For this LAM, some of the default visualizations available were used as they provided information of interest that matched the needs of this game and that helped to provide some insight into some of the learning goals. These visualizations should be driven by the teachers requirements (the same for alerts and warnings). These include  the following visualizations 1, 2 and 3. Some game-dependent visualizations were also developed for this game to show further information of interest for teachers that increments the control of the students’ gameplay such as the following 4 and 5.
   1. (LG1, LG2, LG3, LG4) Progress in each of the three levels (completables) and in the complete game from 0 (not started yet) to 1 (completed): a bar chart with all students in x-axis and, for each of them, four bars containing the progress from 0 to 1 in y-axis of the three levels and the complete game.
   2. (LG5, LG6, LG7) Correct and incorrect answers in each alternative (question in the game): a bar chart with all alternatives in x-axis and, for each of them, a stacked bar with count in y-axis of correct answers (depicted in color green) and incorrect answers (depicted in color red).
   3. (LG6, LG7) Videos seen/skipped by the student. Vertical bar chart that displays a pair of sliced column: left one displaying the videos seen by the student, right one displaying the videos skipped.
   4. Number of times each students asked for help in the game: a game-specific question asked whether the player wants the emergency services to stay on the phone to help the player. This game-specific visualization showed a bar chart with all students that asked for some help
   5. Number of times each video has been seen inside summary menu: the game contains a final menu after each level where it is possible to play again some of the videos seen in the level. This visualization showed how many times each video was played again in this menu. A pie chart was displayed to show the number of times each video (accessible) has been accessed.
6. Personalized alerts and warnings:
   1. “The user has failed, at least, once the question about the emergency number”: important question the teacher wants to be notified about.
   2. “The user has completed Chest Pain or Unconscious and has not used the defibrillator”: situation the teacher wants to be notified about.
   3. “The user has completed Chest Pain or Unconscious and never performed cardiopulmonar reanimation (CPR)”: situation the teacher wants to be notified about.
   4. “The user has failed Chest Pain game mode”: the level has been completed but failed, so the knowledge has not been acquired.

References:

[1] Marchiori EJ, Ferrer G, Fernández-Manjón B, Povar Marco J, Suberviola González JF, Giménez Valverde A. Video-game instruction in basic life support maneuvers. Emergencias. 2012;24:433-7. http://emergencias.portalsemes.org/descargar/video-game-instruction-in-basic-life-support-maneuvers/force_download/english/