Benetech is a nonprofit that empowers communities with software for social good. Benetech received a 2017 Digital Impact Grant for the project “Gamified Digital Security Training.”
Team
[ess_grid alias=”benetech”]
Project Overview
Through gamified training, we created an immersive, simulated environment in which players are required to implement their knowledge in context. This provides a scenario in which players can safely fail and learn from their mistakes.
Challenge
The need for an effective and scalable approach to training human rights defenders is clear. Current security training tools reside in unapproachable and dense manuals and guides, or in-person trainings that are expensive to provide and allow for only a limited number of people to attend. Further, there are few means to measure how participants in these training workshops learn, how much they transmit back to their colleagues, and to what extent they utilize their new knowledge and skill set in their work. In sum, manuals and workshops are not scalable approaches for effectively teaching security best practices to human rights defenders.
Approach
An ideation workshop in December 2017 was key to kicking off collaborations with stakeholders including human rights defenders, game designers, and learning game experts. During the grant period we also produced three paper prototypes that were user-tested by human rights defenders associated with Stanford University, and a digital prototype created and tested virtually by 15 additional human rights defenders from Western Europe, the US, and various post-Soviet States. Additionally, feedback was reviewed and synthesized collaboratively during a design workshop with Benetech, the OSCE Office of Democracy Initiatives and Human Rights (ODIHR), and human rights defenders in November 2018.
Benetech leveraged the excitement generated by the ideation workshop to team up with ThoughtWorks and LRNG, both leaders in the technology development and ed-tech spaces and experts on game design and implementation. The team included Benetech Human Rights Program staff, Andrea Morales, a game designer from ThoughtWorks, and Paula Escuadra of LRNG who specializes in data and reporting in game design.
Conversations
Rachel Bernstein presented the work of Benetech and partners at the Data on Purpose conference at Stanford University in February 2019.
2017 Digital Impact Grantees from Stanford PACS on Vimeo.
Outputs and Progress
- Early Prototype: Designers selected three top learning objectives for each of two game concepts – a narrative-driven game and a puzzle. These games were developed into three paper prototypes: a narrative-focused game teaching secure data back-up, a puzzle game teaching information verification, and a hybrid version combining both methods into one game. The prototypes were then refined with feedback from other designers and the Benetech team.
Once the paper prototypes were tested by six human rights defenders from different regions with varying levels of experience, and familiarity with security practices, we selected the narrative-driven concept and further refined the story, mechanics, and features. This became the basis of our digital prototype. We finalized the prototype in Fall 2018, following additional iteration, and tested it with 15 volunteer human rights defenders. - Digital Prototype: The “game” begins with the player receiving a frantic phone call from an activist friend who is in the hospital. The friend asks the player for help and player starts by exploring the friend’s room for clues. The first question we asked our testers was if the narrative approach was an effective way to teach these core human rights and security concepts. The answer was yes: testers were clearly drawn in by the emotional hook of the story line, which felt authentic to their lived experiences. The key learning objective in the digital prototype is for users to understand security “tradeoffs” between different methods of backing up information. And while we are still working on ways to measure in-game learning, testers were asking themselves the right questions as they played, which they shared with us. The game ends with a readout of the players’ performance on their mission and on each of the learning objectives. The emotional draw of the narrative seemed to extend even into these brief written “consequences” of in-game actions. We were pleased that players appreciated the feedback aspect at the end of the game and followed embedded informational links to learn more about encryption or backing up their data.
Insights
- We are pleased by the clear enthusiasm of users for the narrative approach and the degree to which the prototype is proving to be engaging intellectually and emotionally, with users following informational links to bolster what they have learned about encryption and backing up data. Early feedback indicates likely behavior changes based on what was learned during prototype testing.
- The digital prototype seems clearly extensible in terms of a future build out as a stand-alone tool with “episodes” or “chapters”, as well as a complementary immersive learning and evaluation tool.
- The value of such a simulated environment is the opportunity for applied learning – players who might answer a question correctly on a quiz, actually make the incorrect choice within the game, which tests their knowledge in context. Players report learning the most from these types of failures. Future iterations of the game will draw on these insights to design realistically ambiguous situations for players to navigate as well as design learning measurement systems that include both in-game metrics and complementary evaluation instruments.
- The iterative process is a gradual yet necessary part of designing a successful product. In the video game industry, iterative game design entails repeated ideation, prototyping, play testing and re-evaluation prior to releasing a working product. This gradual refinement based on key criteria (i.e. engagement, effective learning outcomes) increase the potential for developing a successful final product.
Next Steps
- Human rights defenders who participated in testing of our digital prototype universally want to know what happens next in the story line. We are currently working on “Episode 2” to extend the story line and embed more human rights monitoring best practices and security concepts that are relevant to the work of human rights defenders. Refinement of the initial digital design is ongoing and requires additional prototyping of the significant changes made.
- To work on the next level prototype, Benetech brought on a game developer and game designer/artist. This allows us to develop in Unity, an industry standard game engine, resulting in a more polished product with more complex game mechanics. The higher fidelity helps testers to focus their feedback on the content of the game rather than the interface.
- We are testing how different elements of gamification contribute to learning, as well as investing our time and resources to ensure the final product effectively engages users and imparts intended learning objectives. Testing of this refined prototype should be complete by June 2019.
Learn More
Find out more about Benetech’s work here.