I learnt things the hard way and this article is an attempt to reflect on the mistakes I made while collecting feedback for online games-based learning courses and gamified content. In my experience, the following holds true regardless of the type of content – gamified, storified, serious games, elearning, microlearning or nanolearning.  If you are an online learning designer or just someone who is learning about using online courses, I hope you add to my points to help me and other learning designers become better.

Seeking feedback at the far end of the course

It is very likely that you have witnessed innumerable courses with a feedback link at the very end of the course. There’s nothing wrong with it, but let me pop the bubble for all learning designers out here. This is the most predictable thing you can do with your wonderfully crafted gamified content.

Let’s take a step back

We are here talking about content that is not only educational, but also engaging. This means there should be a healthy dose of unpredictability to it. The obvious problem with this is that many learners may never get to the end, here. If this is the case, you have already lost the opportunity of capturing unheard voices. Let’s also address the elephant in the room – Who is this feedback for? If your answer is – for you, so you can make the course better, that is just one part of the answer.

It is true that after spending hours in content curation and experience testing, as learning designers, we would like to know how we fared. We may also like to jump right in, and course correct in case there are some participants who have shared areas of improvement. However, the overlooked, often ignored part, of this is that a lot of this feedback is for the learners themselves.

Think about it – when they are asked a question such as, “Can you think of an example of something similar from your day-to-day life? “, the learners are really reflecting on how the course related to them. In parallel, for the learning designer, if such a question is being left unanswered, it leaves room for some food for thought. Perhaps, the learners don’t have adequate information to relate it back to their lives, or perhaps this question is too repetitive, or asked too soon.

While asking for feedback at the end of the course is the most used mechanism of asking for feedback, there are several subtle ways to ask for feedback all through the course too.  Here are some of the ways that I have tried, tested, failed and then course corrected. I hope they help you, too.

1. Include reflective questions

Reflections or drawing from past experiences is one of the best ways to bring awareness to what learners already know. As a learning designer, you could use reflective questions throughout the course. An effective way to do so is to use a cocktail of video, photo, multiple choice, and text submissions. This ensures that the feedback does not feel repetitive for the learners. It also adds a layer of fun to questions that can otherwise become really mundane.

2. Gauging engagement levels

Social engagement within the learning group, whether it is through upvoting the submissions made by their fellow learners, or comments, or even based on the promptness of the submissions, voluntary engagement (gauged by minimal nudges) – these are all effective measures of seeking feedback. An alternate way to put this is – if the learners are not automatically gravitating towards the gamified content, despite the push notifications and reminders, it is time to course correct as a learning designer.

3. Social media

Of course, this may not work for all situations. It goes without saying that most people are on at least one social media platform. Encouraging opportunities such that learners can share their experience on social media platforms of their choice, using unique hashtags given by the learning designer, is a wonderful way of creating a community that engages in playful way of learning.

Furthermore, if the course is designed to be an open course, this may even create a positive spiral that attracts more participants.

4. Difficult questions

Most of the questions for seeking feedback are usually implemented in the form of a scale ranging from 1-10. The questions reading as – ‘How would you rate your experience?’ There’s no doubt that this is a great starting question but what about asking difficult questions such as “what made you skip a question?”, or “Would you still take this course if your organization did not make it mandatory?”, Or “Did you ever think that this course is not worth your time?” These are the questions that can help elicit real feedback. As learning designers, you want participants to look at the course objectively.

5. Crafting the right questions

As a learning designer, you should consider keeping a mix of questions that are multiple choice (like a star rating), and specific open-ended questions (such as “How you will apply the learning in work / life?”)

Creating a mix of questions will ensure that the learners don’t feel overwhelmed and interrogated, and yet feel heard.

6. Go beyond asking questions just about the content

To a learner, the course is an entire package from registration, payment, ease of access, ease of navigation, to the elements of gamification such as the points systems and badges, etc. Hence, there need to be questions that capture each pain point that the participant may have felt. Even those participants who were not able to access the course at all, should be able to reach the learning designer and provide feedback mentioning their difficulties.

7. Critiques are equally important

If all questions are worded to seek positive feedback, such as “How this course has helped you?” or “What will you do differently?”, learners will automatically let you know *just* that, or skip the question altogether (marking an N/A and moving on falls under this bracket too.) However, asking open ended questions about what the learners would want to change, or what the learners did not like in the course, are good ways to collect feedback that truly helps you design better courses.

8. Personalisation

An abrupt question at the end of the course is the worst way of ending well-curated content. The learners have spent time in engaging with the content. This is especially true if the content is spread out over a few days/ weeks. Hence, it is important to take a moment to acknowledge all the time the learners have devoted to the content. A note that is personalised to that group of learners is a wonderful way of letting the learners feel valued.

Conclusion

Personally, I love going through every feedback that I receive for the online gamified content, and I take the time to reply to each one. It is worth the time and has been my biggest mentor and constructive critic. In case you are not a learning designer but a consumer, please know that as learning designers, you are our greatest strength and supporters. And I really hope that you are able to share candid feedback each time you take up an online course.

If there is anything that has helped you, I’d love to hear your views. Know that I value each and every feedback, regardless of whether it is positive or not.

The post How To Collect Feedback For Gamified & Digital Learning first appeared on Ludogogy.

The Ubuntu Game is a physical and virtual board game designed to teach teenagers the wonders of diversity, inclusion and equality as well as empowering them to discuss critical issues facing us as a society like climate control, overfishing of the oceans and sustainable farming.

It does this through game play that challenges its players through questions, conversation, collaborative activities, experiential media (videos and virtual reality) and discovery about themselves, their beliefs and their likenesses and differences through culture, race, creed and other differentiators.

Today’s Teens are Problem Solvers

At Ubuntu Game we believe today’s youth are smarter, more opinionated and have greater access to information and therefore greater capability in thought processing and problem solving than any prior generation. Because of this, it is essential that their viewpoint on board game design and even their direct input be cultivated and encouraged. The majority of board games are designed by white males who dominate the space but also restrict its diversity, a known industry issue.

To combat this limited view in design, we released a beta or Minimal Viable Product (MVP) version of our game, designed to be played by beta customers to garner feedback and constructive improvements towards the game. It is a generally accepted practice that an MVP is a preliminary beta product with some clear cut shortcomings that can be iteratively improved or even pivoted on, depending on the nature of the feedback. Ubuntu Game solicited direct feedback from teenagers playing the beta game themselves.

With the success of the Ubuntu Game directly tied to the power or quality of our questions, our questions were derived from a diverse collection of young and older team members, some of whom were well versed in the diversity/inclusion space. We plan to crowdsource our game content and all-important game questions through our players ultimately.

This initial vetting of the game is all typically part of a lean startup or EXO initiative, which we adopted during the formative stages of Ubuntu Game to validate what we were doing.  In the EXO framework, much up front thought is given by the game developers towards identifying the Massive Transformative Purpose or MTP. What is the critical impact the game will have in a positive way?

Our Massive Transformative Purpose

With Ubuntu Game, our MTP was to create a movement where diversity and inclusion and equality became the norm in the world to counter the recent outbreak of derision, bullying, racism and marginalization that bubbled to the surface due to Black Lives Matter and other incidents.  This is considered alongside the specific problem which is being solved and how our game can solve the problem. Finally, of course, it makes the mechanics of the game much easier to design.

We felt that the route we wanted to take was getting teens into conversations to explore and discover their differences and to better understand and accept them. Initially, our focus groups validated this hypothesis as a core approach to diversity and inclusion understanding! This allowed us to formulate a Value Proposition Canvas to sketch out the benefits and problems solved by our game.

Within the game, we built online questionnaires and surveys to obtain feedback from game players to improve the game. We also had a Breakthrough Wheel which each player spins at the game’s conclusion,  to determine what was their top takeaway from the game, their biggest personal breakthrough, or who they would like to share something with that they learned from the game.

In addition, they may be asked “What would you do differently after today’s game, or what action will they immediately take after today’s game?” or “What discussion will you have with your parent(s) after today’s game?”

Sharing as Learning

This is all intended to memorialize or anchor the key impacts or impressions from the game for each player, and to get them to verbalize it so that it becomes more present to them. Given the game play is a shared experience with other game players, this sharing can lead to significant breakthroughs and discovery by players.

This learning process sets the players up to look forward to future play sessions and clearly places their shared peer experience as a paramount feature of the game. The crystallization of the game’s outcome for the players makes them more mindful of their own growth, which is perhaps the game’s most important objective.

In conclusion, getting young adults to talk about their experience, contribute to the game’s content and seeking their ongoing feedback is what sets Ubuntu Game apart.

HOW THE UBUNTU GAME IS PLAYED

Players traverse the board using mini cultural icons by rolling a die. They land on spaces that trigger questions and conversations in six categories that matter, including diversity, connection, understanding our differences, our similarities, controversial topics and fun facts.   A mobile app is used to add context, enlightenment or information to kindle the conversation. Players can play as individuals, as pairs or teams. Some players will spin to see what kind of barrier to play they will experience.

Virtual and Augmented Reality

Additionally, players will get to role play as “Avatars” and see themselves, literally, through a Virtual Reality experience, as someone from another culture, creed or place in life. Augmented Reality with accentuate some of the images they’re familiar with in their surroundings.

Players will also participate in activities, one on one or with the group, solving riddles about society and diversity, work collaboratively towards a common goal or even just have fun pitching in together to make a recipe!

The game is over when every player reaches the end of the board. The player, pair or team with the highest score wins. Winners get to spin the “Breakthrough” wheel landing on various options to explain what’s changed in their perceptions and what actions they will take.

Read Richard’s other article about Ubutntu game and watch the play session.

The post Debriefing and Feedback for Ubuntu Game first appeared on Ludogogy.

At the end of a training serial, tank commanders will immediately conduct a ‘hot-debrief’, discussing what went well, what went badly and how they can improve next time.

After several serials the commanders congregate in a trailer and watch video analysis of their performance. Data is gathered throughout the process, everything from exact grid locations to radio chatter snippets.

Throughout the process, there is always a frank, honest and open discussion between the team which doesn’t seek to apportion blame, but simply wishes to be better next time.

These discussions are guided by an external facilitator, but the direction of the discussion is dictated from within.

Within this dynamic the data is the essential building block of accountability, it takes the debate beyond what happened (we can see that, live and in real time) and moves into why it happened – because when you understand the why you can make different decisions in similar situations.

So how can this translate into the Corporate Environment?

As an Army Officer, turned Escape Room Designer I’ve always been fascinated by how the participants interact within our experiences – why certain participants speak continuously or why some of the best puzzle solvers say nothing at all.

We knew that what we were observing was immensely valuable; it was gamified, visual, and showcased teamwork at its very best (and worst) but we needed to understand what we were seeing. By working with Michelle Mills-Porter, a leading behavioural analysist, to understand what was going on within the room we learned that individuals were showcasing their core behaviours.

So how to design an experience which would provide us the best data

The crafting of the experience needed to reflect the problems teams encounter during their working day, allowing the gamified data to prompt team action towards actually achieving tangible change. This was done through the design and implementation of ‘loops’ – specific puzzles and challenges which showcase individual and team dynamics.

We use a wide range of loops, for example:

Time: Of course, the experience is timed, but we also introduce timed challenges within the overall experience, forcing teams to hit tough deadlines – only achievable by managing their priorities.

Asymmetric Communication: ‘I know something you don’t know’ – by sending some individuals out on tasks which are outside of the room, participants rely on their communication via radio to achieve team goals.

Collaboration and Prioritisation: Elements of the puzzle cannot be solved without participants working together, but to achieve all group aims, teams will need to understand where their and (more importantly) where their colleagues’ skills lie.

Of course, there are many more, which gives our facilitators the opportunity to select those most relevant to the team.

And so to feedback

The experience is filmed throughout, we have an analyst on site who observes in real time and can provide the feedback that day or, if client time allows, we can come back the following day armed with more information.

The headings are delivered by us; based on the data we have observed, but the discussions are led by the team – it is critical they find the solutions to where they’ve been going wrong together, whilst also highlighting group successes.

We’ve seen transformational moments during this process; incidents of participants being spoken over, despite having the correct solution and then not proffering this again. It is not a gigantic leap to suggest this may be happening in the work environment.

How to confirm the change

Though this process is transformational, it is critical that actions are agreed on. We base our workshop on how different personality types prefer to be communicated with – from youth we’re told to ‘treat others as you would wish to be treated’ why not go on step further and treat them as they wish to be treated.

To showcase the power of this change we mix behavioural types and use immersive VR exercises to practise these skills, so that participants leave having understood the power of their words and behaviours.

The post Feedback – Creation and Delivery through Live Experience first appeared on Ludogogy.

The pandemic, however forced us to think about taking these workshops online, initially with the idea of improving engagement and seeing how close we could get to “the face-to-face experience” (see this article from Kanban Zone. What we discovered, though, was that we could make the de-brief and learnings from the workshops extremely relevant and engaging, and thus more convincing, in an online setting.

Light-bulb Moments

The debrief part of all of these games is designed to give an epiphanising “light-bulb” moment where participants suddenly “get” the idea. However, many of the explanations rely heavily on data, and it is often tricky to calculate this data quickly enough to replay it in the debrief. Hence, we must use either use stock datasets and graphs to illustrate the concept or have a break while we do calculations and draw graphs. The former doesn’t always convince, however, and the latter leads to us “losing the moment” which lessens the impact.

Taking the workshops online necessarily meant writing software, and that gave us the opportunity to do any data analysis in real-time, based on the actual behaviours of the participants. We quickly realised this was much more relevant and convincing. Sceptics can easily argue against stock datasets, but it is much harder to do so when the conclusions are based on their behaviour in a game they have just played.

We saw a great example of this when playing the No Estimates game at the recent Agile 20 Reflect conference. The No Estimates game demonstrates that estimating is futile, as it is impossible to do it accurately – quite a big mindset change for many people! In the game, we give players more information (e.g. the exact number of tasks to do, and the effort required for each task) than they would have in real-life, so they believe – as they are convinced they can estimate 100% accurately in real-life – that they will be able to do the same in the game.

The image below shows, for two of the teams in the game, their initial estimate, their estimate for the first 11 tasks, and the final number of days it took to do the. As you can see, even though both teams took the same amount of time – 40(ish) days – and had exactly the same information to start with, their initial estimates were wildly different.

Showing this information is much more convincing than saying “typically, people’s estimates are up to 50% out”, and is the first step towards dismantling participants belief in how accurate their estimates actually are.

The reason that estimates are always wrong is that people make many, many assumptions when estimating. Merely pointing this out never convinces anybody, but if we use the actual game data, we can demonstrate this very powerfully. As an example, there is an implicit assumption that bigger tasks will take longer. In the game, we can demonstrate this is not the case by showing the correlation between task size and time taken, as for one of the teams here:

A correlation of 1 means the bigger a task, the longer it will take. As you can see, this is far from the case; 0.49 is quite low. If we plot a distribution of the size of tasks, and how long they took, we will often see very small tasks taken longer than very large tasks. Again, showing this for the actual work the team has just done in the game really highlights this incorrect assumption.

In the face of these challenges to people’s prowess in estimating, one thing they often say “but we know there are variations, so we use an averages and standard deviations to give a range”. Unfortunately, to do this kind of statistical analysis relies on a normal distribution of data, and this is also an incorrect assumption. The game data comes to our rescue to demonstrate this as well. Here is the distribution of how long tasks took for two of the teams in the game

As you can see, neither is close to a normal distribution, so using mean and standard deviation is invalid.

There are many other graphs and datasets we show to highlight incorrect assumptions, but we finish the game with this real light-bulb:

This is a monte carlo simulation showing how long it will take to do N task in the future. It allows us to answer the question everyone wants answering when estimating – “when will it be delivered”. This is the real light bulb moment; given how you just performed in the game, we can predict when 100 tasks will be completed.

And this is the real power of using the real game data. Using the actual data that players have generated themselves, we have not only come to objective conclusions about their incorrect assumptions, but we have also given them tools to rectify the situation that they can use in the real world. We have come full circle; we started by stripping the situation right back to isolate the concept in a game situation, and have ended by demonstrating concepts and techniques that are applicable in the real-world.

We can see the contrast when using real data if we look at another game we implemented – The Coin Game – we haven’t yet implemented the real-time data features, so we just show the graph we usually draw in the face-to-face version of the game. This shows the amount of value delivered (y-axis) over time (x-axis).

This does get the concept across, but how much more convincing would a graph be if it showed the actual amount delivered by participants over time? We could even show the curve developing in real-time as the game is being played to really ram home the point that it is always better to deliver the next highest value items first.

One of the main reasons we brought workshops online was that they are really improved if played in a number of teams; the competitive element gets participants concentrating on the game rather than thinking too much about the concept, and this makes the light-bulb moment even more of a reveal! Break out rooms allow us to re-create this multi-team atmosphere online, but it also allows us to take the concept further; teams can play asynchronously – in different time zones, on different days – making scheduling easier, and, with some games, we can trend performance and scores over time to demonstrate improvements and changes. Again, all this is made possible through the recording of game data.

Online workshops have really given us an opportunity to use real data, in real-time, to convincingly demonstrate complex concepts to often sceptical participants, and has revolutionised how we present our learnings and feedback.

Check out Corrado’s Ludogogy author archive

The post Engaging and Relevant Feedback from Online Agile Workshops first appeared on Ludogogy.

Enter Rocksmith!

Rocksmith is a game which claims to be “the fastest way to learn guitar”.

Load up the game, plug your guitar into your computer and play a song. The game detects the notes you play, and when you play them. It then gives you feedback to help you learn and improve your performance.

Amazing concept right?

I found playing Rocksmith to be an incredible learning experience, particularly as a beginner. There is a lot of inspiration to take from Rocksmith on how to make learning more fun and engaging. Here are two insights along with one caution/consideration!

Let’s start with a couple of areas where Rocksmith rocks.

Adapts to the Learners Skill Level

What makes someone feel like a rockstar? Help them feel like they are smashing out their favourite songs as soon as possible.

Rocksmith has a feature called ‘Dynamic Difficulty’. The difficulty of a song adjusts based on how well you are playing it. When you start out a new song, you are only given 10-20% of the notes to play. As you get better the game adds in additional notes. It also removes notes or segments you find tricky. There are elements of scaffolding at play here too, including built-in videos and mini-games to learn more complex skills. ‘Dynamic Difficulty’ works a treat at creating engagement, even though there are some issues about developing bad habits (which I’ll cover later)

Why ‘Dynamic Difficulty’ creates engagement? Flow Theory

The elegant model from Csíkszentmihályi shows when you get the match between challenge and skill right you can enter a flow state. It’s a beautiful place to be, where you become “completely involved in an activity for its own sake.”

Because the challenge in the game frequently adapts to meet your skill level it creates lots of moments of flow. Whilst I haven’t (yet) mastered playing ‘Seven Nation Army‘, I could start playing parts of the incredibly distinctive and catchy riff pretty quickly. And you know what? That feels great and that creates a motivating effect to continue to improve.

Insight: How can you create more moments of flow for your learners?

We’ve all been there as facilitators. When some of the group are going blazing fast, and others are finding things tricky or prefer going slower. This causes problems for both parties. The ‘faster’ group start getting bored, distracted, checking e-mails. The ‘slower’ group feel the pressure to keep up and can feel less adequate. It’s a lose/lose. Whilst it’s fairly common practice to flex difficulty at the program/workshop level, how can you go beyond this and better flex challenge within your programs/workshops to suit the diversity of every learner you work with?

If you can find ways to create ‘Dynamic Difficulty within your programs, you’ll create more moments of flow. And that’s good for building learners confidence to apply learning back in the real world.

? Make Practice Rewarding for Learners

How do you really become a rockstar? Practice. Lots of practice.

But for most people practising the same songs and notes over and over again can get pretty tedious.

Rocksmith has two clever features to help overcome this problem. Firstly it gives you a mastery score for every song you’ve played. You only need to improve your mastery by .1% to feel like your practice is making a difference and you are improving. And this keeps you playing the same songs over, and over, and over again. It’s a great feedback loop!

And when you get stuck on a tricky segment or can’t improve your mastery score? ‘Riff Repeater’ is here to help. ‘Riff Repeater’ allows you to isolate individual segments you are struggling with, it then allows you to slow those segments right down, and repeat until you’ve mastered them, before gradually ramping up the speed.

When I hit 100%+ mastery on La Sera – Love That’s Gone, (which has a lovely, melodic, yet simple bassline), I got a tremendous sense of satisfaction which immediately made me want to get 100% on another song.

Why is creating a sense of progress and mastery is so important? The Progress Principle

Professor Teresa Amabile and Steven Kramer wrote about “The Progress Principle” in their book, which is also summarised very well in their HBR article “The Power of Small Wins“. The essence of the principle is that making progress, however small, can increase people’s motivation and happiness.

Couple this together with the universal need people have to improve and attain mastery over tasks (Self Determination Theory, Deci & Ryan) and you can see how giving learners visible signs of progress that relate to their mastery is so powerful.

Insight: How can you create a better sense of progress for your learners?

Most people find it challenging to practice new skills back in the workplace, outside of the training environment. Part of the issue is that practising in the workplace feels high-risk for most people. People don’t like messing up and feeling foolish in front of their team members. And this is why creating psychological safety is so important. If there is a high level of psychological safety within your team/company, then people are going to be more willing to try something new, take a risk and be okay with messing up.

From a learning design perspective, this is where it’s essential to think about the whole learning experience and not just the workshop. And this is where many learning programs fall short in my opinion. There usually isn’t enough opportunities following a workshop for learners to repeat and practice their skills. The AGES model by the Neuroldership Institute covers off the key principles for effective long-term learning. Here are some examples of how you can easily build in more practice and a sense of progress in your learning programs:

• Take one of the activities used as part of of a workshop, and repeat it again at a team meeting
• Schedule in micro sessions which are just about creating safe spaces to practice and try out new skills, rather than learn and new theory or concepts
• Set skills-based challenges every week/fortnight which require learners to apply the concepts they’ve been learning about. Start these off nice and easy and incrementally make them more challenging.

? How Rocksmith Hits the Wrong Notes: Picking Up Bad Habits

“I guess I got a bad habit, and it ain’t goin’ away, yeah”

(Offspring, Bad Habit)

Whilst the game gives you tons of feedback, this feedback is focused on if you hit the notes at the right time, not how you do it. So it looks at the outcome, not the process. And when learning something new, the process is the most important thing to really focus on.

I may be hitting all the right notes and the right time, but it can still sound bad (and from experience it does!). The game tries to get around this with in-built lessons. But in reality, I have picked up poor techniques from self-teaching, especially in search of trying to get a higher score.

And whilst ‘Dynamic Difficulty’ is generally well implemented, sometimes you pick up bad habits to learn the easy sequence, habits which are hard to unlearn when you need to play the harder sequence. Some point soon I will need to get a teacher/coach to help me focus on technique and take my performance to the next level.

Consideration: How do you support self-paced training with coaching, mentoring and peer support?

With the continued rise in the demand for self-paced learning, companies need to carefully consider how they integrate coaching or mentoring support into the learning experience. This is to help with learning transfer, but also to make sure people don’t pick up ‘bad habits’ through misinterpretation or misunderstanding.

Similarly pairing self-paced learning with some group-based coaching can help amplify the online self-paced learning experience. Firstly it provides a great space for people to discuss the materials they are learning. And secondly, it adds a layer of peer accountability to the process which can be useful to drive completion and application of learning.

Make Your Learners Feel Like Rockstars

I’ve really enjoyed taking my bass guitar ability from nil to beginner. And a large part of that is due to Rocksmith making me feel like I could play and that I wasn’t completely incompetent.

The other beautiful gift Rocksmith has given me is the discovery of tons of new music and genres I ordinary wouldn’t have known about or thought of playing. By playing Rocksmith I have developed a love for Garage Rock and Blues Rock (go The Strokes and Black Keys). This is actually remarkable given my very narrow taste in music pre-Rocksmith!

The post How to Make Your Learners Feel Like Rockstars first appeared on Ludogogy.

It’s been challenging, but it has also resulted in a great deal of innovation, and discovery of new techniques and possibilities on the part of learning designers and facilitators. For example, many have discovered that learning games and playful activities are effective ways to deliver learning if you are required to use ‘online’, rather than a classroom setting. Along with that, many have also found that this brings with it enhanced opportunities for data collection and analysis of learner decisions and actions – which, in turn, effects opportunities for more detailed debriefing.

But, is there a difference between debriefing games and other kinds of learning activities? I would argue, No, and Yes.  While the skillset is largely the same, the richness of the learning which comes from games and play, comes not only from the experience the learners have while playing, but also from the broader, and deeper ways in which the experiences can be mined for insights, by a skilled facilitator.

Questions

The most fundamental tool in a facilitator’s toolkit is the question. Effectively, their job is to ask the killer questions (which they have carefully honed to elicit the most effective insights) and then get out of the way while the learners reflect and learn from their own responses. Games allow for different questions (or rather, for different levels of reflection in the learners’ answers) than other learning activities, because the learners have just ‘lived’ an experience which allows them to explore the learning domain, rather than being told, or reading or hearing about it. They have, probably, solved problems to attempt to ‘win’ at the game, and have undergone a time-compressed version of exercising a particular skill or piece of knowledge.

Compare the two learning scenarios. In one, learners have watched a video where an individual talks about their experience of leadership in a large company. In another, they play a game where each of them has taken a turn at leading their teammates through a set of scenarios, evaluating data, making decisions, being presented with the impact of those decisions.  Instead of asking ‘What did they do?’ or ‘What did you think of that person’s actions?’, we are able to ask ‘What did you do?’, ‘Why?’, ‘What would you do differently next time?’ ‘What factors did you consider?’ … and so on.

What’s the Score?

One prominent feature of games and game-like activities is scoring. This is even more marked in digital games, where the possibilities for data gathering are limitless. ‘Scoring’ can be much more than a final count of points to see who ‘wins’. Measures can be kept throughout a game including player level (and corresponding capabilities), ‘spendable’ points such as units of effort or currency, fluctuating scores of success on different metrics, and quantity of ‘territory’ held (non-exhaustive list)

But these are just data, and it is the facilitators job to draw out the insights from them. This is perhaps the most marked difference between facilitating debriefs of games and other activities. Facilitators must themselves be familiar with what the data signify and plan questions which will allow learners not only to reach conclusions but also to practise skills in data analysis. It is by no means certain that these skills will exist in the learners, so facilitators will often have to adapt their questioning on the fly to work with the learners’ existing level of competency and scaffold their learning to reach higher levels of skill over the duration of the game.

Failures and successes

Failing, in a safe space, is an often quoted benefit of using games for learning. While this can be thought of as being largely about the psychological safety of learners (or physical safety of themselves, and others, if the learning is on a safety topic), it is something which should be a major resource for the facilitator.

Failure is a great teacher. The games-based learning facilitator must become skilled in getting learners excited about failing. One real danger is that learners become disengaged when they ‘lose’, especially if the game is a competitive one. It must be made clear, through facilitation, that winning at the game is not the same as winning at learning.

One way to do this might be to gamify the debriefing, so that there are visible measures of ‘winning’ at the learning too. Peer assessment of the quality of insights shared with the room is a good way of doing this.  The facilitator can also give greater attention to questioning the insights that emerge from failure, to emphasise the value that is derived from it.

Competition and Other Attractors

Competition has to be treated cautiously, for the reasons stated above. One of the benefits of games is that they engage, so we must be extremely cautious of losing that benefit when people fail. Competition is also an aspect of games which appeal more strongly to some player types than others.

If we are going to use play and games, we should be aware of player types and the particular aspects of play they find attractive, not only while designing and delivering the play experience, but while debriefing it. Using a player type model such as Marczewki’s Hexad and a model which evaluates gamified experience, such as Yu-kai Chou’s Octalysis framework, will allow you to plan your debrief with different player types in mind.

If the play experience itself favours particular player types, you can use the debrief as an opportunity to cater to the needs of the other player types. For example,  players who value the social aspects of the experience could be drawn out with questions which focus on the value of social interaction or utilising people skills in how they approached their play strategy. Those who are rewarded by the exercise of creativity would enjoy the opportunity to share how they worked to find creative solutions.

All learners will benefit from a debrief that gives space for the strength of all player types to be expressed and analysed.

Storytelling

Many games include a narrative element, and all games, I would argue, allow players to construct their own narratives of how they interact with the game. Even really simple games will allow learners to answer the question ‘What was your strategy to play this?’ with a story about their thought processes, or if the game itself has an embedded narrative, to continue the story of the game with their own point of view and ‘plot twists’.

Storytelling holds a special place in our repertoire of learning activities because of the specific cognitive impacts that it has. Among these are greater emotional impact than other types of communication and an enhanced likelihood of persuading the listener. Not only that, but research show that the brain waves of those listening to a story can start to synchronise with those of the person telling it. Stories can enhance empathy, help us to see things from another person’s perspective and result in shifts of core values which persist even after the story is over.

Facilitators should think carefully about how to couch their questioning so that they maximise the potential of learners responding in a narrative way.

Moving Forward and Hypothesising

Along with the ‘safe space’, games are excellent for providing opportunities for ‘do-overs’. In conjunction with failure, being able to try again with a different strategy, or being able to play the same scenario with slightly different parameters, gives us a chance to learn that rarely happens outside of a game setting.

Inviting learners to hypothesise about future results and then to evaluate how well those results match their hypothesis is an effective way to get learners to be active in their learning and to think seriously and deeply about how they are going to play, rather than just ‘winging it’

Games without a Facilitator

While this article has focused on the (cautiously optimistic) idea that we will be moving back into physical learning spaces with our learners, we should also consider that this may not be the case.  The pandemic is far from over, and also, as we know, cost-saving has always been something that clients of learning providers have always been particularly keen on. Having realised that learning can be done remotely, they may wish to stay there.

Many of the things discussed above could, of course, be facilitated via Zoom or similar – and many of us have discovered that platforms such as Zoom, Miro and so on give us things to play with that face-to-face doesn’t – building escape rooms in Zoom, for example, can be done by creatively using the settings of the software itself.

But another and very different proposition is the asynchronous learning experience.  How do we debrief learning (games) which we have implemented as lessons on Mighty Networks or Teachable. Is ‘debriefing’, possible in this case.

While this situation obviously does not allow the spontaneity of a debrief where a facilitator is adapting questions and conversations on the fly, a good compromise is to use all the facilities available in the platform to encourage conversation between learners, and a facilitator can do this by using carefully crafted questions, and by ‘dropping in’ from time to time to prompt. Set ‘assignments’ which incorporate sharing as much as possible, and if possible, ‘gamify’ this participation to reward those who start and maintain peer interaction.

Featuring insightful contribution from learners within platforms, and adjusting subsequent learning activities (and facilitative questions) to indicate that learner input is noticed and valued is a good way of doing this, and some platforms actually have gamification features (allocating points etc.) built into them.

Last thoughts

Although facilitating games-based learning(GBL) does not require different skills than facilitating other kinds of learning, it does present different opportunities for a facilitator. If you have not facilitated GBL before, it is worth spending just that little time more to prepare so that you can ensure that you have taken advantage of that. It is also worth spending sometime becoming familiar with the theoretical basis of games and gamification, because through that you will find much inspiration for the angles you can take in your facilitative questioning.

The post What’s in a Game? – Debriefing Learning Games first appeared on Ludogogy.

An ‘Open World’ Educational game, Liber Domus is currently focused on Grade 6 Mathematics, and is geared to the Portuguese curriculum. However there are plans, not only to cover other subjects and grades throughout K12 – but to regionalise for other countries too.

Educators who are interested in collaborating with Eduardo on these developments, will receive access to the game for their classrooms, and should contact him using the details below.

The post Liber Domus – Interview with Eduardo Nunes first appeared on Ludogogy.

Applying feedback from play testing games, table top games, and educational games is incredibly important. It’s important for multiple reasons. Those reasons extend from shaping the player experience; to connecting learning outcomes; to making sure that the experience stays “fun.”

But how exactly do you collect feedback? How should you interpret it? How should it be used to shape your game design? How can feedback from play testing be used to improve serious games?

This article will review different areas of player feedback from play testing. It includes the steps for collecting feedback as well as how to specifically ask for actionable feedback from your playtesters. It includes the top three questions that I ask from all of my playtesters as well as common interpretations for the kinds of feedback that you’ll receive. The article will cover how to consolidate your feedback from playtesters as well as how to apply it. Review of the feedback process will be provided along with limitations of the entire feedback process.

Steps to feedback

Getting feedback from your game designs is one of the most important steps in the design process. But knowing how to get that feedback and then using it to improve your design can be challenging and daunting.

That’s why getting feedback from your play tests include at least two steps. The first involves interpreting the kind of feedback that you’ll get from your playtesters. Receiving information from your playtesters is part of the playtesting process. But often, data alone isn’t that useful. So knowing how to interpret your feedback is a skill all by itself.

The second step is applying that feedback. Sometimes players will say one thing when they mean another and expect you to do something else. Sometimes your playtesters will be experienced designers who can provide some really actionable and incisive information. But most of the time your playtesters will be other players; casual players; non-designers; or sometimes non-gamers. So it’s important to link the kind of feedback that players provide with your interpretation in order to provide some applicable steps towards improving your design.

How to ask for feedback

Asking for feedback from players can be tricky. This is especially true if you’ve never held a play test before. But it’s an important step in growing as a designer; educator; and serious games designer. Asking players for feedback allows us to zero in on their experience in order to help us tailor exactly what we want it to be.

Table top games are unique in this regard because they are a very social experience. That means that gathering feedback from your playtesters cannot only be about gathering data and putting numbers into columns. Instead, it also has to be about connecting with your playtesters in order to gain a true level of the emotions, fun, and motivation involved.

That means that as the tester and the designer, it’s best to guide your players into the experience of what the game is and what you hope to get out of the play test. Welcome them and allow them to provide any and all types of feedback during the play test.

Top three questions

Often, one of the safest questions to ask your playtesters about your game is if they “thought it was fun?” This is good question to ask to support your own ego; but most of the time the responses to this question are not that actionable. So, if you’re just starting out playtesting this is a question that you can lead with. But, you often won’t get very effective information from your playtesters’ responses.

That’s why I’ve evolved from this single question into asking three specific questions from each of my playtesters. These three questions allow me to extract as much actionable information from the play test as possible. That information allows me to improve on the design. Playtesters’ feedback allows me iterate quickly so that I can get the game to where I want it to be.

Those three questions are:

-What was FUN about the game?

-What is the ONE thing that you would KEEP In the game?

-What is the ONE thing you would REMOVE from the game?

I don’t ask playtesters if they thought the game was fun. Rather, I ask them something more specific: “What – if anything about the game – was fun?” If they say “everything” then I ask them to be more specific. Getting down to this level helps me understand what is really driving the player experience. Playtesters’ answers here will also help me determine what will keep players coming back to this game.

I then ask playtesters that if they could keep only ONE thing in the game what would that one thing be? This is different from the fun question because it requires players to think about the ONE thing that they would keep in the game IF they had to remove everything else. It forces them to prioritize what they thought was the most engaging, endearing, or connected part of the game.

Finally, I ask them what is ONE thing they would remove from the game. This helps me determine if there is anything that is “getting in the way” of the player experience. Then, I determine if that “thing” is auxiliary  and ask myself : “does this need to be part of the design?” If it doesn’t then I have a good reason to cut it.

Feedback interpretations

You’ll often get many different types of feedback from your play testing sessions. While players will often say one thing, what they mean is often completely different. So here are some of the most common pieces of feedback that you’ll receive and some interpretations of what they could mean.

If players say that the game is too long or that they game isn’t fun then engagement might be an issue. Are they doing anything on other players’ turns? Where is their attention when it is their turn? Are they given enough decisions during the game? Or not enough decision? Really, boredom often comes from a lack of engagement.

This can often come about if your players say the game was really slow or if there was a lot of down time between turns. This means that players might not have enough to do on their turn. The game becomes bogged down because of that. Likewise, players could say that aren’t enough “cool” things for them to do on their turn. That could mean that they aren’t given enough interesting options when their turns do come around.

Sometimes your players might say that their decisions felt scripted or that there was no unique outcome for them. Or that there was only one “right” choice. In that case it could mean that your players aren’t given enough unique options when their turn comes around.

You could also get some serious or lighter pieces of feedback. If someone says that they don’t like your art, then that’s something that is often not that actionable. That’s because the version of the game they are playing is often a prototype without finalized components.  Likewise, a player could say that they feel frustrated that they can’t “catch up” in the game. This could mean that there is a serious flaw in your game’s core loop that needs to be addressed.

Consolidating feedback

Now it’s time to make a list of changes to be implemented in your game now that you have some feedback.  Some of those changes can be smaller ones. Those include changing the values of a single card; changing the way that turn order is selected; or re-arranging the phases of play.

These represent some minor issues the can be fixed prior to your next play testing session. With enough time, you can even make these changes during your current play test with your players. You can then start a new game with these new updates.

Applying feedback

You can apply feedback in one of two ways. If your schedule permits it you can “play test” with yourself by playing multiple different opponents.  This will also allow you to reveal any discrepancies with your implemented new changes. Otherwise, you can wait to meet with your playtesters again to implement those new changes in a new play testing session.

No matter how you make those changes you should do so with only one major revision at a time. This will help you determine what changes are having what effect on the game and player experience.

You can take this one step further and test the change in your game at least 10 times before making additional edits. This ensures that your new revision has a lasting change that will affect the game positively.

Based on these new changes you should organize and summarize your information. What was the feedback that you got from your playtesters? What did you think that feedback meant; what change did you make; and what were the results of those changes?

Ideally you should setup your next play test with the same group of playtesters from last game. This will help provide a continuous development overview of your game. Your players will also be able to see how your game has progressed from your last iteration.

Feedback in review

Getting feedback from your playtesters can often be a very incisive and critical process. But it’s important to not take the feedback of playtesters personally. They are criticizing your game and not you as a designer. Part of becoming a successful designer is gathering; interpreting; and implementing the feedback from your playtesters.

It’s important that you get feedback from as many different playtesters as possible. Sometimes you can go back to your own play test group; but it’s also good to be able to branch out and get new insights and feedback. Don’t let one play tester dominate how your game is designed. At the end of the day; this is your game and your design.

Limitations

Feedback will always have its limitations. If you want to play test a social deduction game with players who don’t like social deduction games, then you’re going to have a bad time. Often, nothing you can do will help them like your game.

Also, know that different playtesters of different backgrounds will provide different perspectives and feedback for your game. Casual gamers are different from hobby gamers who are different from game designers; and academics. Knowing how to gather your feedback and how to apply it are all part of becoming a successful designer.

Takeaways

This article reviewed different areas of player feedback from playtesting. It included the steps for collecting feedback as well as how to ask for specific and actionable feedback from your playtesters. It included the top three questions to ask from your playtesters as well as common interpretations for the kinds of playtesting feedback that you’ll receive. The article covered how to consolidate your feedback from playtesters as well as how to apply it. Review of the feedback process was provided along with limitations of player feedback.

This article was about player feedback in table top games testing. To learn more about playtesting in gamification, check out the free course on Gamification Explained.

If you have enjoyed this article – consider getting yourself lifetime access to his Games-Based Learning Digital Library containing all of the content from the past two Games-Based Learning Virtual Conferences; past webinars and courses he’s created; as well as his complete back catalog of articles; podcast episodes; and videos. And more content is being added all the time.

Readers of Ludogogy can get a $50 discount on this valuable resource by using this link.

The post Applying Feedback in Learning Games first appeared on Ludogogy.

This is the second part of an article published in the previous issue of Ludogogy about “Changes” , where we described Pizza KATA, a game designed to help people experimenting with a mindset on continuous improvement, which is a combination of Retrospective and Changes (now it is clear why this article spills over from “Changes” and arrives in “Debriefing and Feedback”).

In this second part we focus on a particular set of components of the game I called mini-boards. This is where the game is played but actually also where players decide how to implement changes discussed in the retrospectives between two “days” (rounds). Pizza KATA is a game played in different rounds (called “days”) and players are able to change strategy every day to achieve better results, like in real teams adopting Scrum.

The line of work of the restaurant

Mini-boards are the players’ world for the duration of the game

In traditional boardgames, you have a board where players interact with the system. However, in card games it is pretty common to place the cards directly on the table. This could work perfectly but for Pizza KATA we have a requirement: make the game both realistic and easy to set up. So I visited a real pizza restaurant for few hours over a couple of days and I was able to design a (simplified but realistic) workflow using a process we usually call Value Stream Mapping.

This image is interesting because other than the flow, it shows a couple of “dependencies”: the preparation of the ingredients (1) and the preparation of the boxes (5). Other than that, there is a starting stage (0) In Pizza KATA this is the source of orders and gives pace to the session. We have a very direct flow to last stage (6) where the pizzas are boxed and delivered to the customer.

This is the reality of the game and you cannot change it, but you can adapt and optimise your effort to maximise the attributes that are relevant for you: number of delivered orders, response time, minimise ingredients, maximise turnover,…

This reality is implemented through a set of mini-boards: each one is a step in the flow and has its intrinsic rules to perform activities. As an example, here you can see the “Kitchen”

Note. The kitchen actually uses two mini-boards and I hope you remember the tokens described in the previous article.

From now on, the images shown were taken working on a real order in a play session, and you will be able to follow it across all the stages, so you can see the game in action with all components. Notice the order is a “type III” order with five pizzas of four different types (see previous article for explanations of different types of orders).

Receiving orders

Phone calls are coming at a the pace of your main timer (or hourglass as I prefer).

The number of calls is defined by the number of order cards selected at the setup of the game (as described in the preparation of the order cards in the previous article).

Everything in the process is based on these incoming calls; and below you can see the mini-boards which manage them.

Here you can see the last received order of this session, which will arrive when the timer has finished.

Nothing really critical here, except that you could record timing using one of the tickets described in the previous article

Preparing ingredients

If you have ever cooked using a recipe, you will know how much time you must spend to prepare the ingredients. For the pizza it is the same, except that here time is critical and ingredients could be prepared in advance, or maybe by another player (this is a team decision), in the kitchen where there is a refrigerator and everything is safely stored.

The position of ingredients is pretty clear, the main rule is that you cannot mix ingredients that must be separated in their own area. This is by design to help you limit Work in Progress (WIP). However there are some icons which require explanation.

You will notice that each of the named food areas has a small circle with a number. This is to remind you that to prepare the ingredients you have to flip the tokens the given number of times, before placing them in the area. For example if you need one portion of onion you have to flip it five times; a portion of olives require three flips, ham just two and so on. The flip icon is different for tomato sauce (1+) and mozzarella (2+). Here the circle reminds how many flips you must do, but also that you can flip them in groups. This is faster, but also requires closer attention. This is a decision that a chef should make!

The dice icon is related to blockers that may come with events which occur when you draw an event card. These can be shuffled in with the orders.

Preparing pizzas

Preparing pizza is not easy, as you can imagine, even if you have prepared ingredients in advance. Pizza KATA is a fast paced game where you will receive an order every minute and so the queue of pizzas to be prepared sometimes gets large (and stressful).

The pizza preparation station is organised over two mini-boards: one to collect ingredients from the kitchen which you can use to prepare the pizza and another where you prepare the crusts on which to place the ingredients following the recipe.

The ingredient boxes is similar to the kitchen but has an important difference. Ingredients placed here cannot be boxed back in the refrigerator for safety reasons, so you need to move the right quantities of ingredients to avoid penalty points at the end of the game. That’s why the team will need to calculate the resources needed, and not simply add stuff without planning.

Then you have the “bar” where pizzas are prepared.

In the image you can notice that a player can prepare no more than six pizzas at the same time (there is a physical constraint on the bar). For this particular order, we have to prepare five pizzas and it is not a problem, but sometimes…

To prepare a pizza you need to first manipulate the dough. To prepare the dough you need to take the card and flip it on the dough side; then use you hand as you would do in reality: you need to stretch each corner twice and at the same moment rotate the dough for the given number of times (two or three). When you have done, you can again flip the card to show the areas where you can place ingredients and, eventually, the extra cooking marker.

The below image shows this stage with the five pizza order (notice the Napoli preparation).

When they have prepared the pizzas, the player must put them in the oven for cooking.

Cooking pizzas

The oven is a limited space where you can cook no more than 3 pizza at once. You can buy a bigger oven during the game but this has a cost and also requires a certain level of turnover to pay it back. The team has to think carefully about that.

In the below image, the player decided to put two Napoli and one Margherita in the oven (maybe not the best decision, but this is part of the learning curve with the game).

As you can see, next to the pizzas, you put a timer (the hourglass) to measure when they will be ready. Again note the Napoli pizza requires a specific timing for placing mozzarella….

Here the situation after the first timer: the Margherita is cooked and has been removed. The two Napoli have instead had extra mozzarella added and are ready for extra cooking time.

The chef decided to place the second Margherita in after removing the first one: again maybe not the best decision in terms of flow, considering the extra cooking required for the Boscaiola. However, this has to be managed. Meanwhile a new order has arrived and so you need to prepare other crusts… There are a lot of points to debrief for the team.

Serving the pizzas

If you ever been in a pizza takeaway you would have noticed that the pizza is placed on a plate before closing it in the delivery box. The pizza, when it is hot, must “take a breath of fresh air” to avoid the ingredients taking the taste from the cardboard of the box (this was revealed by a pizza chef!). The problem is finding the room to make this happen. That, in a takeaway restaurant, is often very limited.

Another important activity in this stage, is checking the order is properly managed, and that all the  ingredients are there so we can meet the expectations of our customers.

As you can imagine this is going to introduce a new constraint into the system, but this could also represents a buffer before the boxing process. Given this situation, I have configured the mini-board so that a standard oven is normally the bottleneck, but if you add an extra oven (yes, you can buy a second one) the bottleneck is moved to the serving stage (yes! This is simulating when you are increasing capacity in the bottleneck stage, without considering the impact on the whole system 🙂 ).

Let’s see how the transition between the cooking and the serving is implemented in the game.

Here you can see what happens when the first pizza in our order is (already cooked and) placed in the serving stage. Note this order is spreading across 3 stages, because there is still a pizza on the pizza bar (the oven is full).

The next stage (after one minute of the hourglass) is the following

As you can see now the pizza bar is free (a new order is probably already received), the oven has one pizza and the serving is full. We need to free space in this last stage and to do so we need to box the order.

Boxing the order

To box your order you need a new skill: you need an origami maker! (any thoughts about T-shaped people in your team?)

The last mini-box is the set of instructions to create the perfect box to collect and safely transport our order. This is a 12 steps process that uses a (recycled) A4 paper sheet.

When the box is ready, you can put the order card inside along with its related pizzas and ingredients.

In the above image you can see the four pizzas were in the serving area, now placed in the box. We have the possibility to serve the fifth pizza, that will be boxed (with the order card) in the next time slot.

This is the last stage of the process. Now you can deliver the pizzas, you can mark the order as completed and collect times in the provided ticket.

But a new order is very likely already in the line and more orders will come in the next minutes….

The importance of Kanban Principles

Mini-boards are designed for the players to experience true Kanban

As you can see in this scenario there are a lot of pitfalls but also opportunities to improve, and this is the reason why this game works. But you can also see the real time effect that is the basis of the game. This creates a really fast paced situation which is hard to manage but also fun to face. Finally, given that a day is 10 to 15 cards, you have 15 to 20 minutes each round, so you can play different days experimenting with all the Kanban Principles:

• Visualise the work: mini-boards are designed to visualise the work.
• Limit WIP: some mini-boards have explicit limitations.
• Manage Flow: you can follow and measure flow of the system on the mini-boards.
• Make Process Explicit: the game rules are transparent and visible. The team can define their own strategy.
• Implement Feedback Loops: even if the customer is not visible, you need to manage a budget.
• Improve Collaboratively, Evolve Experimentally: you have tools to collect data to analyse and improve by taking informed decision and verifying them.

How many people does it take to prepare pizzas?

People playing the game is the main goal of every game designer

This is the first question a facilitator could ask to people in the workshop: how many people can manage a pizza restaurant? Now that we know how the assembly line is composed we can understand how many people could manage it. But we need to consider another attribute of the game: the turnover. In fact the goal of the player is not only to deliver all received orders, but achieve this result with the minimum number of people, because the value delivered must be divided by the number of people that touch the line.

Maybe we can start from a real pizza takeaway where normally there is one person to prepare and cook pizzas and a second person doing all other the activities (receiving calls, preparing ingredients, preparing boxes and boxing pizzas). In this way we have the basic approach.

There is another person required in the game: the timekeeper who manages the pace of incoming orders and tracks all metrics using the tickets described in the previous article. This observer cannot interact with players or components or any artefact of the game. They just track times on tickets, or maybe on a board where the system can be represented. This is not an easy task and the value can bring is massive, so do not underestimate this role. The timekeeper is not to be counted in the turnover if they didn’t touch any component except the tickets.

Clearly, team composition could be different. Maybe you could have two pizza makers, or one person in the kitchen dedicated to ingredients, or any other way of working that the team themselves decides to adopt to be more effective. And this situation reinforces the principle that the team is empowered for  self-management and self-organisation. That is a strong attribute of real agile teams (at least four principles of the agile manifesto touch these topics).

In summary we need at least three players (two + timekeeper), and my suggestion is to play Pizza KATA with no more than six players per restaurant: one time keeper, one observer of the behaviours, one or two pizza bakers, one in the kitchen, one for reception and boxing.

What if we have to manage workshop for 10 or 20 people? These are too many for a single pizza shop, so we need to scale the game. Pizza KATA provides two different opportunities. We can create a competitive situation or a collaborative situation.

The competitive scenario is where you have different pizza restaurants with their own deck of orders (that maybe the facilitator can prepare in the same way), starting together, same pace, same ending and at the end of the day we can compare final result (value, orders, pizzas, rounds, metrics, ….). This is the scenario that players normally prefer.

The collaborative scenario is where you have one deck of orders and many assembly lines (like a big pizza franchising brand), starting and ending together and managing orders in parallel. This is the scenario I prefer because teams have to deal with real scaling dynamics and understand also how to work together as a team of teams.

The number of components required to manage a large scale session is based essentially on how you are scaling. Pizza KATA in fact lets the facilitator decide how to scale, and essentially this process is related on how to scale the WIP (not the people). So we can understand why scaling could be not directly related to teams but to the number of mini-boards used in the session. All rules for scaling are available in the facilitator guide. What we want to highlight now is that the facilitator could apply different approaches to scale the game. This is based essentially on the outcome of the session, but this possibility enables the facilitator to create different sessions with different objectives, simply adapting the scenario.

Having a Print&Play game is an enabler for scaling components, because you have all the material in the package, you need just to print multiple copies of what you need. It also lets the facilitator be creative in  experimenting with new scenarios that can be shared with other facilitators.

Retrospective of the day

People have the opportunity to improve every day, also in the game

Do not forget that this is a serious game and the goal is not only having fun but letting people experiment with how to improve their ways of working given a well defined goal. So we need to consider in the structure of the workshop, to include moments to analyse behaviours and change the processes in the restaurant. This is, IMHO, the most important moment of the session and it is strongly connected with the idea of the Kata. That’s why this is in the name of the game.

Retrospectives should happen at the end of every “day”, when the team discuss about what happened in the last day and take decision on what/how to change to improve. The time for discussion I normally recommend is 15 minutes, which is the duration of the “day” in the game (with 10 order cards).

I want to underline the role of observers. These watch the dynamics of the players and take notes during the session so they can share during the retrospective. Given the external perspective, they are in the best position to help the team to understand what worked and what didn’t.

Finally remember that playing this game without a retrospective for the team is useless (except for the fun, but maybe in that case it is better to play a ‘real’ board game).

Conclusion

Also the best games must end

We have initially identified some reasons why the existing pizza kanban game (a great game!) is not fitting for all situations and we have also identified some requisites an eventual new game should have. We could summarise this as “Before and after the execution, focus on the outcomes and not on the preparation. During the execution, focus on fun and not on boring stuff.” This goal has been achieved adopting a “board game design” approach and introducing game mechanics (components and rules) to enable a different level of game dynamics (player(s) interacting with the game). That’s why at the end this is an original game.

The game is articulated in different “days” where every team tries to improve its performance and increase performance and predictability of the activities to deliver pizza take,away: from order receiving to boxing.

This specific set of components, to Print & Play and reuse, helps facilitator to create the game session and at the same time expands to a lot of new possibilities. In fact, Pizza KATA has a deck of order cards and mini-boards which are able to create an infinite number of different sessions, but above, we define a few rules so the facilitator can create custom sessions focused on a particular situation the team should be able to deal with. We have also described how the game could scale creating situations where multiple teams compete or collaborate.

Pizza KATA has been created as a mindset improving experience. This emerges from the name (the word “KATA”), the way the game has been created (different experimental rounds), but also from the roles (the timekeeper) the mechanics (every “day” has a retro) and the components (the time tracker tickets). Everything has been designed to help the team focus on the real goal of the game: “having fun while experimenting with the effects of continuous improvement”.

Finally, the result is based on a financial perspective similar to our daily life where we do not have infinite resources. The team can buy a larger oven but that impacts their turnover. These are not constraints or limitations but enablers to achieve the best results possible in this scenario, with limited resources like our teams have to face in the real world.

I hope you enjoyed these articles on a game that essentially is about change and retrospectives. The last piece of information is the link to download the game: Pizza KATA, with all other P’n’P games I have designed, is available on DriveThruCards portal at the following link https://www.drivethrucards.com/browse/pub/17909/Agile-Game-Factory

DISCLAIMER. Notes presented here are personal and cannot be related in any way to any of my employers.

The post Pizza KATA II Retrospect is a Mindset and Not an Action first appeared on Ludogogy.

Alyea sent us this video looking at how she uses Play Personalities (from the work of Dr. Stuart Brown, at the Institute for Play) in her gamification work.

If you want to take the quiz mentioned in the video, click https://tinthue.com/play-quiz

The post The Play Personality for Gamification – Alyea Sandovar first appeared on Ludogogy.