One of the most effective ways to demonstrate tricky real-world, Agile concepts is to run a game that strips away any complexity and concentrates on demonstrating the concept in an unambiguous and obvious way. As agile coaches, scrum masters and change agents, we often struggle with convincing people about agile mindset concepts, so there’s nothing better than getting out the Lego, or a set of coins or a pack of playing cards and running a game round a table with one or more teams to really hammer home a salient point.
The pandemic, however forced us to think about taking these workshops online, initially with the idea of improving engagement and seeing how close we could get to “the face-to-face experience” (see this article from Kanban Zone. What we discovered, though, was that we could make the de-brief and learnings from the workshops extremely relevant and engaging, and thus more convincing, in an online setting.
Light-bulb Moments
The debrief part of all of these games is designed to give an epiphanising “light-bulb” moment where participants suddenly “get” the idea. However, many of the explanations rely heavily on data, and it is often tricky to calculate this data quickly enough to replay it in the debrief. Hence, we must use either use stock datasets and graphs to illustrate the concept or have a break while we do calculations and draw graphs. The former doesn’t always convince, however, and the latter leads to us “losing the moment” which lessens the impact.
Taking the workshops online necessarily meant writing software, and that gave us the opportunity to do any data analysis in real-time, based on the actual behaviours of the participants. We quickly realised this was much more relevant and convincing. Sceptics can easily argue against stock datasets, but it is much harder to do so when the conclusions are based on their behaviour in a game they have just played.
We saw a great example of this when playing the No Estimates game at the recent Agile 20 Reflect conference. The No Estimates game demonstrates that estimating is futile, as it is impossible to do it accurately – quite a big mindset change for many people! In the game, we give players more information (e.g. the exact number of tasks to do, and the effort required for each task) than they would have in real-life, so they believe – as they are convinced they can estimate 100% accurately in real-life – that they will be able to do the same in the game.
The image below shows, for two of the teams in the game, their initial estimate, their estimate for the first 11 tasks, and the final number of days it took to do the. As you can see, even though both teams took the same amount of time – 40(ish) days – and had exactly the same information to start with, their initial estimates were wildly different.
Showing this information is much more convincing than saying “typically, people’s estimates are up to 50% out”, and is the first step towards dismantling participants belief in how accurate their estimates actually are.
The reason that estimates are always wrong is that people make many, many assumptions when estimating. Merely pointing this out never convinces anybody, but if we use the actual game data, we can demonstrate this very powerfully. As an example, there is an implicit assumption that bigger tasks will take longer. In the game, we can demonstrate this is not the case by showing the correlation between task size and time taken, as for one of the teams here:
A correlation of 1 means the bigger a task, the longer it will take. As you can see, this is far from the case; 0.49 is quite low. If we plot a distribution of the size of tasks, and how long they took, we will often see very small tasks taken longer than very large tasks. Again, showing this for the actual work the team has just done in the game really highlights this incorrect assumption.
In the face of these challenges to people’s prowess in estimating, one thing they often say “but we know there are variations, so we use an averages and standard deviations to give a range”. Unfortunately, to do this kind of statistical analysis relies on a normal distribution of data, and this is also an incorrect assumption. The game data comes to our rescue to demonstrate this as well. Here is the distribution of how long tasks took for two of the teams in the game
As you can see, neither is close to a normal distribution, so using mean and standard deviation is invalid.
There are many other graphs and datasets we show to highlight incorrect assumptions, but we finish the game with this real light-bulb:
This is a monte carlo simulation showing how long it will take to do N task in the future. It allows us to answer the question everyone wants answering when estimating – “when will it be delivered”. This is the real light bulb moment; given how you just performed in the game, we can predict when 100 tasks will be completed.
And this is the real power of using the real game data. Using the actual data that players have generated themselves, we have not only come to objective conclusions about their incorrect assumptions, but we have also given them tools to rectify the situation that they can use in the real world. We have come full circle; we started by stripping the situation right back to isolate the concept in a game situation, and have ended by demonstrating concepts and techniques that are applicable in the real-world.
We can see the contrast when using real data if we look at another game we implemented – The Coin Game – we haven’t yet implemented the real-time data features, so we just show the graph we usually draw in the face-to-face version of the game. This shows the amount of value delivered (y-axis) over time (x-axis).
This does get the concept across, but how much more convincing would a graph be if it showed the actual amount delivered by participants over time? We could even show the curve developing in real-time as the game is being played to really ram home the point that it is always better to deliver the next highest value items first.
One of the main reasons we brought workshops online was that they are really improved if played in a number of teams; the competitive element gets participants concentrating on the game rather than thinking too much about the concept, and this makes the light-bulb moment even more of a reveal! Break out rooms allow us to re-create this multi-team atmosphere online, but it also allows us to take the concept further; teams can play asynchronously – in different time zones, on different days – making scheduling easier, and, with some games, we can trend performance and scores over time to demonstrate improvements and changes. Again, all this is made possible through the recording of game data.
Online workshops have really given us an opportunity to use real data, in real-time, to convincingly demonstrate complex concepts to often sceptical participants, and has revolutionised how we present our learnings and feedback.
Check out Corrado’s Ludogogy author archive
Be the first to comment