Simulate it,
test it, play it again
Simulation and testing in game design: balance, QA and playtesting
BSc (Hons) Game Design & Development
Name a bug, an exploit or a broken strategy you found in a game.
How did a whole studio miss it… and how did you find it?
By the end of today you can…
Separate simulation from testing
and say which questions each one can answer.
Simulate a mechanic
with a Monte Carlo run, and read its average and its spread.
Pick the right test
functional, regression, usability, balance… and triage what it finds.
Run a fair playtest
with a question, a script, consent and notes that turn into fixes.
Six stops
Simulation vs testing
Two ways to find out, and the design loop
Simulating balance
Monte Carlo, live, on Roll for Reward
Types of testing
QA, test types, triage, tools
Playtesting
Who, how, the golden rule, notes
Ethics & bias
Consent, inclusion, honest questions
Test lab
Simulate, change a rule, playtest, triage
You have been testing all along
You measured flow with a questionnaire, predicted results, and compared groups.
Roll for Reward's scores were checked by simulating 40,000 games per round before you ever played it.
Atoms combine into loops that emerge: behaviour you can't predict just by reading the rules.
If you can't predict a system by reading it, you have to run it: with a model (simulation) or with people (testing). Usually both.
Simulation
vs testing
A model playing the game, or people playing it
Run the model, or run the game
| Simulation | Testing | |
|---|---|---|
| What runs | A model of the rules: a spreadsheet, a script, bots | The real build (or prototype), with people or test code |
| How often | Thousands or millions of times, in seconds | A handful of sessions, in hours |
| Answers | What happens? Averages, spreads, edge cases, broken economies | Does it work? Do people understand it? How does it feel? |
| Blind to | Fun, feel, confusion: anything the model leaves out | Rare events that need thousands of plays to appear |
Simulation before you build it; testing once you have something to hold.
Rules you can't predict by reading
Interactions multiply
Five systems that each look fine can combine into one that doesn't. Emergence cuts both ways.
Randomness hides
A 5% crit, a 1-in-20 drop: ten playtests can't show you what happens over a season.
Behaviour adapts
An AI that reacts to the player changes every run. You need many runs to see its range.
Therese's example: tune a racing game's AI difficulty curve by simulating thousands of AI laps and looking at the spread of lap times, not just the average. A rival who is fast on average but wildly inconsistent feels unfair.
What designers simulate
Physics
Ragdolls, ballistics, vehicles: stable, believable, not exploitable.
bullet drop · car handlingAI behaviour
Pathfinding, NPC routines, directors that pace the action.
Left 4 Dead's AI DirectorEconomy
Resources in and out over time: inflation, scarcity, grinding.
strategy and management gamesPlayer models
Bots of different skill playing the game to predict outcomes.
difficulty curves · win ratesEmergent systems
Ecosystems, crowds, procedural worlds: stress them to find the edges.
The Sims · No Man's SkyEach one connects back to a core loop and to how the player feels.
GDnD wiki · Economy Simulation with QueuesTesting isn't the last stage
Therese's loop: prototype → simulate → test → evaluate → refine → re-test. It runs from paper prototype to launch, and beyond.
Bond's version: analysis → design → implementation → testing, repeated. Each loop answers one clear question.
"Game design is 1% inspiration and 99% iteration." Chris Swain, quoted by Bond
Simulating
balance
Play the game ten thousand times before lunch
Roll it ten thousand times
- List the random events in the mechanic
- Generate them:
=RANDBETWEEN(1,6),=RAND() - Work out the result of one game with
=IF() - Repeat for thousands of rows
- Summarise:
AVERAGE,STDEV,MIN,MAX,COUNTIF
Roll 2d20, keep the higher. Simulated: worth about +3.3 on average, roughly the old +3 bonus.
To get ~10 per 100 attacks needs a 57.5% hit rate, and individual rounds still range from 0 to 29.
When should you bank?
A model only knows what you told it
- Averages, spreads, extremes
- Dominant strategies and runaway economies
- Progression and XP curves for players of different skill
- Rare events: the 1-in-1,000 soft-lock
- Whether it's fun
- Whether players understand the rule
- What players actually do (bots play the way you coded them)
- Anything you left out of the model
In Week 3, banking barely changed the score. Players still argued about it for ten minutes. That is what a playtest finds.
GDnD wiki · Game BalanceTake a mechanic from your Week 3 or Week 4 design.
What number, rate or spread would you want to know before building it?
About understanding, feeling or choice.
Types of
testing
Does it work? Does it still work? Can people use it?
Different questions, different tests
| Test | Asks | Typically |
|---|---|---|
| Smoke | Does the new build start and reach the main loop? | Every build, minutes, before anyone else gets it |
| Functional | Does each feature do what its spec says? | Per feature, against acceptance criteria |
| Regression | Did the fix break something that worked? | After every change; automate it |
| Exploratory | Can a skilled tester break it? | Unscripted, by experienced QA |
| Compatibility & performance | Does it run on every target device, at frame rate? | Hardware labs, profilers |
| Usability & accessibility | Can players learn and use it, whatever their needs? | Observed sessions, guidelines checks |
| Balance | Is any strategy dominant? Is it fair? | Simulation, expert play, public test servers |
How bad, and how urgent?
Severity: how bad it is when it happens. Critical → high → medium → low.
Priority: how soon to act. P1 now → P4 after launch.
| Bug | Severity | Priority |
|---|---|---|
| Crash on the main menu | Critical | P1 |
| Crash in a debug-only scene | Critical | P3 |
| Logo wrong; press copies go out tomorrow | Low | P1 |
| Big bug in a feature cut from this release | High | P4 |
You'll triage your playtest notes the same way in Part D.
Let the machine do the boring runs
Unity Test Framework
Edit Mode and Play Mode tests in C#.
unit + in-game checksUnreal Automation
Automation tests run from the Session Frontend or the command line.
functional + smokeAutomated regression
Every build runs the test suite before anyone plays it.
continuous integrationSpreadsheets
Weapon tables, XP curves, drop rates, Monte Carlo.
Sheets · ExcelRemote playtests
Recorded sessions with target players, at a distance.
PlaytestCloud (mobile)A/B tests
Two versions to two groups; compare what they do.
Firebase Remote ConfigPlaytesting
Don't tell. Watch.
What you designed vs what they played
What's in your head and your design doc: the rules, the feeling, the "obvious" solution.
What a player does and feels with only the game in front of them. MDA's gap, in Week 2's terms.
Runs the session: script, setup, notes. Ideally not the designer, who sees what they hope to see.
Plays. Doesn't know your hypothesis. Their confusion is data.
Move outwards as it matures
You find the obvious. Friends are kind and fast. Acquaintances and target players give the truest picture. The public is wide, noisy and permanent.
Someone who has never seen it. Use them once, for first impressions, tutorials and surprises. Plan who sees what, and when.
How many? Nielsen's rule of thumb from usability work: five users find most of the problems. Test small and often.
Six ways to ask
Silent observation
Watch what they do, where they look, where they stall.
best for: confusion, flow breaksThink-aloud
They say what they're thinking as they play.
best for: rules, mental modelsSurvey
Likert scales after play: compare across sessions.
best for: trends, fairness, funInterview
Short, open questions straight after the session.
best for: why, memorable momentsA/B test
Two versions, two groups, one difference.
best for: choosing between optionsWizard of Oz
A hidden human plays the "system" before it's built.
best for: AI, dialogue, matchmakingDon't explain. Don't defend.
Their confusion is the data.
✗ "Oh, you're meant to bank there, it's safer."
✓ "What are you trying to do right now?"
Before, during, after
- One question you want answered
- A script: what you'll say, how you'll answer questions
- Consent, setup, notes sheet
- Silence. Watch hands, eyes, face
- Timestamped notes: confusion, frustration, delight
- Write what they did, verbatim
- Survey, then a short interview
- Separate symptom from cause
- Prioritise by how many testers hit it
Players are good at diagnosing symptoms, poor at prescribing solutions. Collect the first; design the second.
Write it down before you explain it away
| # | Where | Verbatim (what they did or said) | Underlying issue | Severity | Proposed fix |
|---|---|---|---|---|---|
| 1 | R5 turn 2 | Paid 2 ducks to cancel a bust on a pot of 3 | Insurance cost not weighed against pot | Medium | Show "pot at risk" on the sheet |
| 2 | R5 turn 4 | "Wait, can I insure the bank too?" | Rule unclear | High | One-line rule: pot only |
| 3 | R5 end | Laughed when another player insured a 1-point pot | A funny, social moment | As designed | Keep |
Severity: High · Medium · Low · As designed · Fact of life (can't be changed). Verbatim first; interpretation second.
Numbers say what; words say why
Completion time, success rate, deaths per level, Likert scores, how many testers banked on turn 1.
Frustrations, enjoyment, quotes, what they tried, what they expected to happen.
Together: "4 of 5 testers failed the first wall jump" (what) + "I didn't know I could climb that" (why) → the fix is a signifier on the wall, not easier timing.
Next week: the same idea at the scale of thousands of players, with telemetry and analytics.
GDnD wiki · TelemetryThree games, three kinds of test
Balance in public
Hero changes go to a Public Test Region first, where players try them before they reach the live game.
An AI you have to run
The AI Director changes spawns and items to the team's state. Fair-feeling tension only shows across many runs.
Too big to play
About 18 quintillion planets. No team can visit them; procedural systems need sampling and automated checks.
Which one leans on players, which on simulation, which on automation?
GDnD wiki · AI Experience ManagementEthics
& bias
Testers are people. Data can lie.
Protect the people who test for you
Informed consent
What you'll record, why, and that they can stop at any time without giving a reason.
Privacy
Player IDs, not names (as in Week 2). Store notes safely; delete what you don't need.
Inclusion
Recruit beyond your friends: different abilities, experience and backgrounds.
Care
Warn about flashing lights, horror or motion sickness. Under-18s need extra safeguards.
Class playtests are for design feedback. If you want to use testers' data in a dissertation or publish it, that is research, and it needs approval through the university's ethics process first.
Ask so the answer can surprise you
"Didn't you love the insurance rule?"
"Tell me about the insurance rule."
"Was it too hard?"
"The difficulty felt fair. 1 = disagree … 5 = agree"
Testing only with your own team, once
New testers each round; several small rounds
Keeping the quotes that agree with you
Count every observation; let a teammate interpret
Five things to remember
Simulation runs the model; testing runs the game. You need both, early and often.
Read the spread, not just the average. Monte Carlo shows how a random system will feel.
Match the test to the question, and triage by severity and priority separately.
Don't explain, don't defend. One question, a script, verbatim notes, symptoms before causes.
Consent, inclusion, honest questions. Testers are people, and data can lie.
Test lab
Simulate it, change it, test it, triage it
Simulate by hand, then by machine
- Odd-numbered seats never bank. Even-numbered seats bank at 12.
- Play three solo games of Round 3 (Week 3 rules, 6 turns each).
- Write your three scores on the sheet, then call them out.
- Compare the room with 10,000 simulated games.
Kit per table: two d6 (main + modifier), a d8 or d6 + 2, duck tokens.
–
–
Simulation (10,000 games): –
Design Round 5: one rule change
Change or add one rule to Round 3. Write it in one sentence another group can follow.
What do you want to learn? About feeling, choice or understanding, not just score.
What do you expect testers to do and feel? What would prove you wrong?
Ideas: a second pot · a duck you can spend · banking that costs a duck · a "double or nothing" roll · a shared pot with the player to your left.
Round 5: Insurance
| Rule | Once per round, before you roll, you may pay 2 ducks to insure your pot: if you bust, you keep it. |
| Question | Does insurance make players push their luck more, and does it feel like a smart choice or a tax? |
| Prediction | Most players insure big pots late, then keep rolling. Some will feel ducks are "too precious". |
| Wrong if… | Nobody insures, or everyone insures on turn 1. |
"Thanks for testing. We're testing the game, not you. We'll take notes and a short survey, no names. You can stop at any time. We can't answer rule questions while you play: if you're stuck, tell us what you're trying to do."
If asked a rule question: "What do you think it means?"
Playtest another group's Round 5
Reads the script and the rule. Then silence. Only "What are you trying to do?"
One row per observation: where, verbatim, issue, severity. Interpretation later.
From the other group. Play one round of Round 5, then fill in the survey.
Then swap, so every group's rule gets tested. Testers: the survey is five Likert questions and one open one.
Triage, then report in one minute
- Count: which observations came up most often?
- Symptom → cause for your top three
- Severity and priority for each
- Was your prediction right? What would you simulate next?
"We asked …. We saw …. The cause was …. Next we'll …."
“In my project I'd simulate …, but only a player could tell me ….”
One sentence each, on the back of the playtest pack.
Bond, J. G. (2022) Introduction to Game Design, Prototyping, and Development. 3rd edn. Boston: Addison-Wesley.
Keith, C. (2020) Agile Game Development: Build, Play, Repeat. 2nd edn. Boston: Addison-Wesley.
Metropolis, N. and Ulam, S. (1949) 'The Monte Carlo method', Journal of the American Statistical Association, 44(247), pp. 335–341.
Nielsen, J. (2000) 'Why you only need to test with 5 users', Nielsen Norman Group.
Schell, J. (2019) The Art of Game Design: A Book of Lenses. 3rd edn. Boca Raton: CRC Press.
Sellers, M. (2018) Advanced Game Design: A Systems Approach. Boston: Addison-Wesley.
Further reading on the GDnD wiki
Probability for Designers (Monte Carlo in a spreadsheet)
Economy Simulation with Queues