CRE342 · Week 8
Ulster University
CRE342 · Game Mechanics & Gameplay Design · Week 8

Reading the player

Game and player analytics: what the data says, what it doesn't, and how it changes design

BSc (Hons) Game Design & Development

Think · 30 seconds

What does your favourite game know about you?

Not your name. Your habits: when you play, where you die, what you buy, when you quit.

0:30 click to start · tell the person next to you
Today

By the end of today you can…

01

Explain what analytics is for

where telemetry fits in the analytics value chain, and why it needs qualitative data beside it.

02

Read the core metrics

engagement, retention and churn, progression, economy, social and technical.

03

Turn a number into a design change

evidence, hypothesis, change, measurement, and an A/B test to check it.

04

Collect data responsibly

with privacy, consent and minimisation, and without turning prediction into manipulation.

The route

Six stops

01

Why analytics?

Value chain, telemetry, what and why

02

Engagement & retention

DAU, D1/D7/D30, churn, lifecycle

03

Progression & funnels

Where players stop, and why

04

Economy, social, technical

Currencies, communities, frame rates

05

Data to decisions

A/B tests, prediction, tools, ethics

06

Practical

Pip's Peak: read the climb

Recap · you've already done analytics

Weeks 2, 3 and 7, in one line each

Week 2

You recorded flow scores, processed them in a class sheet and analysed the result. That is the analytics pipeline, by hand.

Week 3

"Retention on its own isn't the goal: a slot machine has great retention." Today we measure retention, so keep that in mind.

Week 7

Playtesting shows you a handful of players closely. Analytics shows you thousands, from a distance.

Today

Analytics tells you what players do, at scale. Playtesting tells you why. A designer needs both.

01

Why
analytics?

From gut feeling to evidence

Why analytics matter

Every design is a hypothesis

Test

"This level is a fair step up." Is it? Data checks the hypotheses built into every design.

Improve

Data-driven changes improve retention, engagement and satisfaction, on purpose rather than by luck.

Settle

When the team disagrees, a shared number ends a lot of arguments.

"Without data, you're just another person with an opinion." Often attributed to W. Edwards Deming.

And with data but no judgement, you're a person with a spreadsheet.

GDnD wiki · Game Analytics
Su, Backlund & Engström (2021)

The game analytics value chain

Development Player Publishing Distribution channel gameplay · interfacesystem · processperformance player segmentationplayer behaviour acquisitionretentionrevenue stores, platformsand channels ACROSS THE CHAIN · PREDICTION churn prediction · revenue prediction ACROSS THE CHAIN · VISUALISATION dashboards · heatmaps · funnels
Telemetry · the raw material

An event is a sentence about play

// one event, sent when a player falls
{
  "event":   "player_death",
  "player":  "P0417",     // pseudonymous
  "time":    "2026-09-03T19:42:07Z",
  "level":   4,
  "x": 212.5, "y": 48.0,
  "cause":   "wind",
  "attempt": 3,
  "fps":     27
}
Who · when · what · where

Every event answers the same four questions, plus the details you'll need later.

Questions first

Decide what you want to know, then which events answer it. "Log everything" gives you a haystack.

Name events well

tutorial_step, level_start, level_fail, purchase: consistent names make queries easy.

GDnD wiki · Telemetry
Quantitative and qualitative

What players do, and why they do it

Quantitative · objective

Measurable, numerical, often system-generated. Captures what players do.

Dashboards · telemetry · logs

Qualitative · subjective

Descriptive, from the player's experience. Explains why it happens.

Surveys · think-aloud · observation · sentiment analysis

Telemetry says40% of players quit after Level 2.
Playtesters say"The puzzle feels unfair." "The controls are confusing."
TriangulateOne interaction in Level 2 reads as breaking the rules. Redesign that, then measure again.
GDnD wiki · Playtesting
02

Engagement
& retention

Do they play, and do they come back?

Engagement metrics

How much, how often, how long

Session length

Average duration of a play session.

Session frequency

How often players return per day or week.

Playtime per player

Total accumulated time.

DAU · WAU · MAU

Daily, weekly and monthly active users.

Stickiness

DAU ÷ MAU. Around 20% or more suggests a habit (Sellers).

Retention

% of players who return after a set time: D1, D7, D30.

Long sessions aren't automatically good. Ask what players were doing.

Churn · the inverse of retention

Who stopped playing?

Churn = players lost during periodplayers at start of period × 100

1,000 players start. 250 stop playing within a week. Churn = 25%; weekly retention = 75%.

Mobile benchmarks (Therese)

64–72% churn by Day 1. 94–97% by Day 30, so D30 retention is often just 2.5–5%.

"Strong" targets (Sellers)

D1 40%+ · D7 20%+ · D30 10%+

Most players decide in the first minutes. The first-time user experience (FTUE) is a design priority, not an afterthought.

GDnD wiki · Games as a Service
Pip's Peak · synthetic launch-week data · 2,000 players

Retention by cohort

Club members stay 3× longer. Should every player be pushed into a club? Careful: the club only unlocks after Level 3.

Clark (2014) · the player lifecycle

Five stages, three retention checkpoints

Discovering Learning Engaging Earning Churning D1D7D30 first screen, no frictiononboarding, early winsloops, social, reasons to returnfair, optional spendingleave well, maybe return

Compare players by days since install (cohorts), not by calendar date.

GDnD wiki · Player Lifecycle
Integrating churn into design decisions

Churn spikes after Level 3. Now what?

1 · Locate

Where exactly?

Event logs and heatmaps: which room, which jump, which screen.

2 · Examine

What changed?

Death rate, difficulty, UI confusion, unclear objectives, an unskippable cutscene.

3 · Ask

Why?

Cross-reference with surveys and playtests (Week 7).

4 · Re-measure

Did it work?

Tutorial, checkpoint, pacing or reward timing; then A/B test.

The usual suspects: onboarding bottlenecks, difficulty spikes and reward droughts.

03

Progression
& funnels

Read the numbers as a story of engagement

Progression metrics · track advancement and completion

How players move through the game

MetricWhat it revealsIf it looks wrong
Level completion rate% of players finishing each levelSharp drop: difficulty spike, unclear goal, poor checkpoints
Checkpoint frequencyHow often players reach milestonesToo few: fatigue. Too many: no tension
Failure / retry countsBalance and frustrationHigh: unclear tutorial. Low: no challenge, a mastery plateau
Item / skill unlocksReward pacingLong gaps disengage: aim for early reward density
Achievement rateWhich goals players pursueRare: obscure or grindy. Universal: no perceived value
GDnD wiki · Progression and Power Curves
Pip's Peak · synthetic data · players still climbing at each step

The funnel

Data visualisation in practice

Heatmaps show where; timelines show when

WIND → Level 4 · Gusty Gap · falls illustrative · darker = more falls
Heatmaps

Deaths, paths, clicks, plotted on the level.

Spatio-temporal maps

Behaviour over time: when in a session problems happen.

Dashboards & funnels

Live KPIs for the whole team, from designers to QA.

Pairs · 2 minutes

Level 4 keeps 74% of starters. Level 3 kept 95%.

Three hypotheses

What could cause the drop? Be specific.

One number each

Which data would support or rule out each one?

One question

What would you ask a playtester?

Also known: deaths per attempt went from 0.24 on Level 3 to 0.64 on Level 4.

2:00
04

Economy, social
& technical

Three more places players leave from

Economic metrics · free-to-play and RPG systems

Follow the currency

Earn / spend ratioIs the economy balanced, inflating or starved?
Sinks (drains)Somewhere to spend; no soft-lock from scarcity
Item popularityPerceived value of what's for sale
Conversion rate% of players who ever pay
ARPU · ARPPURevenue per user · per paying user
Worked numbers

10,000 players. 150 of them pay. Revenue £4,500.

1.5%conversion
£0.45ARPU
£30ARPPU

A tiny group pays for everyone. That's why who you design spending for is an ethical question.

GDnD wiki · Internal Economy
Social and behavioural metrics · multiplayer and community games

The health of the community

Matchmaking wait times

Long waits signal a population imbalance and make the game feel empty. Fixes: flexible rules, cross-play, bots, something to do while waiting.

Player-to-player interactions

Messages, trades, assists, conflicts. Pro-social tools (pings, emotes, assist rewards) raise the good ones.

Toxicity reports · abandon rates

Spikes signal frustration, imbalance or poor communication tools. Data feeds moderation and reputation systems.

Guild and team participation

A fall in participation often comes before churn: a leading indicator.

GDnD wiki · Six Degrees of Socialisation
Performance and technical metrics

Bad performance looks like bad design

FPS

Frame-rate stability

Drops break timing and feel.

Load

Loading & latency

Waiting is the first thing new players do.

Crashes

Crash & error rate

Every crash is a chance to quit.

Input

Responsiveness

Lag reads as "unfair" (Week 3).

Pip's Peak: players on devices under 30 fps keep only 2% at D30, against 10% overall. Is the game too hard, or is it just running badly?

GDnD wiki · QA and Bug Triage
05

From data
to decisions

Hypothesis, change, measure, repeat

Evidence-based design decisions

The loop, and the A/B test

Hypothesis Implement Measure Adjust VALIDATE did the number move?
A/B test

Randomly split new players. A gets the current version, B gets one change. Compare one metric chosen in advance.

Rules

One change at a time · random split · enough players · same time window · decide what "better" means first.

Watch out

A test that raises D7 by adding more daily nags may also raise uninstalls and complaints. Measure the side effects.

Predictive analytics · from reactive to proactive

Who is about to leave?

collect data→build a model→predict→design action
Sessions in first 3 days < 2? High risk 10+ deaths on one level? Medium Low yesnoyesno
Predicts

Churn (who stops) · spending (who buys) · retention trends

Methods you'll hear

Decision trees, random forests, SVMs, neural networks, hidden Markov models

The design question

Help them past Level 4, or send them a discount? Same prediction; very different ethics.

GDnD wiki · Player Modelling
Industry tools

Two you can use this year

Unity Gaming Services Analytics · first-party
  • Sessions, active users, retention, revenue
  • Custom events: level complete, deaths, positions
  • Funnels through gameplay sequences
  • Remote Config and A/B tests without a new build
GameAnalytics · free, third-party
  • Predefined dashboards: sessions, retention, progression, resources, performance
  • Custom events for specific actions
  • Heatmaps (add-on), errors and performance
  • SDKs for Unity, Unreal, Godot, C++

Or start smaller: write your own events to a CSV and open it in a spreadsheet.

Case studies

Analytics that changed the game

Halo 3 · Bungie & Microsoft

Death heatmaps of multiplayer maps showed spots that were too strong. The maps were changed before launch.

Candy Crush Saga · King

Level difficulty tuned from how often players fail and retry each level.

League of Legends · Riot

Report and chat data used to design interventions against toxic play.

In each case the data found the problem. Designers still had to decide what to do.

Ethics · privacy, consent and power

Collect less. Explain more. Use it for players.

The law (UK GDPR)
  • Telemetry linked to a player ID is usually personal data
  • You need a lawful basis and a clear privacy notice
  • Data minimisation: only what answers a question
  • Extra duties for children (ICO Children's code)
The design ethics
  • Predicting who will pay and targeting them is Week 3's dark patterns, automated
  • Optimising D7 alone can reward compulsion over fun
  • Pseudonymise, aggregate, delete what you don't need

The test: would players be comfortable seeing what you collect, and what you do with it?

GDnD wiki · Telemetry, Privacy and Consent
Take away

Five things to remember

Analytics is the what; playtesting is the why. Triangulate before you redesign.

Retention and churn are the headline, but segment them: by cohort, device and progress.

Read funnels as stories. Fix the earliest big drop first; the gain carries through.

Correlation isn't cause. Test changes with one-variable A/B tests and a metric chosen first.

Collect less, explain more, and use prediction to help players, not to target them.

06

Practical:
read the climb

Metrics in a game you know, then a launch week of telemetry

Part A · pairs · 15 minutes · Therese's activity

Metrics in a game you know

Analyse
  • Pick a game: League of Legends, Fortnite, Hades, Stardew Valley, FIFA… or your own
  • 3 in-game metrics (e.g. KDA, gold, creep score) and 2 performance metrics (frame rate, load times)
  • How is each tracked? Which design decision would it inform?
Research
  • Choose one tool or method: an analytics platform, server logs, heatmaps, surveys
  • How does it work? What does it give developers?
Example · Hades

Runs completed, boons collected, time per run → is a weapon or boon underused? Are runs getting shorter as players improve?

Part B · groups of 3–4 · 35 minutes

Pip's Peak: launch week

The game (invented)

A mobile climbing game. A five-step tutorial, ten levels from Mossy Steps to The Summit, gems to spend on skins, a climbing club after Level 3.

The data (synthetic)

2,000 players, followed for 30 days. Made up for teaching: no real players.

SheetOne row per…
Read mecolumn meanings
Playersplayer: device, tutorial step, highest level, D1/D7/D30, club, gems, spend
Attemptsplayer per level: attempts, deaths, completed, minutes

Download CRE342_W8_PipsPeak_Telemetry.xlsx from the module site. Pivot tables are your friend.

Part B · your questions

Find three findings

Look atAskHint
The funnelWhere are the two biggest drops? One is Level 4 (the worked example). Find the other.Players: tutorial_step_reached
RetentionD1, D7, D30 overall; then by device and by club. Is the club a cause?Average of returned_d7 by device_fps
DifficultyCompletion % and deaths per attempt for each levelAttempts: sum by level
EconomyGems earned vs spent by highest level reached. What's missing?Ratio earned ÷ spent
MoneyConversion, ARPU, ARPPUspend_gbp > 0

For each finding: evidence → hypothesis → design change → how you'd measure it → one playtest question.

Worked example · one finding, done properly

Level 4, "Gusty Gap"

Evidence1,198 players started Level 4; 881 finished it (73.5%, against 92–98% for every other level). Deaths per attempt: 0.64, against 0.24 on Level 3.
HypothesisWind is new in Level 4 and never taught: players don't realise the gust is pushing them, so falls feel random.
ChangeAdd a short, safe wind section before the gap, with visible wind lines and a sound cue (Week 3 feedback; Week 4 affordance).
MeasureA/B test on new installs: Level 4 completion is the success metric, chosen first. Watch D7 and Level 5 too.
Playtest question"Why did you fall there?" If players say "the wind", the fix is right; if "the jump button", it isn't.
Part C · data pitch

One chart, one finding, one change

1

The chart

One clear chart. Label the axes; say what one bar means.

2

The finding

One sentence, with a number in it.

3

The change

What you'd change in the game, specifically.

4

The test

How you'd know it worked.

2:00per group · the rest of the room asks: could anything else explain it?
Exit ticket

“In my own game, I'd track …, because it would tell me ….”

One sentence each, before you leave.

References

Clark, O. (2014) Games as a Service: How Free to Play Design Can Make Better Games. Burlington, MA: Focal Press.

El-Nasr, M. S., Drachen, A. and Canossa, A. (eds) (2013) Game Analytics: Maximizing the Value of Player Data. London: Springer.

Information Commissioner's Office (2020) Age Appropriate Design: A Code of Practice for Online Services. Wilmslow: ICO.

Lin, J. (2013) 'The science behind shaping player behavior in online games' [conference talk]. Game Developers Conference 2013.

Sellers, M. (2017) Advanced Game Design: A Systems Approach. Boston: Addison-Wesley.

Su, Y., Backlund, P. and Engström, H. (2021) 'Comprehensive review and classification of game analytics', Service Oriented Computing and Applications, 15, pp. 141–156.

Thompson, C. (2007) 'Halo 3: how Microsoft Labs invented a new science of play', Wired, 15(9).

Yannakakis, G. N. and Togelius, J. (2018) Artificial Intelligence and Games. Cham: Springer.