P-011 · 2026 · Company project · M.Sc. thesis · Study design set · Final evaluation pending

Game revenue forecasting with two foundation models

M.Sc. thesis · Leuphana University Lüneburg · Final evaluation pending

Official title — Forecasting Free-to-Play Game Revenue with Foundation Models and Agent-Compiled Text Covariates

TabPFN-TSChronos-2Hybrid-RAGLocal LLMsTime series

Study design

Forecast90-day daily revenue forecast · DAU × ARPDAU
CorpusSelected wiki sections · previous-day cutoff · German pages filtered in code · JEV-9B (Qwen3.5-9B), 12 categories, instance/type labels
RetrievalBM25 · bge-m3 · RRF hybrid · hybrid + bge-reranker-v2-m3 · hybrid + one context sentence from Qwen · select by Recall@6, report nDCG@6
Readingqwen3.8:27b primary · gemma4:31b independent · JEV-9B passage scores · verbatim quote check · two rotating judges
Text pathsColumn · embedding columns · trained ridge adapter · KV cache edit → TabPFN-TS ARPDAU
H1TabPFN-TS and Chronos-2 with calendar features · three-month plan and LightGBM, ARIMA, ETS, seasonal naïve · plan-to-date error and daily MASE
H2One selected TabPFN-TS path per eligible pair · expanded baseline without text, activity identity and shuffled answers · activity days within forecast days 1–30
DiagnosticsObserved-level ceiling · grade multiplier · stationary bootstrap · Hansen SPA / White’s Reality Check · Holm for H1/H2
Backtest52 monthly forecasts per game · 28 development · 24 held out for one evaluation
Study design set · Final evaluation pending

Forecasting task

I compare frozen TabPFN-TS and Chronos-2 on 90-day forecasts for free-to-play games. Each model forecasts daily active users (DAU) and average revenue per daily active user (ARPDAU). Multiplying the two gives daily revenue. On the development period, I first break down errors by activity type and compare them with quiet days. This diagnostic helps explain where the models miss. It does not determine which activity types enter the test.

Filtering the wiki corpus

I select wiki sections relevant to planned activities. For each forecast, retrieval uses the latest available page version as of the previous day. If the relevant version is missing, the page is omitted for that forecast. A language filter in code removes German-language pages. The local judge JEV-9B, based on Qwen3.5-9B, then classifies pages into 12 families by their main purpose. It excludes results reports, A/B-test pages, retrospectives, status reports and the other category. Within the applicable families, it labels pages as instance for a specific edition or type for a general description. The page labels are assigned on the latest state. I check their stability against older versions, while the search and readers use only eligible historical text.

Five ways to retrieve evidence

I index passages of up to 900 characters with their page titles and headings. On the development period I compare five retrieval variants on a graded set of passages:

  1. Lexical search with BM25.
  2. Vector search with bge-m3.
  3. Hybrid search that combines both rankings with Reciprocal Rank Fusion (RRF).
  4. Hybrid search with reranking, using the local bge-reranker-v2-m3 to reorder the top 30 passages.
  5. Hybrid search with passage context, using one sentence per passage generated by qwen3.8:27b from the same historical page version.

For each game, I select the variant with the highest Recall@6, the share of relevant passages found in the first six. I also report nDCG@6, which gives greater weight to relevant passages ranked higher. The readers get up to six passages, with at most three from any one page.

Local readers and evidence checks

qwen3.8:27b and gemma4:31b independently fill the same form. It records a comparable past run, a grade of weaker, similar or stronger, what changed, and a quote with its page and version. Qwen supplies the primary answer for forecasting. Gemma provides an independent check. JEV-9B scores the past-run and grade options one passage at a time. Code checks that the quote appears verbatim. The two models that did not produce the answer judge its support in rotation. An answer without a valid quote or two yes votes becomes nothing stated.

Four ways to pass information to TabPFN-TS

The text experiment uses an expanded baseline without text. It includes calendar features, activity identifiers, a curve for the days since a world launch and key sale dates. The text intervention targets ARPDAU on scheduled activity days. The DAU forecast stays the same, and Chronos-2 remains in the comparison without text.

PathWhat it changes
ColumnAdds two daily columns for the grade (−1, 0 or +1) and the measured lift of the comparable past run. Lift expresses the level during a run relative to its preceding baseline.
Embedding columnsEncodes the explanation and quote with bge-m3, then uses PCA to reduce the vector to 4, 8 or 16 input columns. PCA is fitted on prior answer texts without using outcomes.
Trained adapterUses ridge regression to predict the expected activity level from the same bge-m3 vector. The prediction adds one input column. The penalty is chosen by forward validation.
KV cache editUses TabPFN-TS in fit_with_cache mode. In each attention layer, it scales the cached keys for past days of the named run and can shift the corresponding values along an ARPDAU level direction. The forecaster weights and input table stay fixed. This path is specific to TabPFN-TS with TabPFN-3.5.

Evaluation and controls

H1 · Forecasting models. For each game, I compare TabPFN-TS and Chronos-2 with the strongest development-period reference among the analysts’ three-month plan, LightGBM, ARIMA, ETS and seasonal naïve. On the test period, the plan-to-date error, which measures the relative error in accumulated revenue, must fall by at least 2.0 percentage points and daily MASE must also be lower. A stationary block bootstrap over forecast origins accounts for overlapping forecasts. Holm correction covers the two forecasters.

H2 · Added value of text. I choose one of the four TabPFN-TS paths on development data for each of 12 eligible game–activity pairs. The test covers events, sales during or outside events, and the first three days of a world launch. Releases are excluded from the final text test because too few release dates were verifiable in advance. The claim is assessed on days within the first 30 forecast days where an accepted answer supplies a past run and grade. The selected path must beat the activity-identity control (name code) and the shuffled-answer control, averaged over three seeds, by more than two standard errors on both MASE and MAPE. It must also match or improve on the baseline without text on both measures. One-sided paired tests on MASE use Holm correction across activity types within a game.

The diagnostic ceiling gives the forecaster the observed activity level to show how much forecast error could be avoided with perfect information. The multiplier applies a simple grade-specific factor to the baseline forecast. I check that the cache mode reproduces the standard forecast and that a neutral cache edit leaves it unchanged. Hansen’s SPA test and White’s Reality Check use a stationary bootstrap on the development data to assess the effect of choosing among many setups.

Protocol. The design uses 52 monthly forecasts per game, with 28 for development and 24 held out. All selections and test rules are fixed before the held-out period is evaluated once. The final evaluation is pending.

← Back to all projects

Timur Salakhetdinov ·