Game revenue forecasting with foundation models

Forecasts usually draw on historical observations and calendar information. Planning documents contain another source of context: descriptions of activities that have not yet taken place. A calendar indicator records their timing, while the documents can describe what is planned. Bringing that information into a forecast requires a way to represent the document content numerically.

My thesis examines whether these representations improve forecast accuracy beyond historical observations and calendar indicators. The question concerns both the information in the documents and the choices made when converting it into model inputs.

One approach uses an LLM agent to extract information into a predefined schema. The pipeline validates the extracted records, then a deterministic script converts them into covariates. These are additional columns that describe the context associated with each forecast date. The extraction can be checked against its source before the columns enter the forecasting model.

The study also compares two alternatives. TF-IDF represents documents through the words they contain and their relative importance in the collection. Embeddings use a language model to encode documents as numerical vectors that capture similarities in meaning. Comparing these representations helps assess whether structured extraction is useful, or whether simpler word-based features or embeddings provide a better input.

TabPFN-TS, using the TabPFN-3.5 checkpoint, produces the numerical forecasts from historical observations, calendar indicators and the selected covariates. The LLM is responsible for extraction. Keeping the forecasting method fixed across the document representations allows their contribution to be examined directly.

The evaluation recreates forecasts at successive points in time. Each forecast uses only observations and document versions available at that date. This matters because a later edit may contain information that was unknown when the original forecast would have been made. A later evaluation period is kept separate from the data used to develop the approach.

Forecasts with document-derived inputs are compared with a calendar-only reference and with shuffled-content controls. The reference establishes performance without document content. Shuffling changes which content is associated with an activity while retaining its timing-related structure. The comparison therefore tests whether the content contributes information beyond the dates and the presence of additional input columns.

The evaluation also checks extraction quality and forecast uncertainty. A valid record can still contain an incorrect interpretation, so schema validation alone is insufficient. Forecast intervals are assessed alongside point predictions to examine how well the model expresses uncertainty.

I developed the workflow from document preparation and LLM extraction through feature construction to forecast evaluation. This included the validation steps, reference and control comparisons, and records of the model, prompt and input versions used in each run. Those records allow a forecast to be traced back to the configuration and information that produced it.

The project forms the basis of my M.Sc. thesis in Management & Data Science at Leuphana University. I am currently writing the thesis. This overview covers the approach; company data and research results are not published here.