P-009 · 2026 · Hackathon entry
Fuel price forecasting with TabPFN-3.5 and LLM event features
Do LLM-extracted event features improve TabPFN-3.5 fuel price forecasts? A hackathon study on EIA gasoline prices with a placebo test separating content from timing.
Study design
Same TabPFN-3.5 model, rows, context, seed and fit schedule
The question
A language model can turn a report into structured features, but does that make a forecast better? I built this study for the Prior Labs TabPFN-3.5 Hackathon to test that on U.S. retail gasoline prices. Jev, a hosted extraction model, reads each weekly issue of EIA’s This Week in Petroleum and labels the market driver, the stated price outlook, the region and any supply disruption. Each label keeps the sentence it came from. The data covers 29 weekly EIA price series and 1,146 issues from 2002 to 2024.
Approach
TabPFN-3.5 forecasts price changes one to four weeks ahead, with quantiles. It runs in several arms that differ only in their inputs: calendar and price history, then the timing of each issue, then Jev features or the raw issue text. A placebo arm keeps the timing but shuffles the extracted labels among issues already published, so it has the same columns without the real content. Every feature uses only information published before the forecast date. I fixed the protocol on 2005 to 2015, then scored a frozen holdout from 2016 to January 2024 once.
Results
The holdout covers 418 weekly forecast dates, 29 series and 48,484 forecasts per arm. The main metric is MASE, an error measure scaled to a naive one-week forecast, where lower is better. Jev features lowered MASE by 0.127 relative to timing alone (95% CI 0.050 to 0.209) and helped in all 29 series. However, the shuffled placebo did just as well (difference +0.013, CI −0.036 to +0.057). The gain therefore comes from the extra columns, not from what the text says. Without event columns, TabPFN-3.5 was less accurate than simply repeating the last price, and its 80% intervals covered only 67 to 73% of outcomes.
Limits
This is a retrospective study: the features are as-of each forecast date, but pretrained models may carry hindsight about 2002 to 2024. The narrative restates recent prices, and EIA discontinued it in January 2024, so the workbench cannot run live. Two corrections made after the holdout run, to interval scoring and to HTML left in 104 issues, are documented and leave the answer unchanged.