P-005 · 2026 · Team project
Knowledge graph question answering with LLMs
A team-built LLM pipeline that turns natural-language questions into SPARQL queries over DBpedia, answers from the query results and is evaluated on LC-QuAD.
From question to grounded answer
- Entity linkingResolve names to DBpedia resources
- Schema probingFind candidate predicates in the graph
- Question decompositionSplit multi-hop questions into steps
- SPARQL generationLLM writes the query; repair and retry on failure
Query executed against the live DBpedia endpoint
The question
A language model can answer a factual question, but it cannot show where the answer came from. This team project takes another route: it translates a natural-language question into a SPARQL query over DBpedia and answers from the query results, so every answer can be traced back to the knowledge graph.
Approach
The pipeline first links names in the question to DBpedia entities, then probes the graph schema for candidate predicates. Multi-hop questions are split into steps before an LLM writes the SPARQL query. If the query fails or returns nothing, repair and retry steps adjust it. The final results become a short answer in a Streamlit app.
Results
We evaluated the system on LC-QuAD 1.0. About 47% of its test questions no longer return results on today’s DBpedia, so the evaluation uses only answerable ones. On the first 50 questions of a fixed stratified sample, run against live DBpedia, the system answered 16 fully and 5 partially correctly, for a macro-averaged F1 of 0.412. Yes/no questions scored highest.
My role and limits
I contributed to this team project for the Advanced Machine Learning course at Leuphana University. The score is a lower bound: answers are compared as exact strings, so a correct resource URI counts as wrong when the benchmark expects a label. Entity linking, predicate choice and changes to the live graph remain the main sources of error, and 50 questions leave a wide confidence interval (±0.145).