Skip to content

Structured Information Extraction from Texts in Spanish Using LLM Systems

I built a system that turns free Spanish text into structured JSON data using small open LLMs. It fills only what the text supports and is designed to return null rather than invent a value.

  • Published paper

    Published in the IberLEF 2026 proceedings (CEUR Workshop Proceedings). See the paper

  • Award

    One of three awards for the best master's theses of the first cohort, sponsored by the Cátedra Tracasa. Details

  • Supercomputer training

    LoRA and full fine-tuning on MareNostrum 5 (NVIDIA H100), across 3 open models from 3B to 14B parameters.

Try the live demo →

Tech stack at a glance

  • PyTorch

    Model training

  • Transformers · PEFT · TRL

    LoRA and supervised fine-tuning

  • vLLM

    Serving base and fine-tuned models

  • Outlines · xgrammar

    Grammar-constrained JSON decoding

  • Pydantic · jsonschema

    Schema handling and output validation

  • SLURM on H100 GPUs

    Batch training on the MareNostrum 5 supercomputer

  • MLflow · Langfuse

    Experiment tracking and tracing

  • Sentence-Transformers · scikit-learn

    Few-shot retrieval and evaluation

Full tech stack

Headline figures

Figure Value What it is
Best zero-shot F1 0.7999 Qwen3-14B with the best inference-time pipeline (run E226), GenSIE development set, 149 tasks
Test-set Micro-F1 0.74 Qwen3-14B, pipeline e226, on the released test set of 145 instances. Self-evaluated by the authors with the official harness and a lighter embedder, so a lower bound on the official score. Reported in the published GenSIE 2026 paper
Experiment scale 280 experiment IDs 452 logged runs in the experiment registry (several runs can share an ID, for example across models)
Synthetic corpus 3,948 pairs Schema–instance pairs over 3,275 distinct schema titles, generated with a five-stage adaptation of GuideX and used both for few-shot retrieval and for training

The 0.7999 figure is on the development set and the 0.74 figure on the test set, so they are not comparable with each other.

About the project

Master's thesis (Máster en Machine Learning, Universidad Pública de Navarra) by Iñigo Goikoetxea Fanega, supervised by Mikel Sesma Sara and Javier Lasheras Navas. It builds a system that fills an arbitrary JSON schema, supplied only at inference time, with information grounded in a Spanish text, using small open LLMs (up to 14B parameters).

The work started as a submission to the GenSIE 2026 shared task at IberLEF, where fine-tuning was forbidden, and was then extended with LoRA and full fine-tuning on a synthetic Spanish corpus. It combines schema-similarity few-shot retrieval, grammar-constrained decoding, schema-aware post-processing and a controlled study of when training helps.

Where to go next