Approach
The work does not set fine-tuning against no fine-tuning. It solves one problem and explores several techniques, with and without training, on the same three small open models. A single synthetic corpus feeds both branches.
| Block | What it does |
|---|---|
| Synthetic data generation | Produces a schema-guided Spanish corpus with a five-stage pipeline adapted from GuideX |
| Zero-shot branch | Keeps the weights frozen and combines prompting, retrieved few-shot examples, constrained decoding and post-processing |
| Fine-tuning branch | Updates the weights with LoRA and with full fine-tuning on the same corpus |
Synthetic data generation
The corpus follows the idea of GuideX (guided synthetic data generation for zero-shot information extraction), adapted to Spanish and to the GenSIE record format.
| Stage | What happens |
|---|---|
| 0 | Prepare a heterogeneous Spanish corpus (mostly Spanish Wikipedia passages), segment it into self-contained passages and tag the domain |
| 1 | A teacher LLM writes a structured summary of each passage and lists null-trap candidates, questions the passage does not answer |
| 2 | Synthesise a unique JSON Schema and instruction for each passage |
| 3 | Extract a JSON instance that conforms to the schema, retrying with validator feedback if it does not |
| 4–5 | Non-LLM quality control (schema violations, degenerate or under-specified records) and packaging into the GenSIE record format |
The main departure from GuideX is that every passage gets its own schema, so a model prompted or trained with the corpus has to read the schema at inference time instead of memorising a fixed ontology.
Zero-shot branch
A generate-then-verify cascade around a single LLM call. No field name or example from the development set is hard-coded.
- Prompt construction. A system prompt fixes the role of a structured extractor. The user message holds the schema, the text and a few few-shot demonstrations retrieved from the synthetic library by schema similarity, using multilingual sentence embeddings. An optional field guide restates each field's description, enumeration and nullability in plain language.
- Constrained decoding. The JSON Schema is compiled into a grammar that masks the model's token logits, so structural validity holds by construction and the model's effort goes to semantic correctness.
- Schema-aware post-processing. A deterministic pass with five conservative layers: source grounding, lexical normalisation, schema-driven canonicalisation, verbatim snapping and a hallucination filter that replaces unsupported values of nullable fields with
null.
Fine-tuning branch
Supervised fine-tuning on chat-formatted examples from the same corpus, with the usual cross-entropy loss on the gold JSON and no reinforcement learning. Two regimes run side by side under the same data and objective:
- LoRA: freezes the base weights and trains low-rank adapters that can be merged back into a standard checkpoint.
- Full fine-tuning: updates every parameter, sharded across several GPUs for the larger models.
Three prompt modes are tried at training time, including in-context tuning, where a few demonstrations are placed inside the training prompt itself.