Tech stack

Only tools used in the two project repositories.

Training and fine-tuning

• PyTorch, Hugging Face Transformers, PEFT and TRL for LoRA and supervised fine-tuning (SFTTrainer).
• Fully Sharded Data Parallel for full fine-tuning of the larger models.
• SLURM jobs on the MareNostrum 5 supercomputer (NVIDIA H100).

Inference and structured output

• vLLM for serving base and fine-tuned models.
• Outlines and xgrammar for grammar-constrained JSON decoding.
• Pydantic and jsonschema for schema handling and output validation.

Retrieval and evaluation

• Sentence-Transformers (multilingual MiniLM) for schema-similarity few-shot retrieval, plus a TF-IDF variant.
• fastembed, rapidfuzz and scikit-learn for the Flattened Schema Scoring metric and the analysis scripts.
• Agent variants for chain-of-thought, schema decomposition, self-correction and multi-agent post-processing.

Experiment tracking

• MLflow and Langfuse for run tracking and tracing, with a self-hosted MLflow server run through Docker Compose.

Data generation

• A five-stage synthetic-data pipeline adapted from GuideX, with jsonschema validation and LLM calls through an OpenAI-compatible API.

Retrieval in this project is retrieval-augmented few-shot selection over a synthetic example library; it does not use a vector database.