Diagrams — Module FTDD-10: distilabel

Module: FTDD-10 — distilabel Diagram count: 4 Tool: Mermaid (primary). Each diagram validated in Mermaid Live Editor.

Diagram 1 — distilabel in the Argilla Ecosystem

Type: Ecosystem map / division of labor Purpose: Where distilabel fits. The data-construction complement to TRL's training side. Reading the diagram: Left = the Argilla ecosystem (data construction + labeling). Right = the training side (TRL). distilabel builds the data; TRL trains the model. They meet at a dataset-format contract.

flowchart LR
  subgraph Argilla["THE ARGILLA ECOSYSTEM (data construction)"]
    AD["Argilla Datasets\nhuman labeling & annotation"]
    DB["distilabel\nsynthetic pipeline:\ngenerate · evolve · filter · format"]
    FD["Feedback Datasets\npreference annotation format"]
  end
  subgraph Train["TRAINING SIDE"]
    TRL["TRL\nSFTTrainer · DPOTrainer\nGRPOTrainer"]
  end

  DB -->|"synthetic SFT data\n(prompt, response)"| TRL
  FD -->|"preference pairs\n(prompt, chosen, rejected)"| TRL
  AD -->|"human-curated data"| TRL

  Thesis["THESIS: data matters more than algorithm.\ndistilabel makes the data good.\nIt is the steering-wheel factory."]

  DB -.-> Thesis

  style AD fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
  style DB fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#5eead4
  style FD fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
  style TRL fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style Thesis fill:#08080c,stroke:rgba(94,234,212,0.4),stroke-dasharray:4 2,color:#5eead4

Diagram 2 — Magpie + Evol-Instruct: Scaling SFT Generation

Type: Pipeline / two-stage generation Purpose: The two techniques that scale and diversify synthetic SFT data without hand-authoring every seed. Reading the diagram: Top = Magpie self-prompting (diverse seed pool). Bottom = Evol-Instruct (complexity stretching). Combined = large, diverse, multi-difficulty candidate pool.

flowchart TD
  Template["Pre-query template\n(system prompt + start of user turn,\nNO actual instruction)"]
  Model["Instruct-tuned model\n(completes with a plausible instruction)"]

  Template --> Model
  Model -->|"sample many times\nwith temperature"| MagpieOut["DIVERSE SELF-PROMPTED\ninstruction set\n(no hand-authored seeds)"]

  Seeds["Seed instructions"]
  Evol["Evol-Instruct evolution\ndeepen reasoning · add constraints\nincrease specificity · branch"]

  Seeds --> Evol
  Evol -->|"N evolution steps"| EvolOut["COMPLEXITY-STRETCHED\ninstruction set\n(wider difficulty range)"]

  MagpieOut --> Combined["CANDIDATE POOL\nlarge · diverse · multi-difficulty"]
  EvolOut --> Combined
  Combined --> Gate["-> JUDGE FILTER (Diagram 3)"]

  style Template fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
  style Model fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
  style MagpieOut fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style Evol fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
  style EvolOut fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style Combined fill:#14141f,stroke:#5eead4,stroke-width:2px,color:#5eead4
  style Gate fill:#08080c,stroke:rgba(94,234,212,0.4),stroke-dasharray:4 2,color:#5eead4

Diagram 3 — Judge Filter + Dedup: The Quality Gate

Type: Sequential filter Purpose: The quality gate that sets the dataset's ceiling. Judge scoring + threshold filtering + sentence-transformers dedup. Reading the diagram: The candidate pool flows through the judge (score), the threshold filter (drop noise), and the dedup (drop near-duplicates). What exits is the curated training set.

flowchart LR
  Pool["CANDIDATE POOL\n(generated, noisy)"]
  Judge["LLM-as-judge\nscore each response on\ncorrectness · helpfulness\nrelevance · conciseness"]
  Threshold["THRESHOLD FILTER\nkeep score >= cutoff\ndrop below"]
  Dedup["SENTENCE-TRANSFORMERS DEDUP\nembed · pairwise similarity\ndrop near-duplicates"]
  Curated["CURATED DATASET\neach example: distinct signal\nquality-validated"]

  Pool --> Judge --> Threshold --> Dedup --> Curated

  Calibrate["JUDGE CALIBRATION:\nspot-check decisions\nwatch verbosity/sycophancy bias\nvalidate on held-out sample"]

  Judge -.must be calibrated.-> Calibrate

  style Pool fill:#14141f,stroke:rgba(255,255,255,0.12),color:#9494a0
  style Judge fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style Threshold fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style Dedup fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style Curated fill:#08080c,stroke:#5eead4,stroke-width:2px,color:#5eead4
  style Calibrate fill:#08080c,stroke:rgba(255,255,255,0.12),color:#9494a0

Diagram 4 — Preference Pair Construction for DPO

Type: Pipeline / one-to-pairs Purpose: The DPO-data pipeline. Generate multiple responses, rank via judge/reward model, emit chosen/rejected pairs in the format TRL's DPOTrainer expects. Reading the diagram: Top = one prompt generates K responses. Middle = judge ranks them. Bottom = top-ranked = chosen, lower-ranked = rejected; filter ambiguous/trivial gaps; emit DPO format.

flowchart TD
  Prompt["Prompt (from Magpie or seed)"]
  Gen["Generate K responses\n(same model, different temps,\nor different models)"]
  Rank["Judge / reward model\nranks the K responses"]
  Pair["PAIR: highest-ranked = CHOSEN\nlower-ranked = REJECTED"]
  FilterPair["Filter pairs:\ndrop if score gap too small (ambiguous)\ndrop if too large (trivial)"]
  Emit["EMIT DPO format\nprompt · chosen · rejected\n-> TRL DPOTrainer"]

  Prompt --> Gen
  Gen -->|"response 1 ... response K"| Rank
  Rank --> Pair
  Pair --> FilterPair
  FilterPair --> Emit

  style Prompt fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
  style Gen fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
  style Rank fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style Pair fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style FilterPair fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8
  style Emit fill:#08080c,stroke:#5eead4,stroke-width:2px,color:#5eead4

Validation notes

All four diagrams use the course design system colors: #14141f panel fill, #5eead4 accent for primary, rgba(94,234,212,0.5) for secondary borders, #e4e4e8 / #9494a0 for text.
Paste each into Mermaid Live Editor to render. All use stable Mermaid syntax (flowchart) supported in current Mermaid (v10.4+).
For the slide deck (artifact 03), these are rendered as static SVG/PNG captures from Mermaid Live, inlined into reveal.js.

# Diagrams — Module FTDD-10: distilabel **Module**: FTDD-10 — distilabel **Diagram count**: 4 **Tool**: Mermaid (primary). Each diagram validated in [Mermaid Live Editor](https://mermaid.live). --- ## Diagram 1 — distilabel in the Argilla Ecosystem **Type**: Ecosystem map / division of labor **Purpose**: Where distilabel fits. The data-construction complement to TRL's training side. **Reading the diagram**: Left = the Argilla ecosystem (data construction + labeling). Right = the training side (TRL). distilabel builds the data; TRL trains the model. They meet at a dataset-format contract. ```mermaid flowchart LR subgraph Argilla["THE ARGILLA ECOSYSTEM (data construction)"] AD["Argilla Datasets\nhuman labeling & annotation"] DB["distilabel\nsynthetic pipeline:\ngenerate · evolve · filter · format"] FD["Feedback Datasets\npreference annotation format"] end subgraph Train["TRAINING SIDE"] TRL["TRL\nSFTTrainer · DPOTrainer\nGRPOTrainer"] end DB -->|"synthetic SFT data\n(prompt, response)"| TRL FD -->|"preference pairs\n(prompt, chosen, rejected)"| TRL AD -->|"human-curated data"| TRL Thesis["THESIS: data matters more than algorithm.\ndistilabel makes the data good.\nIt is the steering-wheel factory."] DB -.-> Thesis style AD fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8 style DB fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#5eead4 style FD fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8 style TRL fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style Thesis fill:#08080c,stroke:rgba(94,234,212,0.4),stroke-dasharray:4 2,color:#5eead4 ``` --- ## Diagram 2 — Magpie + Evol-Instruct: Scaling SFT Generation **Type**: Pipeline / two-stage generation **Purpose**: The two techniques that scale and diversify synthetic SFT data without hand-authoring every seed. **Reading the diagram**: Top = Magpie self-prompting (diverse seed pool). Bottom = Evol-Instruct (complexity stretching). Combined = large, diverse, multi-difficulty candidate pool. ```mermaid flowchart TD Template["Pre-query template\n(system prompt + start of user turn,\nNO actual instruction)"] Model["Instruct-tuned model\n(completes with a plausible instruction)"] Template --> Model Model -->|"sample many times\nwith temperature"| MagpieOut["DIVERSE SELF-PROMPTED\ninstruction set\n(no hand-authored seeds)"] Seeds["Seed instructions"] Evol["Evol-Instruct evolution\ndeepen reasoning · add constraints\nincrease specificity · branch"] Seeds --> Evol Evol -->|"N evolution steps"| EvolOut["COMPLEXITY-STRETCHED\ninstruction set\n(wider difficulty range)"] MagpieOut --> Combined["CANDIDATE POOL\nlarge · diverse · multi-difficulty"] EvolOut --> Combined Combined --> Gate["-> JUDGE FILTER (Diagram 3)"] style Template fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8 style Model fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8 style MagpieOut fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style Evol fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8 style EvolOut fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style Combined fill:#14141f,stroke:#5eead4,stroke-width:2px,color:#5eead4 style Gate fill:#08080c,stroke:rgba(94,234,212,0.4),stroke-dasharray:4 2,color:#5eead4 ``` --- ## Diagram 3 — Judge Filter + Dedup: The Quality Gate **Type**: Sequential filter **Purpose**: The quality gate that sets the dataset's ceiling. Judge scoring + threshold filtering + sentence-transformers dedup. **Reading the diagram**: The candidate pool flows through the judge (score), the threshold filter (drop noise), and the dedup (drop near-duplicates). What exits is the curated training set. ```mermaid flowchart LR Pool["CANDIDATE POOL\n(generated, noisy)"] Judge["LLM-as-judge\nscore each response on\ncorrectness · helpfulness\nrelevance · conciseness"] Threshold["THRESHOLD FILTER\nkeep score >= cutoff\ndrop below"] Dedup["SENTENCE-TRANSFORMERS DEDUP\nembed · pairwise similarity\ndrop near-duplicates"] Curated["CURATED DATASET\neach example: distinct signal\nquality-validated"] Pool --> Judge --> Threshold --> Dedup --> Curated Calibrate["JUDGE CALIBRATION:\nspot-check decisions\nwatch verbosity/sycophancy bias\nvalidate on held-out sample"] Judge -.must be calibrated.-> Calibrate style Pool fill:#14141f,stroke:rgba(255,255,255,0.12),color:#9494a0 style Judge fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style Threshold fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style Dedup fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style Curated fill:#08080c,stroke:#5eead4,stroke-width:2px,color:#5eead4 style Calibrate fill:#08080c,stroke:rgba(255,255,255,0.12),color:#9494a0 ``` --- ## Diagram 4 — Preference Pair Construction for DPO **Type**: Pipeline / one-to-pairs **Purpose**: The DPO-data pipeline. Generate multiple responses, rank via judge/reward model, emit chosen/rejected pairs in the format TRL's DPOTrainer expects. **Reading the diagram**: Top = one prompt generates K responses. Middle = judge ranks them. Bottom = top-ranked = chosen, lower-ranked = rejected; filter ambiguous/trivial gaps; emit DPO format. ```mermaid flowchart TD Prompt["Prompt (from Magpie or seed)"] Gen["Generate K responses\n(same model, different temps,\nor different models)"] Rank["Judge / reward model\nranks the K responses"] Pair["PAIR: highest-ranked = CHOSEN\nlower-ranked = REJECTED"] FilterPair["Filter pairs:\ndrop if score gap too small (ambiguous)\ndrop if too large (trivial)"] Emit["EMIT DPO format\nprompt · chosen · rejected\n-> TRL DPOTrainer"] Prompt --> Gen Gen -->|"response 1 ... response K"| Rank Rank --> Pair Pair --> FilterPair FilterPair --> Emit style Prompt fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8 style Gen fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8 style Rank fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style Pair fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style FilterPair fill:#14141f,stroke:rgba(94,234,212,0.5),color:#e4e4e8 style Emit fill:#08080c,stroke:#5eead4,stroke-width:2px,color:#5eead4 ``` --- ## Validation notes - All four diagrams use the course design system colors: `#14141f` panel fill, `#5eead4` accent for primary, `rgba(94,234,212,0.5)` for secondary borders, `#e4e4e8` / `#9494a0` for text. - Paste each into [Mermaid Live Editor](https://mermaid.live) to render. All use stable Mermaid syntax (`flowchart`) supported in current Mermaid (v10.4+). - For the slide deck (artifact 03), these are rendered as static SVG/PNG captures from Mermaid Live, inlined into reveal.js.