Csilla — RLHF & Human-in-the-Loop Data Operations Lead AI Skill
Instant download · 30-day money-back guarantee. Pay once, keep forever — no subscription. Refund policy
Run the human data layer: write usable guidelines, raise agreement, and collect preference data a model can actually learn from.
- Guideline design, annotator qualification and calibration
- Inter-annotator agreement diagnosis and repair
- Preference and pairwise collection for RLHF and DPO, bias control
- Gold sets, audits, annotator wellbeing, provenance and consent
Teams collecting preference or evaluation data whose labels disagree and whose model will not improve because of it.An annotation operations lead bills $120+/hr, this is one file, yours forever.
Drop Csilla into Claude and get a human data operations lead who fixes the guideline before blaming the annotators, because low agreement is almost always an ambiguous rubric.
Csilla runs the human data layer behind aligned models: annotation program design and what makes a guideline usable; annotator recruitment, qualification and calibration; inter-annotator agreement and what to do when it is low; preference data collection for RLHF and DPO; pairwise comparison design and position bias; rubric-based scoring; red-team data collection; active learning and sampling so annotators see the examples that matter; quality control through gold sets, audits and reviewer-of-reviewers; annotator pay, workload and wellbeing especially on distressing content; vendor management for outsourced labelling; cost per label against value; and the data provenance and consent record a regulator may ask for.
What you get
- →Guideline design, annotator qualification and calibration
- →Inter-annotator agreement diagnosis and repair
- →Preference and pairwise collection for RLHF and DPO, bias control
- →Gold sets, audits, annotator wellbeing, provenance and consent
How to install
Download the .skill package → open Claude → paste SKILL.md into your Project Instructions or system prompt → describe your requirement → Csilla builds the answer. Includes a full worked example so you see exactly what you get.
# Csilla - RLHF & Human-in-the-Loop Data Operations Lead You are Csilla, a human data operations lead. Low inter-annotator agreement is a guideline defect until proven otherwise. ## How you work 1. Measure agreement and trace disagreement to specific rubric clauses 2. Rewrite the guideline with worked boundary examples, then recalibrate 3. Design pairwise collection with position and length bias controlled 4. Hold quality with gold sets and audits; sample where it matters Never blame annotators before fixing the guideline, and never run distressing-content work without a wellbeing policy and rotation.
Excerpt from the actual file you'll download.
Four steps. Any AI chat.
- 01Download the file
After checkout, the download link lands in your inbox. Save the file anywhere on your device.
- 02Open your AI chat
Claude, ChatGPT, Gemini, Grok, or Copilot — whichever one you already use.
- 03Paste the file contents
Drop it into the system prompt, Project instructions, or custom instructions field.
- 04Start working
Your AI is now configured as a specialist. Ask it anything inside its domain.
No technical knowledge required. No subscription. Pay once, keep forever.
Works with every major AI chat.
Drop the file into your AI's system prompt, Project instructions, or custom instructions. No setup. No code. No vendor lock-in.
- Claude
- ChatGPT
- Gemini
- Grok
- Copilot
Works with any AI chat that accepts a system prompt or custom instructions.
Ready to specialise your AI?
One drop-in file. Pay once, keep forever — works with Claude & ChatGPT.