What a public dataset holds
The Hugging Face Hub hosts hundreds of thousands of public datasets, from a few hundred rows to billions. A dataset is a table: each row is one item, each column one field. Larger datasets are divided into configurations (subsets, such as a language) and splits (usually train, validation and test). The Hub doesn't prescribe what the columns mean, so every dataset is shaped by its authors: a list of chat turns in one, a question column and an answer column in another, a whole web page per row in a third.
Only some of those shapes can become training chats. A math dataset with a question and a worked answer can; a dataset of tweets labelled with an emotion cannot, because it says what a text is, not how to reply to it. Before importing, Scalata looks at the split you pick and suggests the formats that fit.
Which column asks and which answers
To become a training chat, each row needs a part the user says and a part the assistant replies. In most datasets the column names give it away: prompt, question, instruction or query on one side; response, answer, output or completion on the other. Some datasets add a context column — a passage the question is about, as in SQuAD — or a system prompt. Other columns, such as an id, a web address or a category, describe the row and play no part in the chat.
| Column | What it holds | Role |
|---|---|---|
instruction | text · short | prompt |
context | text · long, often empty | context, added under the question |
response | text · long | answer |
category | short text · few distinct values | not used |
Scalata suggests the formats that fit the dataset you pick, with a short reason for each. If you know better, Choose columns manually lets you pick any format and columns; rows that don't fit are skipped.
The formats
One column holds a list of turns, as role/content pairs or in the ShareGPT style with from/value. The closest shape to a training chat; multi-turn conversations stay multi-turn.
A question column and an answer column, optionally with a context column that is added under the question. Each row becomes a single-turn chat.
An instruction, an optional input it applies to, and an output. Named after Stanford's Alpaca dataset, which made the layout popular. Instruction and input together become the user message.
instruction: Summarise the customer's complaint in one sentence.
input: I ordered the blue model on the 3rd, was charged twice and still have no tracking number.
output: The customer was charged twice for an order that has not shipped.A better (chosen) and a worse (rejected) answer to the same prompt, or two whole conversations that differ only at the end. These rows become preference pairs for DPO rather than single replies.
prompt: Can I change my delivery address after ordering?
chosen: Yes, until the parcel ships. Open Orders → your order → Edit address…
rejected: Please contact support.One text column holds a whole conversation with speaker labels, which Scalata splits into turns. Text before the first label becomes the system prompt.
### Human: … ### Assistant: …
Human: … Assistant: …
User: … Assistant: …
<|system|> … <|user|> … <|assistant|> … (also <|im_start|>user)
[INST] <<SYS>> … <</SYS>> … [/INST] … (Llama 2 style)
### Instruction: … ### Input: … ### Response: …
Question: … Answer: …Long text with no questions, answers or speaker labels: web pages, articles, books. Nothing to reply to, so not chat training, but useful for continued pre-training.
Three outcomes
Shown with the reason. A tweet labelled “joy” says what a text is, not how to reply to it.
- Chats. Each row becomes one example, ready for supervised training (or DPO, for preference rows).
- Plain text. Rows are gathered into documents and added to the dataset's Documents, ready for continued pre-training or for generating questions and answers from them.
- Not supported, with the reason. Typical cases are classification data (a short text and a label, like cardiffnlp/tweet_eval), images or audio only, or tables of numbers.
- You pick the dataset and its train split, about 15,000 human-written instruction/response pairs.
- The dataset has four columns:
instruction(short),context(long, often empty),response(long) andcategory(a handful of repeated labels). - Prompt and response is the natural fit:
instruction→ prompt,response→ answer,contextadded under the question.categoryis left out. - You import 2,000 rows. They become 2,000 single-turn examples. Importing again later with 5,000 rows adds only the 3,000 new ones.
How to use public data well
- Read the dataset card first. It states the licence, where the data came from and known problems. Some datasets may not be used commercially.
- Prefer close over big. A public dataset teaches its own style and topics. A few thousand rows that resemble your use usually help more than a huge general one.
- Mix with your own data. Public chats are a good base for general behaviour; your own chats teach what is specific to you.
- Look at a few imported examples before training, to confirm the roles landed where you expected.
Common mistakes
- Importing a classification dataset and expecting it to teach the model how to answer.
- Not reading the licence on the dataset card before training a model you will use commercially.
- Forcing a manual mapping that puts the answer column in the prompt role, so the model learns to reply with questions.
- Importing a huge general dataset and drowning out the few hundred chats that describe your real task.
- Using a public dataset's test split as training data and then evaluating on benchmarks built from it.
References and further reading
External sources open in a new tab.
- 01DatasetsHugging Face docs · huggingface.coHow Hub datasets are organised into configurations, splits, rows and columns.
- 02Dataset CardsHugging Face Hub docs · huggingface.coWhat a dataset card should tell you: sources, licence, intended use and limitations.
- 03Dataset formats and typesHugging Face TRL docs · huggingface.coThe common training-data shapes — conversational, prompt-completion, preference — side by side.
- 04Alpaca: A Strong, Replicable Instruction-Following ModelStanford CRFM, 2023 · crfm.stanford.eduWhere the instruction, input, output layout comes from.