OpenAssistant/oasst1
OpenAssistant/oasst1
1
0
Overview Code Discussions Pull Requests Wiki Activity

Update README.md

Browse Source
This commit is contained in:
Andreas Köpf 2023-04-15 13:15:54 +00:00 committed by huggingface-web
parent dee0b5f871
commit a2c461b87b
1 changed files with 26 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

34
README.md
View File

@ -129,19 +129,37 @@ corpus consisting of 161,443 messages distributed across 66,497 conversation tre
35 different languages, annotated with 461,292 quality ratings. The corpus is a product 35 different languages, annotated with 461,292 quality ratings. The corpus is a product
of a worldwide crowd-sourcing effort involving over 13,500 volunteers. of a worldwide crowd-sourcing effort involving over 13,500 volunteers.
The dataset was exported from the open-assistant.io production database on April, 12 2023.
### Dataset Structure ### Dataset Structure
Thes dataset contains demonstrations of of human-assistant conversations that were collected This dataset contains demonstrations of human-assistant conversations which were collected
on the open-assistant.io website. on the open-assistant.io website until April, 12 2023.
All conversations are exported as message trees which contain conversation messages nodes. Each message has a Conversations are exported as message trees which contain conversation messages as nodes.
role which can either be "assistant" or "prompter". The root node of a message tree is called the initial prompt. The root node of a message tree is called the initial prompt. Each message node can have
Nodes with at least two replies of completed trees have a `rank` field which indicates the users' preference consensus. multiple replies. Nodes with more than one reply can have a `rank` field indicating the
The lower the rank the better the message. order among the siblings sorted by user preference (the most preferred message has rank 0).
All messages have a role which can either be "assistant" or "prompter". The roles in
conversation threads from prompt to leaf node in a message tree are stricly alternating
between "assistant" and "prompter".
## Main Dataset Files
Data is provided either as nested as a message tree or as flat list (table) of messages.
Names of files containing message trees end in `.trees.jsonl.gz` while files containing
a list of messages with a file name ending in `.messages.jsonl.gz`.
Mesages
```
2023-04-12_oasst_ready.trees.jsonl.gz 10364 trees with 88838 total messages
2023-04-12_oasst_ready.messages.jsonl.gz 88838 messages
```
```
2023-04-12_oasst_all.trees.jsonl.gz 66497 trees with 161443 total messages
2023-04-12_oasst_all.messages.jsonl.gz 161443 messages
```