OpenOrca/README.md

language	license	task_categories	pretty_name	size_categories
en	mit	conversational text-classification token-classification table-question-answering question-answering zero-shot-classification summarization feature-extraction text-generation text2text-generation	Open Orca	10M<n<100M

Table of Contents

• Dataset Attribution
• Dataset Summary
• Supported Tasks and Leaderboards
• Languages
• Dataset Structure
  • Data Instances
  • Data Fields
  • Data Splits
• Dataset Creation
  • Curation Rationale
  • Source Data
• Dataset Use
  • Use Cases
  • Usage Caveats
  • Getting Started

🐋 The Open Orca Dataset! 🐋

We are thrilled to announce the release of the Open Orca dataset! This rich collection of augmented FLAN data aligns, as best as possible, with the distributions outlined in the Orca paper. It has been instrumental in generating high-performing model checkpoints and serves as a valuable resource for all NLP researchers and developers!

We would like to give special recognition to the following contributors for their significant efforts and dedication:

Teknium                     
Caseus
Eric Hartford
NanoBit
Pankaj
Winddude
Rohan

http://AlignmentLab.ai:
Autometa
Entropi
AtlasUnified
NeverendingToast
lightningRalf
NanoBit
Caseus

Also of course, as always, TheBloke, for being the backbone of the whole community.

Many thanks to NanoBit and Caseus, makers of Axolotl, for lending us their expertise on the platform that developed and trained manticore, minotaur, and many others!

We are welcoming sponsors or collaborators to help us build these models to the scale they deserve. Please reach out via our socials: http://Alignmentlab.ai https://discord.gg/n9hXaBPWxx

This repo is the original repo from which the entire team had agreed to work out of and publish out of from the outset. Eric's repo represents his duplication and augmentation of the team's collective effort, initiated after he had chosen to depart the team.

Dataset Summary

The Open Orca dataset is a collection of unaugmented and augmented FLAN data. Currently ~1M GPT-4 completions, and ~3.5M GPT-3.5 completions. It is tabularized in alignment with the distributions presented in the ORCA paper and currently represents a partial completion of the full intended dataset, with ongoing generation to expand its scope. The data is primarily used for training and evaluation in the field of natural language processing.

Supported Tasks and Leaderboards

This dataset supports a range of tasks including language modeling, text generation, and text augmentation. It has been instrumental in the generation of multiple high-performing model checkpoints which have exhibited exceptional performance in our unit testing. Further information on leaderboards will be updated as they become available.

Languages

The language of the data primarily is English.

Dataset Structure

Data Instances

A data instance in this dataset represents an augmented and unaugmented set of text data, containing fields for the original and modified text content.

Data Fields

The primary fields of interest are 'Original Text' and 'Augmented Text'. Other metadata fields, as well as specifics of the augmentation process used for each instance, are also included.

Data Splits

Details regarding data splits (train/test/validate) will be updated as the data generation progresses.

Dataset Creation

Curation Rationale

The dataset was created to provide a source of augmented text data for researchers and developers. It is particularly valuable in advancing the capabilities of language models, and fostering the generation of high-performing model checkpoints.

Source Data

The data is generated using techniques in alignment with the distributions outlined in the ORCA paper. The original unaugmented data comes from the FLAN dataset.

Dataset Use

Use Cases

The dataset can be used for tasks related to language understanding, natural language processing, machine learning model training, and model performance evaluation.

Usage Caveats

Given that this is a work-in-progress dataset, it's recommended to regularly check for updates and improvements. Further, the data should be used in accordance with the guidelines and recommendations outlined in the ORCA paper.

Getting Started

For information on getting started, please refer to the Hugging Face dataset loading utilities. Regular updates and data generation progress can be monitored through the Open Orca repository on Hugging Face.