Skip to content

Commit

Permalink
Apply automatic changes
Browse files Browse the repository at this point in the history
  • Loading branch information
mdingemanse authored and github-actions[bot] committed May 6, 2024
1 parent 26f2cb6 commit 64c4e16
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 2 deletions.
1 change: 1 addition & 0 deletions docs/df.csv
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ https://huggingface.co/LLM360/AmberChat,,Amber,ShareGPT + Evol-Instruct (synthet
https://open-assistant.io/,,Pythia 12B,OpenAssistant Conversations,Apache 2.0,LAION-AI,https://open-assistant.io/,,open,https://github.com/LAION-AI/Open-Assistant,Code includes guide for developers,open,https://github.com/LAION-AI/Open-Assistant/tree/main/data/datasets,Datasets documented in detail and recipes for cleaning up and downloading provided in code notebooks.,open,https://huggingface.co/OpenAssistant,Model weights in several variants downloadable through HuggingFace,open,https://huggingface.co/datasets/OpenAssistant/oasst1,"OpenAssistant Conversations is 'a human-generated, human-annotated assistant-style conversation corpus consisting of 161443 messages distributed across 66497 conversation trees, in 35 different languages, annotated with 461292 quality ratings' (preprint)",closed,,RLHF weights not separately released,open,https://projects.laion.ai/Open-Assistant/docs/faq#what-license-does-open-assistant-use,Apache 2.0,open,https://projects.laion.ai/Open-Assistant/docs/intro,Separate website provides entry point to comprehensive documentation,open,https://github.com/LAION-AI/Open-Assistant/tree/main/model,Instructions to tune the pipeline on training data,partial,https://arxiv.org/abs//2304.07327,"Preprint describes creation of OpenAssistant Conversations corpus for instruction tuning, but not the base LLM, hence partial.",closed,,No peer-reviewed paper or published data audit found,closed,,,closed,,,open,,,open,https://projects.laion.ai/Open-Assistant/api,,/projects/Open-Assistant.yaml,9.5
https://github.com/imoneoi/openchat,,Mistral 7B,ShareGPT with C-RLFT,Apache 2.0,Tshinghua University,https://github.com/imoneoi,OpenChat notes 'We are a student team from Tsinghua University',open,https://github.com/imoneoi/openchat/tree/master/ochat,Repository offers a large amount of fairly well-organized code for data curation and model,closed,,Pretraining data for Mistral is nowhere disclosed or documented,open,https://github.com/mistralai/mistral-src#download-the-model,Mistral 7B weights available via Mistral repository,closed,,Preprint says shareGPT dataset 'collected from sharegpt.com' but not disclosed or made available by this project,open,https://huggingface.co/openchat/openchat_3.5/tree/main,Instruction-tuned model weights made available via HuggingFace,open,https://github.com/imoneoi/openchat/blob/master/LICENSE,Code and model released under Apache 2.0,partial,https://github.com/imoneoi/openchat/tree/master/ochat,There is plenty of code in the github repository but only some of it is documented,open,https://arxiv.org/abs/2309.11235,Architecture quite well described in preprint,open,https://arxiv.org/abs/2309.11235,"Preprint describes the model architecture and instruction tuning approach, though is hampered by building on notoriously closed Llama2",open,https://openreview.net/forum?id=AOJyfhWYHf,Paper reviewed and accepted for ICLR 2024,partial,https://huggingface.co/openchat/openchat_v3.2,There is a model card that provides some details on architecture and evaluation,closed,,Datasheet not provided.,open,https://github.com/imoneoi/openchat/tree/master#installation,Python package 'ochat' provided through pip,partial,,"Model too large to load onto HuggingFace free inference API, so only available through Inference Endpoints or package",/projects/OpenChat.yaml,9.5
https://huggingface.co/togethercomputer/Pythia-Chat-Base-7B,,EleutherAI pythia,OIG,Apache 2.0 license,togethercomputer,https://github.com/togethercomputer,,open,,,open,https://github.com/togethercomputer/OpenDataHub,Training data curated and shared in separate repository,open,https://huggingface.co/togethercomputer/Pythia-Chat-Base-7B/tree/main,Model weights available via HuggingFace,open,https://huggingface.co/datasets/laion/OIG,From the documentation 'This is our attempt to create a large instruction dataset of medium quality along with a smaller high quality instruciton dataset (OIG-small-chip2).',closed,,RL weights not separately made available,open,https://huggingface.co/togethercomputer/Pythia-Chat-Base-7B#model-details,Apache 2.0,open,https://github.com/togethercomputer/OpenChatKit,Actively maintained repository,open,https://github.com/togethercomputer/OpenChatKit#reproducing-pythia-chat-base-7b,Architecture and recipe for reproducing model provided,partial,https://arxiv.org/abs/2304.01373,Preprint describes LM base (Pythia) but not instruction tuning details,closed,,No peer-reviewed paper or data audit found,partial,https://huggingface.co/togethercomputer/Pythia-Chat-Base-7B,Model card partially available but fairly minimally specified,partial,https://huggingface.co/datasets/laion/OIG,OIG instruction dataset documented,open,,,closed,,,/projects/pythia-chat-base-7B.yaml,9.5
https://huggingface.co/Intel/neural-chat-7b-v3-1,A mistral-based Orca-finetuned chat model,Mistral 7B,Orca,Apache 2.0,Intel,,,partial,https://github.com/intel/intel-extension-for-transformers/tree/main/intel_extension_for_transformers/neural_chat/examples/finetuning/finetune_neuralchat_v3,"Mistral base model is not open, but repo gives sample code for fine-tuning based on that",closed,,Mistral has not disclosed anything about its training data,open,https://huggingface.co/mistralai/Mistral-7B-v0.1,Based on Mistral 7B 0.1,open,https://huggingface.co/datasets/Open-Orca/SlimOrca,RL dataset used for post-training is shared and available,open,https://huggingface.co/Intel/neural-chat-7b-v3-1/tree/main,finetuned model made openly available,open,https://huggingface.co/Intel/neural-chat-7b-v3-1/blob/main/LICENSE,Apache 2.0,partial,,Mistral remains closed so only documentation pertains to fine-tuning steps,open,https://medium.com/intel-analytics-software/the-practice-of-supervised-finetuning-and-direct-preference-optimization-on-habana-gaudi2-a1197d8a3cd3,Described in on HuggingFace model card and a Medium post,partial,https://medium.com/intel-analytics-software/the-practice-of-supervised-finetuning-and-direct-preference-optimization-on-habana-gaudi2-a1197d8a3cd3,A medium post is apparently the only scientific documentation of this model,closed,,No peer-reviewed paper found,open,https://huggingface.co/Intel/neural-chat-7b-v3-1,,partial,https://huggingface.co/datasets/Open-Orca/OpenOrca,SlimOrca dataset is described as part of OpenOrca (but partial since Mistral data is incrutable),partial,https://huggingface.co/Intel/neural-chat-7b-v3-1#fp32-inference-with-transformers,Code for running with transformers provided by Intel,partial,,"Provided through HuggingFace but model too large to run via inference API, local deployment or paid access needed",/projects/neuralchat-7b.yaml,9.0
https://huggingface.co/SebastianSchramm/Cerebras-GPT-111M-instruction,,Cerebras,Alpaca (synthetic),Apache 2.0,Cerebras + Schramm,https://github.com/Cerebras,,partial,https://github.com/Cerebras/gigaGPT,"Some of the training code available in GigaGPT, but fine-tuning",open,https://huggingface.co/datasets/EleutherAI/pile,Eleuther AI's The Pile,open,https://huggingface.co/cerebras/Cerebras-GPT-111M,base model available via Cerebras,open,https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM/blob/main/data/alpaca_gpt4_data.json,Alpaca GPT4,open,https://huggingface.co/SebastianSchramm/Cerebras-GPT-111M-instruction/tree/main,Finetuned model weights available,partial,https://huggingface.co/SebastianSchramm/Cerebras-GPT-111M-instruction/,Licensing situation unclear as model page mentions no license (base model is licensed Apache 2.0),closed,,Code only sparsely documented,open,,Described in preprint,partial,https://arxiv.org/abs/2304.03208,,closed,,,closed,https://huggingface.co/cerebras/Cerebras-GPT-111M,Only serves as as advertising for the model,open,https://arxiv.org/abs/2201.07311,Datasheet available for The Pile,closed,,No package found,open,,Available through HuggingFace inference API,/projects/cerebras-gpt-111m-instruction.yaml,8.5
https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Instruct,,RedPajama-INCITE-7B-Base,various (GPT-JT recipe),Apache 2.0,TogetherComputer,https://together.ai/,,partial,https://github.com/togethercomputer/redpajama.cpp/tree/master/examples/redpajama,Code for datasets made available in exemplary ways; code for training and tuning harder to find,open,https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T,RedPajama-Data-1T made available on HuggingFace,open,https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Base,Base is RedPajama-INCITE-7B-Base,open,https://huggingface.co/datasets/togethercomputer/RedPajama-Data-Instruct,"The model was trained on a large collection of diverse data, including Chain-of-Thought (CoT), Public Pool of Prompts (P3) dataset, Natural-Instructions (NI) dataset.",open,https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Instruct,Instruction-tuned version made available in paralle with base version,partial,https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Instruct/blob/main/README.md,"Models licensed under Apache 2.0, but note that the data itself is variably licensed and so imposes some limitations.",partial,,Code for base LLM and instruction tuning datasets beautifully documented; code specifying training and fine-tuning sparsely documented.,partial,https://together.ai/blog/redpajama,"Architecture detailed on model card, but crucial parts appear to be forked from GPT-NeoX",closed,,No preprint found,closed,,No paper found,open,https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Instruct,Model card and readme provide details on datasets and,open,https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T,Data sheet includes links to data and recipes to create from scratch,closed,,No separate package found,partial,https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Instruct,Hosted inference API available through HuggingFace,/projects/RedPajama-INCITE-Instruct-7B.yaml,8.5
https://github.com/databrickslabs/dolly,,EleutherAI pythia,databricks-dolly-15k,,databricks,https://www.databricks.com,,open,,,open,,,open,,,open,,,closed,,,open,,,open,,,open,,,partial,https://arxiv.org/abs/2304.01373,,closed,,,closed,,,closed,,,open,,,closed,,,/projects/dolly.yaml,8.5
Expand Down
Loading

0 comments on commit 64c4e16

Please sign in to comment.