Reduce reproduction cost 96%, from $600 to $24, by releasing the instruct dataset only

[MarkSchmidty]

The blog post says $500 was spent producing the dataset.
The blog post also says $100 was spent on 3xA100 80GB for 3 hours.
The market rate for 4xA100 is around $8 per hour. (See vast.ai for example)

If the dataset is provided for fine tuning then Alpaca could be reproduce for just about $24 and we would not have to wait for Facebook's response regarding sharing of the pre-trained model.

[MarkSchmidty]

Or is alpaca_data.json the dataset?

In which case the reproduction cost is already $24, not $600.

[lxuechen]

Hi Mark,

You're right alpaca_data.json is our released dataset. We're also releasing the recipe for producing the dataset, so other researchers can build on this.

You're correct that excluding the cost of reproducing the data, the cost of training the model is much lower.