Will We Run Out of Data? Limits of LLM Scaling Based on Human-Generated Data

The data-supply backdrop to the chapter: high-quality human text is finite, which raises the value of provenance as generated material occupies more of the web.

Villalobos, P., et al. 'Will We Run Out of Data? Limits of LLM Scaling Based on Human-Generated Data.' ICML 2024. arXiv:2211.04325