{"id":73846,"date":"2024-08-20T00:02:25","date_gmt":"2024-08-20T04:02:25","guid":{"rendered":"https:\/\/world.edu\/?p=73846"},"modified":"2024-08-20T00:02:44","modified_gmt":"2024-08-20T04:02:44","slug":"what-is-model-collapse-an-expert-explains-the-rumours-about-an-impending-ai-doom","status":"publish","type":"post","link":"https:\/\/world.edu\/what-is-model-collapse-an-expert-explains-the-rumours-about-an-impending-ai-doom\/","title":{"rendered":"What is \u2018model collapse\u2019? An expert explains the rumours about an impending AI doom"},"content":{"rendered":"

\"\"<\/p>\n

Artificial intelligence (AI)\u00a0prophets<\/a>\u00a0and\u00a0newsmongers<\/a>\u00a0are forecasting the end of the generative AI hype, with talk of an impending catastrophic \u201cmodel collapse\u201d.<\/p>\n

But how realistic are these predictions? And what is model collapse anyway?<\/p>\n

Discussed in\u00a02023<\/a>, but popularised\u00a0more recently<\/a>, \u201cmodel collapse\u201d refers to a hypothetical scenario where future AI systems get progressively dumber due to the increase of AI-generated data on the internet.<\/p>\n

The need for data<\/strong><\/h4>\n

Modern AI systems are built using machine learning. Programmers set up the underlying mathematical structure, but the actual \u201cintelligence\u201d comes from training the system to mimic patterns in data.<\/p>\n

But not just any data. The current crop of generative AI systems needs\u00a0high quality<\/em>\u00a0data, and lots of it.<\/p>\n

 <\/p>\n

\n

To train GPT-3, OpenAI needed over 650 billion English words of text \u2013 about 200x more than the entire English Wikipedia. But this required collecting almost 100x more raw data from the internet, up to 98% of which was then filtered and discarded https:\/\/t.co\/MjF9zf6hAv<\/a><\/p>\n

\u2014 Aaron J. Snoswell, PhD (@aaronsnoswell) August 14, 2024<\/a><\/p><\/blockquote>\n