![\"\"](\"https:\/\/world.edu\/wp-content\/uploads\/2024\/04\/AIPhotoshop.jpeg\")
<\/p>\n<\/p>\n
If you\u2019ve spent time on Facebook over the past six months, you may have noticed photorealistic images that are too good to be true: children holding paintings that look like the work of professional artists, or majestic log cabin interiors that are the stuff of Airbnb dreams.<\/p>\n
Others, such as renderings of Jesus made out of crustaceans, are just bizarre.<\/p>\n
Like the AI image of the\u00a0pope in a puffer jacket<\/a>\u00a0that went viral in May 2023, these AI-generated images are increasingly prevalent \u2013 and popular \u2013 on social media platforms. Even as many of them border on the surreal, they\u2019re often used to bait engagement from ordinary users.<\/p>\n Our team of researchers from the Stanford Internet Observatory and Georgetown University\u2019s Center for Security and Emerging Technology\u00a0investigated over 100 Facebook pages<\/a>\u00a0that posted high volumes of AI-generated content. We published the results in March 2024 as a\u00a0preprint paper<\/a>, meaning the findings have not yet gone through peer review.<\/p>\n We explored patterns of images, unearthed evidence of coordination between some of the pages, and tried to discern the likely goals of the posters.<\/p>\n Page operators seemed to be posting pictures of AI-generated babies, kitchens or birthday cakes for a range of reasons.<\/p>\n There were content creators innocuously looking to grow their followings with synthetic content; scammers using pages stolen from small businesses to advertise products that don\u2019t seem to exist; and spammers sharing AI-generated images of animals while referring users to websites filled with advertisements, which allow the owners to collect ad revenue without creating high-quality content.<\/p>\n Our findings suggest that these AI-generated images draw in users \u2013 and Facebook\u2019s recommendation algorithm may be organically promoting these posts.<\/p>\n Internet spammers and scammers are nothing new.<\/p>\n For more than two decades, they\u2019ve used\u00a0unsolicited bulk email<\/a>\u00a0to promote pyramid schemes. They\u2019ve targeted\u00a0senior citizens<\/a>\u00a0while posing as Medicare representatives or computer technicians.<\/p>\n On social media, profiteers have used clickbait articles to drive users to ad-laden websites. Recall the 2016 U.S. presidential election, when Macedonian teenagers shared sensational political memes on Facebook and\u00a0collected advertising revenue<\/a>\u00a0after users visited the URLs they posted. The teens didn\u2019t care who won the election. They just wanted to\u00a0make a buck<\/a>.<\/p>\n In the early 2010s, spammers captured people\u2019s attention with ads promising that anyone could lose belly fat or learn a new language with \u201cone weird trick<\/a>.\u201d<\/p>\n AI-generated content has become another \u201cweird trick.\u201d<\/p>\n It\u2019s visually appealing and cheap to produce, allowing scammers and spammers to generate high volumes of engaging posts. Some of the pages we observed uploaded dozens of unique images per day. In doing so, they followed Meta\u2019s\u00a0own advice<\/a>\u00a0for page creators. Frequent posting, the company suggests, helps creators get the kind of algorithmic pickup that leads their content to appear in the \u201cFeed,\u201d formerly known as the \u201cNews Feed<\/a>.\u201d<\/p>\n Much of the content is still, in a sense, clickbait: Shrimp Jesus makes people pause to gawk and inspires shares purely because it is so bizarre.<\/p>\n An image of \u2018shrimp Jesus\u2019 posted from the page \u2018Love God & God Love You.\u2019<\/p><\/div>\n Many users react by liking the post or leaving a comment. This signals to the algorithmic curators that perhaps the content should be pushed into the feeds of even more people.<\/p>\n Some of the more established spammers we observed, likely recognizing this, improved their engagement by pivoting from posting URLs to posting AI-generated images. They would then comment on the post of the AI-generated images with the URLs of the ad-laden content farms they wanted users to click.<\/p>\n But more ordinary creators capitalized on the engagement of AI-generated images, too, without obviously violating platform policies.<\/p>\n When we looked up the posts\u2019 captions on CrowdTangle \u2013 a social media monitoring platform owned by Meta and\u00a0set to sunset<\/a>\u00a0in August \u2013 we found that they were\u00a0\u201ccopypasta\u201d captions<\/a>, which means that they were repeated across posts.<\/p>\n Some of the copypasta captions baited interaction by directly asking users to, for instance, rate a \u201cpainting\u201d by a first-time artist \u2013 even when the image was generated by AI \u2013 or to wish an elderly person a happy birthday. Facebook users often replied to AI-generated images with comments of encouragement and congratulations.<\/p>\n A post from the Facebook page \u2018Life Nature\u2019 with an image that was likely generated by AI. Life Nature was reportedly stolen from a band and was no longer live on Meta after the authors of this article published their investigatio<\/p><\/div>\n Our investigation noticeably altered our own Facebook feeds: Within days of visiting the pages \u2013 and without commenting on, liking or following any of the material \u2013 Facebook\u2019s algorithm recommended reams of other AI-generated content.<\/p>\n Interestingly, the fact that we had viewed clusters of, for example, AI-generated miniature cow pages didn\u2019t lead to a short-term increase in recommendations for pages focused on actual miniature cows, normal-sized cows or other farm animals. Rather, the algorithm recommended pages on a range of topics and themes, but with one thing in common: They contained AI-generated images.<\/p>\n In 2022, the technology website\u00a0Verge detailed<\/a>\u00a0an internal Facebook memo about proposed changes to the company\u2019s algorithm.<\/p>\n The algorithm, according to the memo, would become a \u201cdiscovery-engine,\u201d allowing users to come into contact with posts from individuals and pages they didn\u2019t explicitly seek out, akin to TikTok\u2019s \u201cFor You\u201d page.<\/p>\n We analyzed Facebook\u2019s own \u201cWidely Viewed Content Reports<\/a>,\u201d which lists the most popular content, domains, links, pages and posts on the platform per quarter.<\/p>\n It showed that the proportion of content that users saw from pages and people they don\u2019t follow steadily increased between 2021 and 2023. Changes to the algorithm have allowed more room for AI-generated content to be organically recommended without prior engagement \u2013 perhaps explaining our experiences\u00a0and those of other users<\/a>.<\/p>\n Since Meta currently does not flag AI-generated content by default, we sometimes observed users warning others about scams or spam AI content with infographics.<\/p>\n Meta, however, seems to be aware of potential issues if AI-generated content blends into the information environment without notice. The company has released\u00a0several<\/a>\u00a0announcements<\/a>\u00a0about how it plans to deal with AI-generated content.<\/p>\n An example of an infographic posted by users on Facebook posts with AI-generated images.<\/p><\/div>\n In May 2024, Facebook will begin applying a \u201cMade with AI\u201d label to content it can reliably detect as synthetic.<\/p>\n But the devil is in the details. How accurate will the detection models be? What AI-generated content will slip through? What content will be inappropriately flagged? And what will the public\u00a0make of such labels<\/a>?<\/p>\n While our work focused on Facebook spam and scams, there are broader implications.<\/p>\n Reporters have\u00a0written about<\/a>\u00a0AI-generated videos targeting kids on YouTube and influencers on TikTok who use generative AI to\u00a0turn a profit<\/a>.<\/p>\n Social media platforms will have to reckon with how to treat AI-generated content; it\u2019s certainly possible that user engagement will wane if online worlds become filled with artificially generated posts, images and videos.<\/p>\n Shrimp Jesus may be an obvious fake.\u00a0But the challenge of assessing what\u2019s real<\/a>\u00a0is only heating up.<\/p>\nGenerative AI meets scams and spam<\/strong><\/h4>\n
![\"\"](\"https:\/\/world.edu\/wp-content\/uploads\/2024\/04\/ShrimpJesus.jpg\")
Rate \u2018my\u2019 work!<\/strong><\/h4>\n
![\"\"](\"https:\/\/world.edu\/wp-content\/uploads\/2024\/04\/LifeNature.jpg\")
Algorithms push AI-generated content<\/strong><\/h4>\n
\u2018This post was brought to you by AI\u2019<\/strong><\/h4>\n
![\"\"](\"https:\/\/world.edu\/wp-content\/uploads\/2024\/04\/ScamAlert.jpg\")