{"id":80380,"date":"2026-05-27T00:13:25","date_gmt":"2026-05-27T04:13:25","guid":{"rendered":"https:\/\/world.edu\/?p=80380"},"modified":"2026-05-27T00:13:25","modified_gmt":"2026-05-27T04:13:25","slug":"should-ais-be-required-to-report-a-human-user-contemplating-violence","status":"publish","type":"post","link":"https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/","title":{"rendered":"Should AIs be required to report a human user contemplating violence?"},"content":{"rendered":"

\"\"<\/p>\n

On Feb. 10, 2026, an 18-year-old woman, Jesse Van Rootselaar, killed eight people and herself in a\u00a0mass shooting<\/a>\u00a0in Tumbler Ridge, British Columbia. OpenAI had previously flagged her ChatGPT conversations as having a disturbing fascination with extreme violence, and suspended her account, but reportedly the company\u00a0did not notify<\/a>\u00a0law enforcement.<\/p>\n

On Oct. 2, 2025, a young man named Jonathan Gavalas in Jupiter, Florida, took his own life after developing what his\u00a0father\u2019s lawsuit<\/a>\u00a0described as a romantic attachment to Google\u2019s Gemini chatbot. The suit claimed that Gemini coached Gavalas to shed his own body. The suit said Google had flagged Gavalas\u2019s account 38 times over five weeks for sensitive content, but didn\u2019t restrict or cut off the account.<\/p>\n

These tragedies and others show that generative AI can\u00a0potentially play a role<\/a>\u00a0in\u00a0harming people, organizations and the environment<\/a>. I\u2019m a\u00a0legal scholar<\/a>\u00a0who has focused on AI liability for nearly a decade and explored new ways of analyzing AI companies\u2019 responsibilities. In my view, cases like these force questions the legal community has not come to terms with: If an AI company becomes aware of warning signs about harm, does it have a legal obligation to at least warn the appropriate authorities? And if the company doesn\u2019t intervene, should its failure to act be considered negligence?<\/p>\n

A need to raise red flags<\/strong><\/h4>\n

U.S. tort law provides a framework for thinking about this type of responsibility. In 1969 a\u00a0University of California psychiatric patient<\/a>\u00a0named Prosenjit Poddar told his therapist he intended to kill a woman named Tatiana Tarasoff. The therapist notified campus police, who briefly detained Poddar but eventually let him go. Nobody warned Tarasoff, and Poddar killed her shortly after.<\/p>\n

Her family sued the university, arguing that its lack of warning amounted to negligence. In 1976 the California Supreme Court ruled that when a mental health professional has good reason to believe a client poses\u00a0a serious danger<\/a>\u00a0to an identifiable person, they have a legal duty to take reasonable steps to protect that person, including warning them or notifying law enforcement. Today, most U.S. states\u00a0recognize some version of the Tarasoff duty<\/a>\u00a0to protect or warn.<\/p>\n

The logic is simple: If you have special knowledge of a serious threat and are in a position to address it, even if only to warn the authorities or the potential victim, the law may require you to act. But does that logic apply to AI companies?<\/p>\n

The argument for yes\u00a0is appealing<\/a>. AI platforms interact with millions of users daily, often about\u00a0deeply personal matters<\/a>\u00a0such as mental health struggles, relationship problems and violent thoughts. Most companies have\u00a0systems to detect<\/a>\u00a0conversations that raise red flags.<\/p>\n

Requiring a response might be less controversial for AI than for a human therapist. Therapists are bound by strict confidentiality obligations that make warning third parties ethically and legally complicated. AI companies operate under much\u00a0weaker rules<\/a>, at least in the U.S., where no comprehensive federal privacy law exists.<\/p>\n

That lesser restriction makes it easier to justify requiring AI companies to act when it seems that someone\u2019s life may be at risk. But balancing that with\u00a0protecting privacy<\/a>\u00a0is still important.<\/p>\n

Who to warn, and when<\/strong><\/h4>\n

The first challenge in applying the Tarasoff framework to the AI world is accuracy.\u00a0Predicting violence is hard<\/a>, even for trained mental health professionals. AI systems, or human moderators who review flagged content, are not clinicians. Requiring them to judge who poses a genuine threat could lead to numerous false positives, with real consequences for people whose accounts are suspended or whose information is shared with authorities based on misread signals.<\/p>\n

The second challenge is scale. A therapist sees dozens of patients. AI platforms have\u00a0hundreds of millions of users<\/a>. Imposing a duty to monitor and act on worrisome content could create perverse incentives. AI companies might reduce their monitoring to avoid acquiring knowledge that would trigger a legal duty, reasoning that\u00a0what they do not know cannot make them liable<\/a>.<\/p>\n

The third challenge is identifying who is at risk. In the 1969 case, Poddar had named Tarasoff as a potential victim. But in many AI interactions, violent or self-destructive language is diffuse and doesn\u2019t identify a target. Courts will need to develop clear standards for when a threat is specific enough to trigger a duty to warn, and to whom any warning or protective action should be directed.<\/p>\n

Growing urgency<\/strong><\/h4>\n

The AI industry is\u00a0expanding rapidly<\/a>, yet the legal rules governing what AI companies owe their users and the public are deeply unclear. Courts are beginning to grapple with questions case by case, such as whether OpenAI\u00a0bears any responsibility<\/a>\u00a0for a gunman accused of killing two students at Florida State University on April 17, 2025. The gunman in that case was armed with a semi-automatic pistol and allegedly had extensive conversations with ChatGPT about how to use the weapon most effectively .<\/p>\n

A narrow, carefully defined duty to warn, triggered only when an AI system flags a user\u2019s behavior and it is reviewed by humans, would be a meaningful step forward. And it could focus initially on the most serious and credible threats.<\/p>\n

The practice could also shift the conversation away from thorny technical debates about whether AI chatbots are products, services or media, which\u00a0complicates legal claims<\/a>, toward a more human question: Did this company know someone was in danger, and did it do enough to warn them and authorities?<\/p>\n

Author Bio:<\/strong> Anat Lior is Assistant Professor of Law at Drexel University<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

On Feb. 10, 2026, an 18-year-old woman, Jesse Van Rootselaar, killed eight people and herself in a\u00a0mass shooting\u00a0in Tumbler Ridge, British Columbia. OpenAI had previously flagged her ChatGPT conversations as having a disturbing fascination with extreme violence, and suspended her account, but reportedly the company\u00a0did not notify\u00a0law enforcement. On Oct. 2, 2025, a young man […]\n… learn more→<\/a><\/p>","protected":false},"author":33,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[85],"tags":[1087],"coauthors":[11733],"class_list":["post-80380","post","type-post","status-publish","format-standard","hentry","category-social-change","tag-violence"],"acf":[],"yoast_head":"\nShould AIs be required to report a human user contemplating violence? - World leading higher education information and services<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:locale\" content=\"en_US\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:type\" content=\"article\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:title\" content=\"Should AIs be required to report a human user contemplating violence? - World leading higher education information and services\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:description\" content=\"On Feb. 10, 2026, an 18-year-old woman, Jesse Van Rootselaar, killed eight people and herself in a\u00a0mass shooting\u00a0in Tumbler Ridge, British Columbia. OpenAI had previously flagged her ChatGPT conversations as having a disturbing fascination with extreme violence, and suspended her account, but reportedly the company\u00a0did not notify\u00a0law enforcement. On Oct. 2, 2025, a young man […]\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:url\" content=\"https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:site_name\" content=\"World leading higher education information and services\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"article:published_time\" content=\"2026-05-27T04:13:25+00:00\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:image\" content=\"https:\/\/world.edu\/wp-content\/uploads\/2026\/05\/shooting-at-Florida-State-University.-.jpg\" class=\"yoast-seo-meta-tag\" \/>\n\t<meta property=\"og:image:width\" content=\"680\" class=\"yoast-seo-meta-tag\" \/>\n\t<meta property=\"og:image:height\" content=\"453\" class=\"yoast-seo-meta-tag\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" class=\"yoast-seo-meta-tag\" \/>\n<meta name=\"author\" content=\"Kevin\" class=\"yoast-seo-meta-tag\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" class=\"yoast-seo-meta-tag\" \/>\n\t<meta name=\"twitter:data1\" content=\"Kevin\" class=\"yoast-seo-meta-tag\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" class=\"yoast-seo-meta-tag\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" class=\"yoast-seo-meta-tag\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/world.edu\\\/should-ais-be-required-to-report-a-human-user-contemplating-violence\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/world.edu\\\/should-ais-be-required-to-report-a-human-user-contemplating-violence\\\/\"},\"author\":{\"name\":\"Kevin\",\"@id\":\"https:\\\/\\\/world.edu\\\/#\\\/schema\\\/person\\\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee\"},\"headline\":\"Should AIs be required to report a human user contemplating violence?\",\"datePublished\":\"2026-05-27T04:13:25+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/world.edu\\\/should-ais-be-required-to-report-a-human-user-contemplating-violence\\\/\"},\"wordCount\":930,\"image\":{\"@id\":\"https:\\\/\\\/world.edu\\\/should-ais-be-required-to-report-a-human-user-contemplating-violence\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/world.edu\\\/wp-content\\\/uploads\\\/2026\\\/05\\\/shooting-at-Florida-State-University.-.jpg\",\"keywords\":[\"violence\"],\"articleSection\":[\"Social Change\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/world.edu\\\/should-ais-be-required-to-report-a-human-user-contemplating-violence\\\/\",\"url\":\"https:\\\/\\\/world.edu\\\/should-ais-be-required-to-report-a-human-user-contemplating-violence\\\/\",\"name\":\"Should AIs be required to report a human user contemplating violence? - World leading higher education information and services\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/world.edu\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/world.edu\\\/should-ais-be-required-to-report-a-human-user-contemplating-violence\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/world.edu\\\/should-ais-be-required-to-report-a-human-user-contemplating-violence\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/world.edu\\\/wp-content\\\/uploads\\\/2026\\\/05\\\/shooting-at-Florida-State-University.-.jpg\",\"datePublished\":\"2026-05-27T04:13:25+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/world.edu\\\/#\\\/schema\\\/person\\\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/world.edu\\\/should-ais-be-required-to-report-a-human-user-contemplating-violence\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/world.edu\\\/should-ais-be-required-to-report-a-human-user-contemplating-violence\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/world.edu\\\/should-ais-be-required-to-report-a-human-user-contemplating-violence\\\/#primaryimage\",\"url\":\"https:\\\/\\\/world.edu\\\/wp-content\\\/uploads\\\/2026\\\/05\\\/shooting-at-Florida-State-University.-.jpg\",\"contentUrl\":\"https:\\\/\\\/world.edu\\\/wp-content\\\/uploads\\\/2026\\\/05\\\/shooting-at-Florida-State-University.-.jpg\",\"width\":680,\"height\":453},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/world.edu\\\/should-ais-be-required-to-report-a-human-user-contemplating-violence\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/world.edu\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Should AIs be required to report a human user contemplating violence?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/world.edu\\\/#website\",\"url\":\"https:\\\/\\\/world.edu\\\/\",\"name\":\"World leading higher education information and services\",\"description\":\"Global Higher Education Jobs, University jobs, Lecturer & tenure jobs.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/world.edu\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/world.edu\\\/#\\\/schema\\\/person\\\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee\",\"name\":\"Kevin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g8ba4c352a279c450aa9435d770c1ebf4\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g\",\"caption\":\"Kevin\"},\"description\":\"Kevin is founder of the world.edu project. The past 28 years have been involved in publishing to the education sector in print and the internet. Kevin has a degree in Education and has a many years experience in developing companies and projects.\",\"sameAs\":[\"https:\\\/\\\/world.edu\"],\"url\":\"https:\\\/\\\/world.edu\\\/author\\\/kevin\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Should AIs be required to report a human user contemplating violence? - World leading higher education information and services","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/","og_locale":"en_US","og_type":"article","og_title":"Should AIs be required to report a human user contemplating violence? - World leading higher education information and services","og_description":"On Feb. 10, 2026, an 18-year-old woman, Jesse Van Rootselaar, killed eight people and herself in a\u00a0mass shooting\u00a0in Tumbler Ridge, British Columbia. OpenAI had previously flagged her ChatGPT conversations as having a disturbing fascination with extreme violence, and suspended her account, but reportedly the company\u00a0did not notify\u00a0law enforcement. On Oct. 2, 2025, a young man […]","og_url":"https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/","og_site_name":"World leading higher education information and services","article_published_time":"2026-05-27T04:13:25+00:00","og_image":[{"width":680,"height":453,"url":"https:\/\/world.edu\/wp-content\/uploads\/2026\/05\/shooting-at-Florida-State-University.-.jpg","type":"image\/jpeg"}],"author":"Kevin","twitter_misc":{"Written by":"Kevin","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/#article","isPartOf":{"@id":"https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/"},"author":{"name":"Kevin","@id":"https:\/\/world.edu\/#\/schema\/person\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee"},"headline":"Should AIs be required to report a human user contemplating violence?","datePublished":"2026-05-27T04:13:25+00:00","mainEntityOfPage":{"@id":"https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/"},"wordCount":930,"image":{"@id":"https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/#primaryimage"},"thumbnailUrl":"https:\/\/world.edu\/wp-content\/uploads\/2026\/05\/shooting-at-Florida-State-University.-.jpg","keywords":["violence"],"articleSection":["Social Change"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/","url":"https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/","name":"Should AIs be required to report a human user contemplating violence? - World leading higher education information and services","isPartOf":{"@id":"https:\/\/world.edu\/#website"},"primaryImageOfPage":{"@id":"https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/#primaryimage"},"image":{"@id":"https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/#primaryimage"},"thumbnailUrl":"https:\/\/world.edu\/wp-content\/uploads\/2026\/05\/shooting-at-Florida-State-University.-.jpg","datePublished":"2026-05-27T04:13:25+00:00","author":{"@id":"https:\/\/world.edu\/#\/schema\/person\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee"},"breadcrumb":{"@id":"https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/#primaryimage","url":"https:\/\/world.edu\/wp-content\/uploads\/2026\/05\/shooting-at-Florida-State-University.-.jpg","contentUrl":"https:\/\/world.edu\/wp-content\/uploads\/2026\/05\/shooting-at-Florida-State-University.-.jpg","width":680,"height":453},{"@type":"BreadcrumbList","@id":"https:\/\/world.edu\/should-ais-be-required-to-report-a-human-user-contemplating-violence\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/world.edu\/"},{"@type":"ListItem","position":2,"name":"Should AIs be required to report a human user contemplating violence?"}]},{"@type":"WebSite","@id":"https:\/\/world.edu\/#website","url":"https:\/\/world.edu\/","name":"World leading higher education information and services","description":"Global Higher Education Jobs, University jobs, Lecturer & tenure jobs.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/world.edu\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/world.edu\/#\/schema\/person\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee","name":"Kevin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g8ba4c352a279c450aa9435d770c1ebf4","url":"https:\/\/secure.gravatar.com\/avatar\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g","caption":"Kevin"},"description":"Kevin is founder of the world.edu project. The past 28 years have been involved in publishing to the education sector in print and the internet. Kevin has a degree in Education and has a many years experience in developing companies and projects.","sameAs":["https:\/\/world.edu"],"url":"https:\/\/world.edu\/author\/kevin\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/posts\/80380","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/users\/33"}],"replies":[{"embeddable":true,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/comments?post=80380"}],"version-history":[{"count":1,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/posts\/80380\/revisions"}],"predecessor-version":[{"id":80382,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/posts\/80380\/revisions\/80382"}],"wp:attachment":[{"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/media?parent=80380"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/categories?post=80380"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/tags?post=80380"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/coauthors?post=80380"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}