{"id":72173,"date":"2024-03-22T01:01:55","date_gmt":"2024-03-22T05:01:55","guid":{"rendered":"https:\/\/world.edu\/?p=72173"},"modified":"2024-03-22T01:01:55","modified_gmt":"2024-03-22T05:01:55","slug":"learn-sign-language-with-generative-ai-and-3d-avatars","status":"publish","type":"post","link":"https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/","title":{"rendered":"Learn sign language with generative AI and 3D avatars"},"content":{"rendered":"

\"\"<\/p>\n

Faced with accessibility problems that affect a growing number of deaf people, researchers are interested in new technologies for learning sign languages.\u00a0In the digital field, research is now focusing on the development of artificial intelligence (AI) making it possible to automatically translate written or spoken vocal language into sign language and vice versa, using avatars signing in 3D.<\/p>\n

The targeted applications concern in particular automatic translation which makes it possible to produce content in sign language from text (SMS, web content, etc.) or speech, to textually subtitle videos in sign language, or even to design educational applications and videobooks\u00a0intended<\/em>\u00a0for deaf children.\u00a0Facilitating communication for deaf people and their access to information and thus promoting their inclusion in society is the primary objective of this work.<\/p>\n

Constraints specific to sign languages<\/strong><\/h4>\n

Sign languages \u200b\u200bare languages \u200b\u200bin their own right that constitute pillars of deaf identity and cultures.\u00a0There are as many sign languages \u200b\u200bas there are communities of deaf people, with their national variations, their specificities and their regional variants.\u00a0They are practiced not only by deaf people, but also by all people who are part of their family, educational and professional environment.\u00a0French sign language (LSF) is the gesture language practiced by deaf people in France.\u00a0Banned for a long time, it was only recognized in 2005, which may explain the fact that it is poorly equipped with resources such as lexicographic, grammatical, or even encyclopedic reference works, or even annotated digital data.<\/p>\n

The few existing sign language databases generally consist of videos, possibly accompanied by translations into written vocal language.\u00a0This is particularly true for the LSF which essentially has lexical dictionaries accessible on the Web.\u00a0Furthermore, there is no consensus in the international community working on sign languages \u200b\u200bto define a written form.\u00a0It is therefore difficult to align, as with spoken languages, between a video in sign language and the corresponding text.\u00a0The development of digital tools dedicated to sign languages \u200b\u200btherefore raises significant linguistic and technological issues.<\/p>\n

Linguistic specificities<\/strong><\/h4>\n

With their own grammars, sign languages \u200b\u200bare shaking up the theories associated with vocal or written languages.\u00a0Carried by visual and gestural modalities, they are characterized by the paralleling and simultaneity of movements and configurations of the hands, but also non-manual movements including the orientation of the bust, facial expressions and the direction of gaze.<\/p>\n

In addition, they rely on iconic and spatial dynamics.\u00a0Thus, the deaf speaker deploys his speech in the space surrounding him: he positions through his gestures the entities of his speech, animated or not, at specific locations in the space which allow these entities to be referenced and recalled.<\/p>\n

Iconicity, which is expressed by a relationship of resemblance between the gesture and what it means, manifests itself in different ways.\u00a0Certain manual configurations, carrying meaning, can represent lexical signs (a plane, a person), or be inserted in a statement.\u00a0For example, the sentence “the plane is taking off” is represented in LSF by a single sign made with two hands: one of the hands (flat shape) represents the static runway, the other hand, dynamic, represents the plane whose the movement follows its trajectory relative to the track.<\/p>\n

These iconic and spatial characteristics, omnipresent in sign languages, presuppose grammatical mechanisms which exploit the geometry in space as well as the temporality of movements.\u00a0This constitutes a technical challenge to overcome for the synthesis of movements.<\/p>\n

Create animated signing avatars with\u00a0motion capture<\/em><\/strong><\/h4>\n

From a digital point of view, video, which remains the preferred medium for the deaf, does not guarantee anonymity and imposes strong constraints on the storage and transport of data.\u00a0A suitable response for sign language production is found in\u00a0signing avatars<\/a>\u00a0, 3D animated virtual characters capable of signing utterances.\u00a0Thanks to interactive 3D interfaces integrating these avatars, it then becomes possible to edit the animations, slow them down, replay them, change point of view or zoom in on specific parts of the avatar’s body, or even modify its graphic appearance.\u00a0Designing such signing avatars requires first recording the movements of a deaf person using motion capture techniques (\u00a0motion capture<\/em>\u00a0or\u00a0mocap<\/em>\u00a0) and adapting this data to a 3D character model.<\/p>\n

Over the past decade, translating text into signed animation has been accomplished using algorithms based on\u00a0mocap<\/em>\u00a0data .\u00a0These algorithms make it possible to synthesize new statements by concatenation of existing movements.\u00a0While these approaches are satisfactory in terms of precision and quality of the sign language produced, they require very long, tedious and complex processing of\u00a0mocap<\/em>\u00a0data to automate.\u00a0Furthermore, the creation of new utterances remains constrained by the vocabulary and grammatical structures of the utterances initially recorded.\u00a0With this type of technology, it seems difficult to build large corpora of data with sentences containing semantic variations in diverse thematic domains.<\/p>\n

Generate sign language content with generative AI<\/strong><\/h4>\n

Recent advances in generative AI in the field of text, image or video generation open up obvious perspectives in terms of translation between vocal language (spoken or written) and sign language.\u00a0Current research aims to integrate these advances by developing\u00a0deep neural network<\/a>\u00a0architectures adapted to automatic language processing.\u00a0Using machine learning algorithms, they learn, by exploiting large databases, to identify and recognize trends and correlations hidden in the data in order to produce large language models that are increasingly efficient and versatile.<\/p>\n

In the context of sign languages, neural networks learn in parallel and in context linguistic (e.g. a manual configuration) and kinematic (e.g. a sequence of 2D or 3D skeletal postures) features extracted from sentences and movements respectively. .<\/p>\n

AI for the integration of deaf people<\/strong><\/h4>\n

If the performance of these systems, particularly in terms of precision of the animations produced, can be improved in the near future, they already make it possible to considerably increase the available resources, in particular by replacing videos with avatar animations. High resolution 3D.\u00a0In addition, qualified end -to-end<\/em>\u00a0methods\u00a0will make it possible to design a global neural network for complex translation tasks from one language to another.<\/p>\n

In the short term, it becomes possible to include this type of digital automatic translation tool in educational applications exploiting the respective logics of vocal and signed language grammars, thus promoting coordinated learning of the two languages.\u00a0These language resources and digital tools are essential to ensure access for deaf people to educational pathways, essential passages for the acquisition of the knowledge and skills necessary for successful integration into society.<\/p>\n

Author Bio: <\/strong>Sylvie Gibet is University Professor of Computer Science at the Universit\u00e9 Bretagne Sud<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

Faced with accessibility problems that affect a growing number of deaf people, researchers are interested in new technologies for learning sign languages.\u00a0In the digital field, research is now focusing on the development of artificial intelligence (AI) making it possible to automatically translate written or spoken vocal language into sign language and vice versa, using avatars […]\n… learn more→<\/a><\/p>","protected":false},"author":33,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[15645],"tags":[434888],"coauthors":[11733],"class_list":["post-72173","post","type-post","status-publish","format-standard","hentry","category-education","tag-sign-languages"],"acf":[],"yoast_head":"\nLearn sign language with generative AI and 3D avatars - World leading higher education information and services<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:locale\" content=\"en_US\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:type\" content=\"article\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:title\" content=\"Learn sign language with generative AI and 3D avatars - World leading higher education information and services\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:description\" content=\"Faced with accessibility problems that affect a growing number of deaf people, researchers are interested in new technologies for learning sign languages.\u00a0In the digital field, research is now focusing on the development of artificial intelligence (AI) making it possible to automatically translate written or spoken vocal language into sign language and vice versa, using avatars […]\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:url\" content=\"https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:site_name\" content=\"World leading higher education information and services\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"article:published_time\" content=\"2024-03-22T05:01:55+00:00\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:image\" content=\"https:\/\/world.edu\/wp-content\/uploads\/2024\/03\/signing-laptop.jpeg\" class=\"yoast-seo-meta-tag\" \/>\n<meta name=\"author\" content=\"Kevin\" class=\"yoast-seo-meta-tag\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" class=\"yoast-seo-meta-tag\" \/>\n\t<meta name=\"twitter:data1\" content=\"Kevin\" class=\"yoast-seo-meta-tag\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" class=\"yoast-seo-meta-tag\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" class=\"yoast-seo-meta-tag\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/world.edu\\\/learn-sign-language-with-generative-ai-and-3d-avatars\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/world.edu\\\/learn-sign-language-with-generative-ai-and-3d-avatars\\\/\"},\"author\":{\"name\":\"Kevin\",\"@id\":\"https:\\\/\\\/world.edu\\\/#\\\/schema\\\/person\\\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee\"},\"headline\":\"Learn sign language with generative AI and 3D avatars\",\"datePublished\":\"2024-03-22T05:01:55+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/world.edu\\\/learn-sign-language-with-generative-ai-and-3d-avatars\\\/\"},\"wordCount\":1094,\"image\":{\"@id\":\"https:\\\/\\\/world.edu\\\/learn-sign-language-with-generative-ai-and-3d-avatars\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/world.edu\\\/wp-content\\\/uploads\\\/2024\\\/03\\\/signing-laptop.jpeg\",\"keywords\":[\"sign languages\"],\"articleSection\":[\"Education\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/world.edu\\\/learn-sign-language-with-generative-ai-and-3d-avatars\\\/\",\"url\":\"https:\\\/\\\/world.edu\\\/learn-sign-language-with-generative-ai-and-3d-avatars\\\/\",\"name\":\"Learn sign language with generative AI and 3D avatars - World leading higher education information and services\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/world.edu\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/world.edu\\\/learn-sign-language-with-generative-ai-and-3d-avatars\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/world.edu\\\/learn-sign-language-with-generative-ai-and-3d-avatars\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/world.edu\\\/wp-content\\\/uploads\\\/2024\\\/03\\\/signing-laptop.jpeg\",\"datePublished\":\"2024-03-22T05:01:55+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/world.edu\\\/#\\\/schema\\\/person\\\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/world.edu\\\/learn-sign-language-with-generative-ai-and-3d-avatars\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/world.edu\\\/learn-sign-language-with-generative-ai-and-3d-avatars\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/world.edu\\\/learn-sign-language-with-generative-ai-and-3d-avatars\\\/#primaryimage\",\"url\":\"https:\\\/\\\/world.edu\\\/wp-content\\\/uploads\\\/2024\\\/03\\\/signing-laptop.jpeg\",\"contentUrl\":\"https:\\\/\\\/world.edu\\\/wp-content\\\/uploads\\\/2024\\\/03\\\/signing-laptop.jpeg\",\"width\":680,\"height\":453},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/world.edu\\\/learn-sign-language-with-generative-ai-and-3d-avatars\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/world.edu\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Learn sign language with generative AI and 3D avatars\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/world.edu\\\/#website\",\"url\":\"https:\\\/\\\/world.edu\\\/\",\"name\":\"World leading higher education information and services\",\"description\":\"Global Higher Education Jobs, University jobs, Lecturer & tenure jobs.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/world.edu\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/world.edu\\\/#\\\/schema\\\/person\\\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee\",\"name\":\"Kevin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g8ba4c352a279c450aa9435d770c1ebf4\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g\",\"caption\":\"Kevin\"},\"description\":\"Kevin is founder of the world.edu project. The past 28 years have been involved in publishing to the education sector in print and the internet. Kevin has a degree in Education and has a many years experience in developing companies and projects.\",\"sameAs\":[\"https:\\\/\\\/world.edu\"],\"url\":\"https:\\\/\\\/world.edu\\\/author\\\/kevin\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Learn sign language with generative AI and 3D avatars - World leading higher education information and services","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/","og_locale":"en_US","og_type":"article","og_title":"Learn sign language with generative AI and 3D avatars - World leading higher education information and services","og_description":"Faced with accessibility problems that affect a growing number of deaf people, researchers are interested in new technologies for learning sign languages.\u00a0In the digital field, research is now focusing on the development of artificial intelligence (AI) making it possible to automatically translate written or spoken vocal language into sign language and vice versa, using avatars […]","og_url":"https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/","og_site_name":"World leading higher education information and services","article_published_time":"2024-03-22T05:01:55+00:00","og_image":[{"url":"https:\/\/world.edu\/wp-content\/uploads\/2024\/03\/signing-laptop.jpeg","type":"","width":"","height":""}],"author":"Kevin","twitter_misc":{"Written by":"Kevin","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/#article","isPartOf":{"@id":"https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/"},"author":{"name":"Kevin","@id":"https:\/\/world.edu\/#\/schema\/person\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee"},"headline":"Learn sign language with generative AI and 3D avatars","datePublished":"2024-03-22T05:01:55+00:00","mainEntityOfPage":{"@id":"https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/"},"wordCount":1094,"image":{"@id":"https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/#primaryimage"},"thumbnailUrl":"https:\/\/world.edu\/wp-content\/uploads\/2024\/03\/signing-laptop.jpeg","keywords":["sign languages"],"articleSection":["Education"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/","url":"https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/","name":"Learn sign language with generative AI and 3D avatars - World leading higher education information and services","isPartOf":{"@id":"https:\/\/world.edu\/#website"},"primaryImageOfPage":{"@id":"https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/#primaryimage"},"image":{"@id":"https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/#primaryimage"},"thumbnailUrl":"https:\/\/world.edu\/wp-content\/uploads\/2024\/03\/signing-laptop.jpeg","datePublished":"2024-03-22T05:01:55+00:00","author":{"@id":"https:\/\/world.edu\/#\/schema\/person\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee"},"breadcrumb":{"@id":"https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/#primaryimage","url":"https:\/\/world.edu\/wp-content\/uploads\/2024\/03\/signing-laptop.jpeg","contentUrl":"https:\/\/world.edu\/wp-content\/uploads\/2024\/03\/signing-laptop.jpeg","width":680,"height":453},{"@type":"BreadcrumbList","@id":"https:\/\/world.edu\/learn-sign-language-with-generative-ai-and-3d-avatars\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/world.edu\/"},{"@type":"ListItem","position":2,"name":"Learn sign language with generative AI and 3D avatars"}]},{"@type":"WebSite","@id":"https:\/\/world.edu\/#website","url":"https:\/\/world.edu\/","name":"World leading higher education information and services","description":"Global Higher Education Jobs, University jobs, Lecturer & tenure jobs.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/world.edu\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/world.edu\/#\/schema\/person\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee","name":"Kevin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g8ba4c352a279c450aa9435d770c1ebf4","url":"https:\/\/secure.gravatar.com\/avatar\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g","caption":"Kevin"},"description":"Kevin is founder of the world.edu project. The past 28 years have been involved in publishing to the education sector in print and the internet. Kevin has a degree in Education and has a many years experience in developing companies and projects.","sameAs":["https:\/\/world.edu"],"url":"https:\/\/world.edu\/author\/kevin\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/posts\/72173","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/users\/33"}],"replies":[{"embeddable":true,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/comments?post=72173"}],"version-history":[{"count":2,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/posts\/72173\/revisions"}],"predecessor-version":[{"id":72186,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/posts\/72173\/revisions\/72186"}],"wp:attachment":[{"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/media?parent=72173"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/categories?post=72173"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/tags?post=72173"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/coauthors?post=72173"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}