Description – easy to read
Leuven is a city in Belgium.
The University of Leuven is making a computer program
to change words into pictures.
This program will help people with intellectual disabilties
better understand the information they are reading.
It will also help them use social media
like Facebook, Twitter or WhatsApp.
They are using two picture languages:
one is called Beta
and the other is called Sclera
Beta pictures are in color. They look like this:
Sclera pictures are black and white, like this:
In some countries, students learn these picture languages in school.
In order to ease the understanding of text based communications, pictograms incorporate a visual layer to transmit information. A large proportion of people with IDD require this type of communication layer. There are estimates that between 2 and 5 million people in the European Union could benefit from symbols or symbol-related text as a means of written communication (Keskinen et al. 2012). Pictograms are standardized image based representations of words or concepts that serve people with reduced speech or reading capabilities to express themselves or understand written texts.
The technology Text2Picto, co-developed by partners KU Leuven and TM, provides a text-to-pictogram and a pictogram-to-text translation assuring an intuitive communication between IDD users and the rest of the society. It also allows translation between different sets of pictograms. This technology, especially when applied in combination with the text simplification tool, is able to reduce cognitive complexity of tasks such as reading work documents, emails, web pages, chats, etc.
The technology maps text onto pictograms. The simplest version uses part-of-speech tagging to partially disambiguate between homonyms (different words with the same spelling), and uses lemmatisation to improve the coverage of the pictogram lexicon, by mapping word forms onto their lemmas. This system has been developed for Dutch and for the Beta and Sclera pictogram lexicons. Localisation of this version that will be used also in the pilots in Spain and UK will require the translation of the pictogram lexicon into the target language, or the use of a pictogram lexicon in the local language. Beta and Sclera pictograms are already available in English and Spanish, as well as part-of-speech taggers and lemmatizers.
As not all pictogram sets are organized around one pictogram per word, the simple approach is not fully satisfactory, as the coverage is too low. For instance, Sclera pictograms are often representing a verb and its object in one pictogram (e.g. feed the dog)
To allow the translation of text into such complex pictograms, the pictograms have been mapped onto Cornetto, a lexico-semantic database for Dutch which is linked to the WordNet grid.
On top of the coverage of the simple approach, this approach also covers the conversion of synonyms and hypernyms (for instance, if the word Labrador occurs in a text, and there is no pictogram, the pictogram for the hypernym dog will be shown).
The localisation of this approach requires the mapping of the selected pictogram sets onto the synonym sets of the wordnets of these languages. Within the project, we will be using the Open Multilingual Wordnet which maps wordnets of 20 different languages, and the Eurowordnet . Both are open source and included in the Global Wordnet project.
Try out our Text 2 Picto tool!!!