One day in the not-so-distant future, robots could help humans out in the workplace by taking notes or sketching helpful diagrams.
That’s one of the objectives of a new robot created by researchers from Brown University that can learn to write languages and sketch drawings practically on its own.
After learning to write Japanese characters, the robot was able to teach itself how to copy words in 10 different languages, including Hindi, Greek and English, just by studying various examples.
Scroll down for video
It uses an algorithm that helps the robot decide where and how to place each pen stroke that distinguishes each letter in the alphabet, as well as what order to place them in to make the correct word.
‘Just by looking at a target image of a word or sketch, the robot can reproduce each stroke as one continuous action,’ Atsunobi Kotani, who led the study, said in a statement.
‘That makes it hard for people to distinguish if it was written by the robot or actually written by a human.’
The robot is fed with two different algorithm models to help it learn to write on its own.
A ‘global’ model allows the robot to look at the image as a whole, to help it decide where the likely starting point is for the particular word or character, as well as how to move onto the next character.
The robot was able to teach itself words from 10 different languages, including Hindi, Greek and English
Additionally, a ‘local’ model helps the robot finish the character it’s currently working on, i.e. making the right movements, placement, etc.
This enabled the robot to be able to write out languages it had never seen before, much to the surprise of researchers.
It could reproduce the characters and the strokes that created them with about 93 percent accuracy, according to Brown University.
The robot also could recreate different character types it had never seen before, such as English print and cursive.
Being able to recreate cursive writing ‘raises the question’ of whether or not it might be able to copy handwritten signatures, the study noted.
‘Given an image of a signature, our approach could infer a policy for reproducing that signature with a pen held by a robot,’ the study explained.
‘This capability has ramifications in areas where signatures written with a writing utensil are used to verify agreement, such as legal documents.’
Researchers were equally surprised when the robot was capable of recreating a sketch of the Leonardo da Vinci’s Mona Lisa just by looking at it.
The robot uses an algorithm that helps the robot decide where and how to place each pen stroke for each letter, as well as what order to place them in to make the correct word
Researchers were equally surprised when the robot was capable of recreating a sketch of the Leonardo da Vinci’s Mona Lisa just by looking at it
Unlike an ink jet printer, which can recreate an image by going line by line, the robot could copy the drawing of the Mona Lisa using ‘human-like’ drawing strokes, researchers said.
They gave it a further test when a group of 6-year-olds wrote the word ‘hello’ in less-than-perfect handwriting on the board.
Again, the robot was able to copy their handwriting with seeming ease.
‘We would have been happy if it had only learned the Japanese characters,’ Stefanie Tellex, an assistant professor of computer science at Brown and Kotani’s advisor, said in a statement.
‘But once it started working on English, we were amazed. Then we decided to see how far we could take it.’
Kotani said he hopes the insights gathered through his research can be used to build robots capable of leaving Post-It notes, or doing dictation and sketching diagrams.
