Image Recognition

Deep-learning algorithm helps to diagnose conditions that aren’t readily apparent to doctors or researchers.

In a paper1 published on 7 January in Nature Medicine, researchers describe the technology behind the diagnostic aid, a smartphone app called Face2Gene. It relies on machine-learning algorithms and brain-like neural networks to classify distinctive facial features in photos of people with congenital and neurodevelopmental disorders. Using the patterns that it infers from the pictures, the model homes in on possible diagnoses and provides a list of likely options.

Doctors have been using the technology as an aid, even though it's not intended to provide definitive diagnoses. But it does raise a number of ethical and legal concerns, say researchers. These include ethnic bias in training data sets and the commercial fragmentation of databases, both of which could limit the reach of the diagnostic tool.

Researchers at FDNA, a digital-health company in Boston, Massachusetts, first trained the artificial intelligence (AI) system to distinguish Cornelia de Lange syndrome and Angelman syndrome — two conditions with distinct facial features — from other similar conditions. They also taught the model to classify different genetic forms of a third disorder known as Noonan syndrome.

Then the researchers, led by FDNA chief technology officer Yaron Gurovich, fed the algorithm more than 17,000 images of diagnosed cases spanning 216 distinct syndromes. When presented with new images of people’s faces, the app’s best diagnostic guess was correct in about 65% of cases. And when considering multiple predictions, Face2Gene's top-ten list contained the right diagnosis about 90% of the time.

Narrowing the field

Eventually, FDNA wants to develop this technology to help other companies filter, prioritize and interpret genetic variants of unknown significance during DNA analysis. But to train its models, FDNA needs data.

So the Face2Gene app is currently available for free to healthcare professionals, many of whom use the system as a kind of second opinion for diagnosing rarely seen genetic disorders, says study co-author Karen Gripp, a medical geneticist at the Nemours/Alfred I. duPont Hospital for Children in Wilmington, Delaware. It can also provide a starting point in cases in which a doctor doesn’t know what to make of a patient’s symptoms. “It’s like a Google search,” Gripp says.

Gripp, who is also FDNA’s chief medical officer, used the algorithm to help diagnose Wiedemann–Steiner syndrome in a young girl she treated last August. Although a little short for her age, the four-year-old didn’t have many of the syndrome’s distinguishing physical features, other than the fact she had lost most of her baby teeth and several adult teeth were already coming in.

Gripp had read case reports describing premature dental growth in children with Wiedemann–Steiner syndrome, an exceedingly rare disorder caused by mutations in a gene called KMT2A. To shore up confidence in the diagnosis, Gripp uploaded a photo of her young patient to Face2Gene. Wiedemann–Steiner syndrome appeared among the software’s top hits.

Gripp subsequently confirmed the girl’s diagnosis with a targeted DNA test. But she says that the AI approach helped her to narrow down the possibilities and saved the cost of more expensive multi-gene panel testing.

‘Killing it’

The program’s accuracy has improved slightly as more healthcare professionals upload patient photos to the app, says Gurovich. There are now some 150,000 images in its database.

And in an unofficial comparison conducted between Face2Gene and clinicians last August at a workshop on birth defects, the program outperformed the people. Charles Schwartz, a geneticist at the Greenwood Genetic Center in Greenwood, South Carolina, distributed facial pictures of ten children with “fairly recognizable” syndromes and asked attendees to come up with the correct diagnoses.

In only two instances did more than 50% of the 49 participating clinical geneticists pick the right syndrome. Face2Gene made the right call for seven of the pictures.

“We failed miserably, and Face2Gene killed it,” says Paul Kruszka, a clinical geneticist at the US National Human Genome Research Institute in Bethesda, Maryland. Soon, he says, “I think every paediatrician and geneticist will have an app like this and will use it just like their stethoscope”.

Silos and bias

But the algorithm is only as good as its training data set — and there’s a risk, especially where rare disorders that affect only small numbers of people worldwide are concerned, that companies and researchers will begin to silo and commodify their data sets. “That threatens the main potential good of this technology,” says Christoffer Nellåker, a computational biologist at the University of Oxford, UK, who has spearheaded efforts to facilitate data-sharing in this field.

And ethnic bias in training data sets that contain mostly Caucasian faces remains a concern. A 2017 study2 of children with an intellectual disability found that whereas Face2Gene’s recognition rate for Down syndrome was 80% among white Belgian children, it was just 37% for black Congolese children. With a more-diverse training data set, however, the algorithm’s accuracy for African faces improved, showing that more-equitable representation of diverse populations is achievable.

“We know this problem needs to be addressed,” says Gurovich, “and as we move forward we’re able to have less and less bias.”


Unlike humans and animals, machines have a very hard time in order to recognize a particular object and images. With recent developments in the field of machine learning and deep learning, researchers have been successful in solving this issue and proving the machines with image recognition ability.

In simple terms, image recognition is the machine’s ability to see and recognize certain objects, people, actions etc. A combination of machine vision technology and artificial intelligence is used in order to achieve image recognition.

How does it work?

The working of this technology is a fairly complex procedure. In basic terms, the computer system is trained with tons of images in the building and developing process so that it is able to detect an object from different angles, analyse the corners and edges of the object etc. and form a rough image or a 3D model and then compare it with the images that gets automatically stored in its system with the help of deep learning technology.

The more images the computer recognizes, the more accurate it will be. The best and most accurate image recognition is performed on the convolutional neural net processor which is a processor that is developed just like the neurons in the human brain. 

Uses: -

Image recognition has been extensively used in two major industries that are listed below.

  1. E-commerce: - E-commerce industry is the largest industry that has taken measures in order to adopt this technology in their businesses. The companies aim at converting your smartphones into virtual showrooms and presenting the people the ability to have a more interactive experience while shopping online by making everything they see searchable.
  2. Automobiles: - Image recognition is an important part of the smart self-driving cars which are expected to see an obstacle and take necessary measures like braking, warning the driver etc. Reading road signs, slowing down near zebra crossings are examples of the inculcation of this technology in the industry.
  3. Image recognition on Social Media: - Various social media sites are extensively using image recognition in their operations such as tagging people, recognizing the location of the photo uploaded on Facebook, distinguishing between food, people, locations etc. and presenting the image to respective communities or pages etc.
  4. Security: - Facial and image recognition is a major substitute for the login password and id that we put in while logging in our favourite apps and websites like Facebook, YouTube, Snapchat etc.

 Other Applications: -

Current and future application examples of this technology is interactive media, smart photo libraries, image viewing for the visually impaired, specific target advertisements, image search, augmented reality, solving Sudoku puzzles etc.

Page 2 of 2

© copyright 2017 All Rights Reserved.

A Product of HunterTech Ventures