Detection and Labeling of Dermatological Phenomena by Data Annotators without any Medical Background
Industry: Electronics manufacturing
Size: 1,001 - 5,000 employees
Our client is a Belgium-based global technology leader that develops networked visualization solutions for the healthcare market. Their products can be found ranging from the radiology department to the operating room.
In order to develop a smart dermatoscope that uses Artificial Intelligence to automatiсally detect skin pathologies, our client needed to label huge datasets. These datasets included images showing five different pathologies: pigment networks, negative networks, streaks, milia-like cysts, and globules. Our goal was to prove that annotators without a medical background could identify and label the right pathologies.
Our client produces innovative healthcare devices, the development of which involves lots of manual data labeling. The type of data to be labeled made the situation more complicated, as labeling needed to be done by professional dermatologists. In order to help our client optimize labor costs and get the labeling job done, Mindy Support created a team of eight experienced annotators, one medical supervisor, and a project manager.
Before our annotators started the job, our medical supervisor provided training materials and a short tutorial. However, it was still challenging for them to mark the pathologies quickly and with maximum accuracy. In order to deal with all of the issues that arose, Mindy Support decided to expand the training process:
The dermatologist delivered in-depth training while providing more examples to the annotators;
After training, the annotators had the chance to experiment in a test environment, where they had the chance to see 10 examples of each pathology so that they were familiar with what to look for;
Once they had gone through this set, they were asked to label 50 images. After they were finished, the annotators received feedback from the supervisor on which images contained a specific pathology and which images did not.
After this process, the annotators were ready to process bigger datasets of 500, 1,000, and 10,000+ images quickly and accurately.
All the annotated data was checked by the medical supervisor before delivery to the client.
The annotation team:
Completed 80 hours of data training with a dermatologist
Annotated 10,400 images of pathologies with 98% accuracy
Proved that our client can accurately label their large datasets using Mindy Support annotation teams, saving as much as 83% on the cost of annotating this data