Detection and Labeling of Dermatological Phenomena by Data Annotators without any Medical Background

  • Client Profile

    Industry: Electronics manufacturing
    Location: Belgium
    Size: 1,001 - 5,000 employees

  • Company Bio

    Our client is a Belgium-based global technology leader that develops networked visualization solutions for the healthcare market. Their products can be found ranging from the radiology department to the operating room.

Overview

In order to develop a smart dermatoscope that uses Artificial Intelligence to automatiсally detect skin pathologies, our client needed to label huge datasets. These datasets included images showing five different pathologies: pigment networks, negative networks, streaks, milia-like cysts, and globules. Our goal was to prove that annotators without a medical background could identify and label the right pathologies.

  • Team Members

    10

  • Annotation accuracy

    98%

  • Cost savings

    83%

Challenge

Our client produces innovative healthcare devices, the development of which involves lots of manual data labeling. The type of data to be labeled made the situation more complicated, as labeling needed to be done by professional dermatologists. In order to help our client optimize labor costs and get the labeling job done, Mindy Support created a team of eight experienced annotators, one medical supervisor, and a project manager.

Solution

Before our annotators started the job, our medical supervisor provided training materials and a short tutorial. However, it was still challenging for them to mark the pathologies quickly and with maximum accuracy. In order to deal with all of the issues that arose, Mindy Support decided to expand the training process:

  • The dermatologist delivered in-depth training while providing more examples to the annotators;

  • After training, the annotators had the chance to experiment in a test environment, where they had the chance to see 10 examples of each pathology so that they were familiar with what to look for;

  • Once they had gone through this set, they were asked to label 50 images. After they were finished, the annotators received feedback from the supervisor on which images contained a specific pathology and which images did not.

After this process, the annotators were ready to process bigger datasets of 500, 1,000, and 10,000+ images quickly and accurately.

All the annotated data was checked by the medical supervisor before delivery to the client.

Mindy Support is ISO 9001 certified. Our information security management system (ISMS) is built on the basis of ISO 27001:2013 international standards that helps organizations keep information assets secure.
We are committed to respecting all rights of the data subjects under the General Data Protection Regulation (GDPR) (EU) 2016/679. According to the Article 28 of the GDPR, the relationship between the controller and the processor are regulated by Data Processing Agreement, which we put in place with every Client.

Results

The annotation team:

  • Completed 80 hours of data training with a dermatologist

  • Annotated 10,400 images of pathologies with 98% accuracy

  • Proved that our client can accurately label their large datasets using Mindy Support annotation teams, saving as much as 83% on the cost of annotating this data

  • 10

    TEAM MEMBERS

  • 98%

    ANNOTATION ACCURACY

  • 83%

    COST SAVINGS