Labeling Biopsied Lesions in DBT Movies

  • Client Profile

    Industry: Medical Software
    Location: California, USA
    Size: 50-100 employees

  • Company Bio

    Our client is developing investigational physics based artificial intelligence and deep machine learning solutions to help radiologists get more accurate readings of breast images.

  • Speed

    200

    studies/day

  • Target Time

    10

    min/study

  • Volume

    4,000

    studies

Challenge

Our team needed to annotate with bounding boxes approximately 4,000 mammograms with 2D and 3D biopsy studies that were extracted from multiple clinical sites. All mammograms were supported by radiologist notes that consisted of:
a. Which breast the lesion was found in (left or right)
b. The location of the lesion in the breast (e.g. depth, clock angle.)
c. The type of lesion (e.g. mass, calcs.)

Human Resources

Our client wanted annotators without a medical background for this task. Therefore, Mindy Support built a team of one project manager, one team lead with 5+ years of medical experience, and four data annotators experienced in medical annotation.

Tool

For this project, we used Three Palm Software Workstation One. We consider this tool to be one of the best on the market for reviewing and annotating digital mammograms with bounding boxes.

Solution Process

  • To help annotators better understand the location of lesions, our operations team designed a scheme for both CC and LMO projections with zones and clock positions.

  • The team processed about 200 studies per day, working five days a week in order to complete the whole task in one month.

  • The annotations were checked by the team lead for final approval.

Mindy Support is ISO 9001 and ISO 27001 certified. Our information security management system (ISMS) is built on the basis of international standards that helps organizations keep information assets secure.

We are committed to respecting all rights of the data subjects under the General Data Protection Regulation (GDPR) (EU) 2016/679. According to the Article 28 of the GDPR, the relationship between the controller and the processor are regulated by Data Processing Agreement, which we put in place with every Client.

Results

  • We detected two types of lesions: microcalcifications and architectural distortions.

  • We reviewed and annotated 4,000 positive studies.

  • Each annotator processed 50 studies working full time over 4 weeks.

  • 98%

    Accuracy

  • 70%

    Cost-effectiveness

  • 6K+

    Lesions Detected