LUNA16 Banner
Join

Motivation

Lung cancer is the leading cause of cancer-related death worldwide. The National Lung Screening Trial (NLST), a randomized control trial in the U.S. including more than 50,000 high-risk subjects, showed that lung cancer screening using annual low-dose computed tomography (CT) reduces lung cancer mortality by 20% in comparison to annual screening with chest radiography [1]. In 2013, the U.S. Preventive Services Task Force (USPSTF) has given low-dose CT screening a grade B recommendation for high-risk individuals [2] and early 2015, the U.S. Centers for Medicare and Medicaid Services (CMS) has approved CT lung cancer screening for Medicare recipients. As a result of these developments, lung cancer screening programs using low-dose CT are being implemented in the United States and other countries. Computer-aided detection (CAD) of pulmonary nodules could play an important role when screening is implemented on a large scale.

Large evaluation studies investigating the performance of different state-of-the-art CAD systems are scarce. Therefore, we organize a novel CAD detection challenge using a large public LIDC-IDRI dataset. The detailed description of the challenge is now available in this article. We believe that this challenge is important for a reliable comparison of CAD algorithms and to encourage rapid development of new algorithms using state-of-the-art computer vision technology.

Challenge tracks

We invite the research community to participate in one or two of the following challenge tracks: 1. Nodule detection (NDET) Using raw CT scans, the goal is to identify locations of possible nodules, and to assign a probability for being a nodule to each location. The pipeline typically consists of two stages: candidate detection and false positive reduction. 2. False positive reduction (FPRED) Given a set of candidate locations, the goal is to assign a probability for being a nodule to each candidate location. Hence, one could see this as a classification task: nodule or not a nodule. Candidate locations will be provided in world coordinates. This set detects 1,162/1,186 nodules.

Open challenge

LUNA16 is a completely open challenge. This means that unlike other challenges, images and reference standard are publicly available. The goal of LUNA16 is to provide an opportunity for participants to test their algorithm on common database with a standardized evaluation protocol. With the spirit of speeding-up scientic progress, the results listed on the website can be used as an indication on how well state-of-the-art CAD algorithms perform. We hope LUNA16 will yield several results that are worthwhile for the CAD research community.

We are committed to maintain this site as a public repository of benchmark results for nodule detection on a common database in the spirit of cooperative scientific progress. In return, we ask everyone who uses this site to respect the rules below.

Rules

The following rules apply to those who register a team and download the data:

  • All information entered when registering a team, including the name of the contact person, the affiliation (institute, organization or company the team's contact person works for) and the e-mail address must be complete and correct. In other words, anonymous registration is not allowed. If you want to submit anonymous, for example because you want to submit your results to a conference that requires anonymous submission, please contact the organizers.
  • The LUNA16 organizers reserve the right to request a pdf file describing the system to accompany the submitted result. The organizers may refuse to evaluate systems whose description does not meet minimal requirements.
  • Results uploaded to this website will be made publicly available on this site (see the Results Section), and by submitting results, you grant us permission to do so. Obviously, teams maintain full ownership and rights to the method.
  • Teams must notify the maintainers of this site about any publication that is (partly) based on the data on this site, in order for us to maintain a list of publications associated with the LUNA16 study.

References

[1] Aberle D. R., Adams A. M., Berg C. D., Black W. C., Clapp J. D., Fagerstrom R. M., Gareen I. F., Gatsonis C., Marcus P. M., and Sicks J. D. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med, 365:395–409, 2011.   [2] Moyer VA, U.S. Preventive Services Task Force. Screening for lung cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med, 160:330-338 2014.   [3] Armato SG, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP et al. The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. Med Phys, 38:915–931, 2011.