AI Assists in COVID-19 Diagnosis and Prognosis

^{Published April 22, 2020}

On January 30, 2020, the 2019 novel coronavirus disease (COVID-19) was declared to be a global health emergency by the World Health Organization. Four months later, the virus is still spreading all over the globe—more than 3.3 million confirmed cases and 235,000 deaths worldwide—with the United States the most affected nation, numbering more than 1.1 million cases and over 65,000 deaths. Dramatic containment measures have been put into place to halt the diffusion of the virus, yet worldwide health care systems are still struggling with the massive influx of COVID-19 patients.

Currently, reverse transcription–polymerase chain reaction (RT-PCR) serves as the gold standard for the diagnosis of COVID-19. However, chest radiography and CT play an important role in the management of patients affected by COVID-19 from diagnosis to treatment response assessment, depending on the clinical situation and particularly in the early days of the outbreak and in specific geographic areas where RT-PCR tests are not readily available. In these situations, chest radiography as first-line imaging and chest CT in complex cases can provide assistance to clinicians by identifying suspicious findings for COVID-19.Xu Z, Shi L, Wang Y, et al. Case report pathological findings of COVID-19 associated with acute respiratory distress syndrome. Lancet Respir 2020; 8:420–422

Wong HYF, Lam HYS, Fong AH-T, et al. Frequency and distribution of chest radiographic findings in COVID-19 positive patients. Radiology 2019; 27:201160

Zhong B-L, Luo W, Li H-MH, et al. Sensitivity of chest CT for COVID-19: comparison to RT-PCR. Lancet 2020; 395:A1–A2

Lee YP, Jin Y, Fangfang Y, et al. Imaging profile of the COVID-19 infection: radiologic findings and literature review. Radiology 2020 Feb 13 [Epub ahead of print] Besides diagnosis, these images can be used to analyze or predict disease progression and severity. In the long term, chest CT imaging will likely play a role in the follow-up of patients with COVID-19, with possible development of long-term sequela, such as pulmonary fibrosis.

Artificial intelligence (AI) algorithms applied to patients with confirmed COVID-19 or subjects under investigation offer the potential to develop a more accurate automated approach for early detection and prognostication using the combination of clinical and imaging data. At the moment, several AI solutions are being developed for application in different stages of the COVID-19 diagnostic workflow, from diagnosis to prognosis.

AI for Classification of COVID-19 Pneumonia

In the early COVID-19 outbreak, radiographic and CT evaluations have been extensively utilized for diagnostic purposes due to their fast acquisition times. AI can be applied to develop algorithms that quickly learn COVID-19 pulmonary patterns from large datasets, as well as using similar manifestations from other types of pneumonia.

Radiography-Based AI Classification

Chest radiography is often used as an initial imaging test. Although generally considered less sensitive than chest CT, chest radiography can provide important information about the pulmonary status of COVID-19 patients, especially in more severe cases. A study by Wong et al. reported that abnormal chest radiographic examinations were found in 69% of patients at admission and 80% of patients at a later time during hospitalizationWong HYF, Lam HYS, Fong AH-T, et al. Frequency and distribution of chest radiographic findings in COVID-19 positive patients. Radiology 2019; 27:201160. COVID-19 presents itself mainly as airspace opacities, ground-glass opacity (GGO), and consolidation at a later stage. Bilateral, peripheral, and lower-zone involvement is observed in 90% of cases, while pleural effusion is rarely described. There are a few AI studies using radiographic images to detect and diagnose COVID-19-related pneumonia from other types of pneumonia and healthy subjects. Wang et al. proposed a deep convolutional network to classify COVID-19-related pneumonia using the largest COVID-19-related database so far, including radiographic examinations in 1,203 healthy patients, 660 patients with viral pneumonia, and 45 patients with COVID-19 Wang L, Wong A. COVID-Net: a tailored deep convolutional neural network design for detection of COVID-19 cases from chest x-ray images. arXiv website. arxiv.org/abs/2003.09871. Published Mar 22, 2020. Updated Apr 15, 2020. Accessed May 7, 2020 . They achieved an overall accuracy of 83.5%. Ghoshal et al. reported the use of a Bayesian convolutional neural COVID-19 classification using 70 chest radiographic images of patients with COVID-19, obtained from an online COVID-19 dataset, and images of patients without COVID-19 obtained from Kaggle’s Pneumonia Chest X-Ray Challenge Ghoshal B, Tucker A. Estimating uncertainty and interpretability in deep learning for coronavirus (COVID-19) detection. arXiv website. arxiv.org/abs/2003.10769. Published Mar 22, 2020. Updated Mar 27, 2020. Accessed May 7, 2020. This study showed heat maps to visualize the locations used by the network to classify COVID-19-related pneumonia, increasing the transparency of the AI process, and they obtained a 92.9% accuracy for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) detection.

From a recent review paper, the overall accuracy of AI-based radiographic algorithms for the classification of COVID-19-related pneumonia was pretty good, ranging between 83.5% and 98% Shi F, Wang J, Shi J, et al. Review of artificial intelligence techniques in imaging data acquisition, segmentation and diagnosis for COVID-19. IEEE Rev Biomed Eng 2020 Apr 16 [Epub ahead of print].

CT-Based AI Classification

Chest CT images are considered more sensitive for the visualization of COVID-19-related pulmonary manifestations. Several studies have described radiological chest CT patterns, characterizing different stages of the disease. Early signs of the disease are ground-glass nodules, especially subpleural in the lower lobes, which can be found both unilaterally and bilaterally. In the following stages, diffuse ground-glass nodules, “crazy-paving” pattern, and even consolidation can be found, often bilaterally in distribution encompassing multiple lobes Lee YP, Jin Y, Fangfang Y, et al. Imaging profile of the COVID-19 infection: radiologic findings and literature review. Radiology 2020 Feb 13 [Epub ahead of print]

Pan F, Ye T, Sun P, et al. Time course of lung changes on chest CT during recovery from 2019 novel coronavirus (COVID-19) pneumonia. Radiology 2020 Feb 13 [Epub ahead of print]

Bernheim A, Mei X, Huang M, et al. Chest CT findings in coronavirus disease-19 (COVID-19): relationship to duration of infection. Radiology 2020 Feb 20 [Epub ahead of print]. At the most severe stage, dense consolidations become more prevalent. At the recovery stage, consolidation patterns are gradually resolved, while GGOs are still present for a longer time.

Studies on the AI-based classification of COVID-19-related pulmonary manifestations on chest CT are more prevalent than the ones on radiographic images. One of the largest studies performed by Shi et al. Shi F, Xia L, Shan F, et al. Large-scale screening of COVID-19 from community acquired pneumonia using infection size-aware classification. arXiv website. arxiv.org/abs/2003.09860. Published Mar 22, 2020. Accessed May 7, 2020 used chest CT images of 2,685 patients, of which 1,658 patients tested positive for COVID-19, while 1,027 images represented patients with non-COVID-19-related pneumonia. A Size Aware Random Forest method (iSARF) was used to train the algorithm to not only classify the different pneumonia causes, but also segment the image to calculate the involved lung volume. With an accuracy of 87.9%, additionally, their results showed that small volumes have a lower sensitivity for detection. Another large study performed by Li et al. Li L, Qin L, Xu Z, et al. Artificial intelligence distinguishes COVID-19 from community acquired pneumonia on Chest CT. Radiology 2020 Mar 19 [Epub ahead of print] of 4,356 chest CT images (1,296 COVID-19, 1,735 community-acquired pneumonia, and 1,325 non-pneumonia) using a pre-trained deep convolutional network (ResNet50) showed an excellent accuracy rate of 96% for the classification of COVID-19-related pneumonia.

AI Prediction of Disease Severity and Progression

With increasing laboratory test availability for COVID-19 diagnosis, the focus of medical imaging is shifting to the assessment of disease severity and disease progression, which can be used for treatment planning optimization and treatment efficiency evaluation Pan F, Ye T, Sun P, et al. Time course of lung changes on chest CT during recovery from 2019 novel coronavirus (COVID-19) pneumonia. Radiology 2020 Feb 13 [Epub ahead of print]

Bernheim A, Mei X, Huang M, et al. Chest CT findings in coronavirus disease-19 (COVID-19): relationship to duration of infection. Radiology 2020 Feb 20 [Epub ahead of print]

Zhao W, Zhong Z, Xie X, Yu Q, Liu J. Relation between chest CT findings and clinical conditions of coronavirus disease (COVID-19) pneumonia: a multicenter study. AJR 2020 Feb 19 [Epub ahead of print]

Li K, Wu J, Wu F, et al. The clinical and chest CT features associated with severe and critical COVID-19 pneumonia. Invest Radiol 2020; 55:1

Li M, Lei P, Zeng B, et al. Coronavirus disease (COVID-19): spectrum of CT findings and temporal progression of the disease. Acad Radiol 2020; 27:603-608. Specific manifestations and affected lung volumes can be used as an indication of disease severity. Tang et al. Tang Z, Zhao W, Xie X, et al. (2020) Severity assessment of coronavirus disease 2019 (COVID-19) using quantitative features from chest CT images. arXiv website. arxiv.org/abs/2003.11988. Published Mar 26, 2020. Accessed May 7, 2020 proposed a random forest model to quantify disease severity using chest CT images of 176 patients with confirmed COVID-19. They reported an accuracy of 87.5% with 0.91 AUC. More interestingly, they showed that specific quantitative features, such as the volume of GGO and its ratio with respect to the whole lung volume, are good indicators of the severity of COVID-19.

A study by Huang et al. Huang L, Han R, Ai T, et al. Serial quantitative chest CT assessment of COVID-19: deep-learning approach. Radiol Cardiothorac Imaging 2020 Mar 30 [Epub ahead of print] used a deep learning algorithm to automatically quantify CT lung opacification percentage, evaluating longitudinal changes of these quantitative parameters in sequential examinations and taking into account the clinical parameters and disease severity. A total of 126 patients were included, representing mild (6), moderate (94), severe (20), and critical (6) cases. They showed that the opacification progression was mainly present between baseline and first follow up, but not in later stages, and they observed that the opacification percentage increased with worsening disease severity.

Emory AI Project: The PREDICTION Study

At Emory University, in collaboration with the Georgia Institute of Technology, we have started an AI project on COVID-19, entitled “Predictive Model of COVID-19 Outcome Using a Convolutional Neural Network Applied to Chest Imaging and Clinical Parameters: Early Detection and Prognostication for Optimal Resource Allocation (COVID-19 PREDICTION Study)” (Fig. 1).

We have two objectives:

1. Use supervised learning methods to build a predictive model that can distinguish COVID-19 pneumonia from other common lung pathologies using chest imaging and clinical parameters.
2. Monitor the disease progression over time detecting different evolution patterns, ideally finding imaging and clinical parameters that can predict the evolution to the most severe cases of COVID-19, which result in intensive care unit admission and the need for respiratory assistance.

With this project, we hope that an AI-powered solution for COVID-19 early detection and prognostication will have a major impact on patient outcome and optimization of the resource allocation, in particular in areas with limited medical resources and access to ventilators. 

Future Developments and Perspective

In the near future, more AI-based solutions will be developed and applied for the evaluation of COVID-19 using medical imaging. Whereas the first AI approaches were mostly focused on COVID-19 diagnosis, we now see more algorithms focusing on disease severity and progression quantification. The first step for the development and training of these AI algorithms is the creation of large, representative databases, followed by proper algorithm validation. At the moment, there are several worldwide initiatives for the creation of open-source databases for both radiographic and chest CT images Zhao J, Zhang Y, He X, Xie P. COVID-CT-dataset: a CT scan dataset about COVID-19.  arXiv website. arxiv.org/abs/2003.13865. Published Mar 30, 2020. Accessed May 7, 2020

Cohen JP, Morrison P, Dao L. COVID-19 Image Data Collection. arXiv website. arxiv.org/abs/2003.11597. Published Mar 25, 2020. Accessed May 7, 2020. Recently, the Radiology Society for North America announced a call to develop an open-data repository for international COVID-19 imaging research and education efforts. Creating open-source databases and sharing AI algorithms online offer powerful tools for clinical validation. In the long term, we expect that AI will also play a role in the follow-up of COVID-19, predicting which patients will have permanent damage and assessing the disease evolution.

The COVID-19 pandemic presents an exceptional challenge for the international health care community. The social impact has been dramatic and will be lasting. Although no country was fully prepared at the beginning of this pandemic, we can now use the lessons learned—together with the large volume of generated clinical data and developing AI techniques—to prepare more efficient global response strategies.