Author: Logan Young

The partnership between radiology and artificial intelligence (AI) has been developing for some time. The availability of increasingly powerful deep learning algorithms and computer hardware, along with accessible databases, is driving this partnership. Potential applications for organ-specific imaging analyses are increasing daily. Thus, it is not surprising to see AI and deep learning applied to cardiopulmonary imaging.

Pulmonary AI applications include the identity and characterizations of pulmonary nodules, characterization of lung malignancies, identification of possible pneumonia, and the detection and quantification of obstructive lung disease and emphysema.

It is highly likely that future radiologists will benefit from the automatic detection, measurement, characterization, comparison, and recording of pulmonary nodules on chest CT scans. Presumably with the help of deep learning, computer-assisted nodule detection programs will reach acceptable reliability levels. The addition of automatic texture analysis might enhance Fleischner Society guidelines for more effective radiology reports with specific clinical follow-up recommendations.

Although considerable effort has been placed on radiomic analysis of pulmonary malignancies for potential tissue genotype prediction, findings of image-based statistical correlation with specific tumor genes are unlikely to compete with biopsy-confirmed results. It will be interesting to see if a clinical role develops from the radiomic analysis of lung cancers.

Can AI methods be used for detecting pulmonary opacities likely to represent pneumonia? This was the basis for the Radiological Society of North America’s (RSNA) Pneumonia Detection Challenge, where more than 1400 teams from around the world participated. With 346 teams submitting results during the evaluation phase, the finalists interrogated a training set of 25,684 radiographs and a test set of 1000 radiographs, where 5659 of the training set images were reported to have pneumonia by a panel of nonthoracic radiologists. The goal was to place bounding boxes around appropriate pulmonary opacities as accurately as possible. Successful competitors created training models and selected methods for optimal performance with a sensitivity approaching 90%.

The advancements demonstrated at RSNA’s Pneumonia Detection Challenge revealed that we are on the path to automatic detection of suspicious pulmonary opacities. One potential clinical role for such an advancement will be in prescreening and prioritizing chest radiographs. This would allow for earlier communication of possible pneumonia to appropriate practitioners and patients with a report that includes annotated imaging.

It is remarkable that ordinary chest CT images may be analyzed for air trapping by locating and summing appropriate voxels with attenuations of less than –940 HU. Apparently, this could be performed automatically and now more efficiently with deep learning methods that might be able to locate and quantitate findings of emphysema directly to the CT report.

The best example of a clinically useful application of AI in cardiac imaging is the success of Tao and colleagues in developing a deep learning–based approach to the automatic ROI selection and analysis of left ventricular volumes and ejection fraction, as measured from routine cardiac MRI [5]. Though it is fairly easy to manually perform this task, typically 10 minutes of operator time is required to outline all of the appropriate ROIs. Tao’s deep learning–trained system reliably performs this task automatically in a fraction of a second. This robust cardiac MR quantitation program is now available for workstation application for use with any MR system.

CT-based fractional flow reserve (FFR) uses computational fluid dynamics to quantify coronary artery stenosis. Physics-based models can noninvasively estimate FFR from patient-specific CT attenuation values. CT FFR analysis is a complex iterative process with high computational demand. CT FFR processing is particularly slow when performed on many existing radiology workstations. Computation time using deep learning–prepared programs solve CT FFR computation flow analysis in 20% of the time required by standard computational systems. Thus, with a deep learning–trained computer system, CT FFR calculations can be available in a fraction of the time required by current software analysis using typical workstations. It is predictable that the combination of faster computer systems coupled with deep learning–trained software will allow for all patients undergoing coronary CT angiography to benefit from efficient FFR processing and automatic incorporation of results into the radiology report.

As the partnership between radiology and AI continues to advance daily, both the patient and the radiologist stand to benefit. The integration of AI and deep learning into practice will lead to more efficient, effective, and valuable quantitative cardiopulmonary radiology reporting.

---

The opinions expressed in InPractice magazine are those of the author(s); they do not necessarily reflect the viewpoint or position of the editors, reviewers, or publisher.

Artificial intelligence (AI) algorithms are under development to interpret datasets to perform tasks for which a computer receives rules or patterns for search. Machine learning is a part of AI in which computer algorithms and statistical models perform a specific task using patterns and training data such that performance improves as more and more actual data are presented. Deep learning is a subset of machine learning in which multilayered processing is performed, such as with convolutional neural networks, to directly interpret unstructured or unlabeled data without supervision.

Breast imaging is likely to be the most successful imaging subspecialty to adopt the principles and practice of augmented intelligence or AI. The infrastructure is already in place. The Mammography Quality Standards Act (MQSA), which outlines performance metrics, was mandated in 1994 and has been updated since then. Risk stratification for mammography based on pattern recognition is standardized with a structured approach using BI-RADS for breast imaging. Breast imaging is digital and in PACS, which makes for streamlined feature analysis and data extraction. Computer-aided detection has been relatively widely used since its approval by the U.S. Food and Drug Administration in 1998. There are training data sets and tumor registries available for training and validation of proposed AI algorithms.

Because the deluge of data in medical imaging is ever growing, even the most capable imaging specialists are reaching human limits for data extraction. We desperately need help with prolific requirements for data extraction, analysis, and pattern recognition. AI has the potential to become a physician extender by automating some of this work by finding previously hidden but important information. It can enhance and improve our performance, thereby enabling us to better serve our patients.

In addition to assistance with faster image interpretation, AI has the potential to speed up workflow in breast and medical imaging while improving cancer detection at screening. Use of computer-aided diagnosis should reduce the number of missed breast cancers, which has been estimated at 20% and higher from interval and screen-detected cancers in retrospect.

“Breast imaging is likely to be the most successful imaging subspecialty to adopt the principles and practice of augmented intelligence or AI.”

—Marcia C. Javitt

One interesting question relevant to breast imaging is whether or not machine learning can add value to breast cancer risk assessment. Machine learning applied to genetic profiles holds promise for generating more accurate risk profiles using information gleaned from single nucleotide polymorphisms. Imaging biomarkers may likewise be incorporated into risk profile. For example, increased breast density, a known risk factor for the development of breast cancer, can be categorized using texture analysis to improve risk stratification. There are already commercial products in use to perform automated mammographic breast density assessment.

When compared with human readers, automated breast density assessment has been found to have less interobserver variability but similar accuracy. Increased breast density is not only a known risk factor for the development of breast cancer but also can mask cancers that are hidden in dense tissue.

As pointed out by participant Toula Destounis in a recent AJR webinar titled “The Value Proposition for Artificial Intelligence in Women’s Imaging,” the American College of Radiology started its Data Science Institute to develop algorithms in AI that can assist with lesion detection, characterization, and treatment selection. The Institute will also examine the safety, efficacy, and clinical applicability of such algorithms.

Witness the fact that on March 27, 2019, the U.S. Food and Drug Administration issued a new amendment to the MQSA for breast cancer screening. The changes will require that patients receive a lay letter with information about their breast density and the appurtenant risks. In addition, breast imaging reports to referring providers will also communicate more information about patients’ risks of increased density and breast cancer when appropriate. Improved communication should improve the patient’s and her health care provider’s preparedness to make appropriate management decisions, such as supplemental screening with other modalities for women with increased breast density on screening mammography. In 2019, 37 states have laws requiring that patients receive information about mammographic breast density generally, with fewer requiring that patients receive their own personal breast density information.

Further to this discussion, some of the most exciting research being done is in radiomics of breast cancer. In this era of radiomics, clinical decision-making will be based on harmonizing clinical and imaging biomarkers to achieve personalized patient care. A multiparametric approach is evolving in which lesion detection, tissue characterization, risk stratification using molecular subtypes, treatment choices, prognosis, response to therapy, and prediction of recurrence risk are dependent on integration of clinical and imaging biomarkers. Clinical, histologic, and genomic information about a patient are married to feature extraction, segmentation, and intelligent data management to offer new insights. Precision medicine is becoming a realistic future prospect and will benefit from implementation of AI algorithms. The intelligent use of radiomics should also enable cost minimization and reduced overdiagnosis and overtreatment from screening.

As we move from screening based on the calendar to personalized care, can we achieve risk-based breast cancer screening based on modeling? The published literature is unclear so far, largely because the models are complex, varied, and without standardization. As pointed out by participant Linda Moy in the AJR webinar mentioned earlier, the Breast Cancer Surveillance Consortium case control study found that the addition of breast density to risk models helps to identify women with high risk.

However, another study of more than 15,000 patients suggested that more variables are needed to perform better risk assessment. Although AI-driven, risk-based screening will require further development before it can be implemented without harm, we welcome the opportunity to use such tools to increase accuracy, improve breast cancer detection, and relieve the exhaustion associated with cognitively demanding and ever-increasing workloads in the clinical practice of breast imaging today.

---

The opinions expressed in InPractice magazine are those of the author(s); they do not necessarily reflect the viewpoint or position of the editors, reviewers, or publisher.

All signs point to artificial intelligence (AI) as radiology’s next frontier, as it integrates automation with ever-improving accuracy. Be it the hidden truths in women’s imaging revealed by Marcia C. Javitt in this issue of InPractice or Patrick M. Colletti’s examination of deep learning cardiopulmonary developments, indeed, AI abides—begging questions of and fostering debates on how to ethically implement and assess AI, machine learning, predictive analytics, and other emerging algorithms to best serve both radiologists and patients.

In recent months, international imaging societies and the U.S. Food and Drug Administration (FDA) have unveiled documents endeavoring to establish guidelines for ethical concerns arising from the accelerating advances of AI in medical imaging. Having convened a working group comprised of practicing radiologists, computer scientists, data scientists, and related AI professionals, on February 21, the Royal Australian and New Zealand College of Radiologists (RANZCR) published a 52-page primer, Ethical Principles for AI in Medicine, intended to “complement existing medical ethical frameworks” for the training, deployment, and fair use of AI tools in radiology and radiation oncology. Accompanied by a call for public comment, for six days at least, RANZCR’s eight principles were the only ones of their kind proffered by a professional healthcare body:

• Safety
• Avoidance of Bias
• Transparency and Explainability
• Privacy and Protection of Data
• Decision Making on Diagnosis and Treatment
• Liability for Decisions Made
• Application of Human Values
• Governance

“Of all the places that we could jump in,” RANZCR President, Lance Lawler responded to InPractice, “why start with ethics? In radiology, where we heavily regulate the doctors and the imaging machines, lest they drift out of specification and patient harm happens, it is inconceivable that the AI tools wouldn’t be regulated in some way. In an attempt to have something on which to base these conversations, we started with ethics. The theory is that ethics help build practice standards, and standards form the basis for regulation”.

Less than a week later, on February 26, a consortium of seven major radiology organizations—American College of Radiology, European Society of Radiology, Radiological Society of North America, Society for Imaging Informatics in Medicine, European Society of Medical Imaging Informatics, Canadian Association of Radiologists, American Association of Physicists in Medicine—published Ethics of AI in Radiology. This 38-page document was the product of a diverse cohort: North American and European radiologists, imaging informaticists, medical physicists, patient advocates, attorneys, and a bona fide philosopher. Noting that its preliminary draft was “aspirational rather than prescriptive” and that it sought to “foster trust among all parties that radiology AI will do the right thing for patients and the community,” the multi-society assembly highlighted three principles accordingly:

• The ethics of data—including informed consent, data privacy, ownership and transactions of patient data—and technical and social issues related to bias
• The ethics of algorithms and considerations to verify their safety and moral use
• The ethics of practice, including practice-level policies to do the right things for patients, in order to minimize inequalities related to resources and potential gain

“Having kept the writing group fairly small to facilitate an accelerated turnaround, it’s now critical to get extensive comments from the broader imaging and healthcare ecosystem,” wrote Geraldine McGinty, chair of the ACR Board of Chancellors. “We also anticipate that, given the pace of change in this sphere, this document will be a living one.” Apropos of AI’s quickening evolution, the committee intends to release an updated version.

In both the RANZCR and the multi-society papers, patient data is paramount, each one stressing the consequence of protecting access to and preserving the security of information utilized for algorithmic research and training. Acknowledging that no system’s firewall is unassailable, each document nonetheless insists that every effort must be made to protect patient privacy. Specifically, the multi-society paper emphasized threats with data transfer because any unsecured transmission creates a “risk that bad actors with access to medical data could extort patients who have aspects of their medical history that they wish to remain private.”

The two papers also discuss the adoption of best practices for the avoidance of bias in the AI apparatus. To help minimize prejudicial potential, RANZCR asserted that “the characteristics of the training data set and the environment in which it was tested must be clearly stated when marketing an AI tool to provide transparency and facilitate implementation in appropriate clinical settings.” Moreover, declared RANZCR, “particular care must be taken when applying an AI tool trained on a general population to indigenous or minority groups.” The more varied the data it acquires, the more equitable an AI tool’s intelligence becomes, welcoming what the multi-society assembly deemed an “opportunity to invite diverse stakeholders to audit the models for bias.”

Right now, however, both the RANZCR and the multi-society papers affirmed that the most urgent ethical question facing AI in radiology is a determinate one: what role should artificial intelligence play in the decision-making processes of radiologists at large? Once more, RANZCR and the multi-society papers sustained restriction; all AI-guided means must be confined to advisory roles only. “Final decisions,” RANZCR maintained, “are recommended by the doctor with due consideration given to the patient’s presentation, history, and preferences.” In addition, the two papers advised radiologists to be wholly and progressively more transparent regarding the diagnostic and therapeutic functions they, themselves, perform. “As complex dynamic networked systems evolve, it may be difficult to attribute responsibility among different AI agents, let alone between machines and humans,” warned the multi-society paper.

Amplifying radiology’s response to the need for oversight of AI, on April 2, the FDA released a white paper, Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning-Based Software as a Medical Device, concerning a classification of adaptive AI systems wherein performance changes based upon uninterrupted exposure to new clinical data (i.e., “a continuous learning algorithm”). In an accompanying press release outlining his agency’s reaction to this real-world paradigm, outgoing FDA Commissioner Scott Gottlieb noted that the AI technologies cleared by the FDA thus far have been “locked,” that they do not recursively learn each subsequent iteration. Locked algorithms are updated at predetermined intervals by the manufacturers, who are equipped to sequence the procedural mechanism and verify that any revision manifests as intended.

In its new agenda, the FDA addressed an inflection point for healthcare professionals and patients alike—acclimatizing algorithms that learn without manufacturer intervention. Perhaps more so than AI as a tool, at this moment in contemporary radiology, the ethical protocols for machine learning, predictive analytics, and other algorithms are in a fledgling state. As the latest developments in the field illustrate, now is the time to consider the myriad ethical implications of artificial intelligence in radiology.

---

The opinions expressed in InPractice magazine are those of the author(s); they do not necessarily reflect the viewpoint or position of the editors, reviewers, or publisher.