Artificial intelligence in radiology
====================================

* Mawya A. Khafaji
* Mohammed A. Safhi
* Roia H. Albadawi
* Salma O. Al-Amoudi
* Salah S. Shehata
* Fadi Toonsi

## Are Saudi residents ready, prepared, and knowledgeable?

## Abstract

**Objectives:** To assess the knowledge and perception of artificial intelligence (AI) among radiology residents across Saudi Arabia and assess their interest in learning about AI.

**Methods:** An observational cross-sectional study carried out among radiology residents enrolled in the Saudi Board of Radiology, Saudi Arabia. An anonymized, self-administered questionnaire was distributed in April 2020 and responses were collected until July 2020.

**Results:** A total of 154 residents filled the questionnaire. The top 3 aspects of AI participants wanted to learn were: clinical use of AI applications, advantages and limitations of AI applications, and technical methods. Approximately 43.5% of participants did not expect AI to affect job positions, while 42% anticipated that job positions will decrease. Approximately 53% expected a reduction in reporting workload, while 28% expected an increase in workload.

**Conclusion:** Currently, the exposure of radiologists to the use of AI is inadequate. It is imperative that AI is introduced to radiology trainees and that radiologists stay updated with advances in AI to be more knowledgeable on how to benefit from it.

*   artificial intelligence
*   radiology
*   medical imaging

**R**esearch on the use of artificial intelligence (AI) has been gaining popularity in the field of medicine.1-6 Recently, various kinds of AI programs have been developed based on the concept of “big data”. This concept can be defined as “extremely large datasets characterized by the large volume, high velocity of generation, variety, and veracity or credibility of the data”.7 With access to big data, machine learning (ML), a branch of AI, analyzes a vast amount of information in a rapid, accurate, and efficient manner through the use of complex computing and statistical algorithms.8,9 Currently, the development and utilization of AI programs in the medical field are entering the stage of commercialization.10,11

Several medical specialties are benefiting from the emerging advances in AI. In medical physics and radiation oncology, AI aids in auto-segmentation, prognostic prediction, and decision support.3 In diagnostic radiology, there are multiple examples of the use of AI, including screening, staging and restaging of cancer, support for structured reporting, detection of incidental findings, and imaging protocol optimization.1

As the concept of AI emerged in the field of radiology, scientists and doctors initially predicted that AI would replace radiologists.11,12 Obermeyer et al,11 argued that the digitized images used by radiologists can be turned over to algorithms using AI. However, experts say that radiologists would not be replaced by AI, in 2019, Langlotz,13 found that the roles of radiologists may change and that “radiologists who use AI will replace radiologists who do not”. He argued that despite the initial claims on their oncoming redundancy, radiologists will master new technologies with each emerging advance in their field.13 They will be better equipped to diagnose uncommon or rare entities that fall under the “long tail” of disease distribution compared to AI. Langlotz,13 also compared these advancements to the autopilot option for human pilots and found that AI could assist the radiologist’s judgement with accurate detection and measurement, making him an important component of precision healthcare. Unfortunately, the initial response of radiologists to AI has been persistent with a lot of misconceptions.13 Several studies were carried out to gauge the opinions and perceptions of medical students, residents, and radiologists. Medical students were queried to understand if current developments in AI would deter them from choosing radiology as a specialty. In Canada and the United Kingdom, researchers found that one-sixth and almost one-half of students would not choose radiology as a specialty because of the predicted impact of AI.14,15 Interestingly, a study by Dos Santos et al,1 surveying medical students in 3 German universities, found that a third of the respondents were somewhat concerned on the recent developments and the majority did not think that AI would replace radiologists. Furthermore, over half of the surveyed participants said that AI should be a part of medical training.1

Several studies showed that radiology residents and radiologists do not think that AI will replace them and that these advancements would not change their choice of career. Most participants showed an active interest in learning and getting involved with AI and ML.1,16 In a national multi-center study carried out in Singapore, most respondents (84%) said that AI should be taught in residency.16 A study surveying members of the European Society of Radiology showed that opinions were nearly equally divided on whether AI will increase or decrease job opportunities and workload.17 The overall perception on these innovations leading to more time for completing sub-specialization or working with other specialties and patients was positive. Interestingly, all participants believed radiologists will play a role in developing and validating AI applications to medical imaging. Also, 64.3% believed radiologists should supervise all development stages of an AI system applied to radiology.17

This study aimed to measure the knowledge and perception of AI among radiology residents in Saudi Arabia and understand their level of interest in learning on AI.

## Methods

An observational cross-sectional study carried out on residents enrolled in the diagnostic radiology training program in Saudi Arabia. The program is developed, supervised, and approved by the Saudi Commission for Health Specialties and is the only approved diagnostic radiology program in Saudi Arabia.18 It is a 4-year program divided into 2 junior and 2 senior years. Most residents rotate in several hospitals which provide different levels of care. These include: community hospitals, tertiary hospitals, stroke, trauma, and oncology centers. In addition to core radiology rotations, the curriculum includes courses and workshops covering imaging basics such as imaging informatics and advanced visualization, ultrasound, and medical physics. However, these courses are not mandatory. There is no formal education related to AI.

An email with a link to an online survey was sent to all residents by the program administrative assistant. The minimum required sample size was calculated as 585 (total resident population across the 4 years). The sample size calculator tool was used (Raosoft, Inc., Seattle, WA, USA).19 A 5% margin of error and a 95% confidence level were set, which yielded a required sample of 233.

Participation was voluntary and electronic consent was collected before data collection. The participants were informed regarding the aim, objective, and process of the study, and there were no additional benefits for involvement other than participants’ reflection on their understanding of AI.

An anonymous self-administered questionnaire collection from letrutre reviews and was created on Google forms (Googleplex, Mountain View, CA, USA).15-17,20,21 A link to the survey was sent by email and WhatsApp messages (Facebook, Inc., Menlo Park, CA, USA) in April 2020 and the responses were collected in July 2020. Responses were tabulated on Google spreadsheets.

The questionnaire consists of 5 parts. The first part inquired demographic information and training levels. The second assessed exposure to AI. The third inquired applications of AI in radiology. The fourth assessed their knowledge of AI/ML, and the fifth part evaluated the effect of AI on radiology and medicine.

The study was approved by the Research Committee of the Unit of Biomedical Ethics of King Abdulaziz University-Faculty of Medicine, Jeddah, Saudi Arabia.

### Statistical analysis

Data was carried out using Microsoft Excel 2016 and Statistical Package for the Social Sciences, version 25 (IBM, Armonk, NY, USA). Statistical differences between the qualitative variables were carried out using the Chi-squared test and Fisher’s exact test. *P*-values of <0.05 and 95% confidence intervals were considered significant.

## Results

A total of 154 radiology residents answered the questionnaire, with an overall response rate of 26.3%. A total of 85 (55.2%) respondents were males and 69 (44.8%) were females. A total of 75 (48.7%) respondents were from the Central region, while the rest are from the remaining 4 regions. A total of 40 (25.9%) residents were in the first year of training, 34 (22.1%) were in the second year, 52 (33.8%) were in the third year, and 28 (18.2%) were in the fourth year.

A total of 64 (41.6%) residents reported being familiar with AI. Upon further inquiry of this subgroup, 55 respondents reported reading articles or searching websites related to radiology, while 10 had taken AI and ML courses. A total of 6 participants carried out experiments in computer science involving AI.

When asked regarding “which radiological subspecialties would be more influenced by AI in the next 5-10 years?” the top 3 responses were breast imaging (99; 64.3%), followed by molecular/nuclear imaging (56; 36.4%) then both neuroradiology and thoracic imaging (54; 35.1%). In regards to the most important fields of AI applications in the next 5-10 years, 91 (59.1%) selected mammography, followed by positron emission tomography/nuclear imaging (46.8%) then computed tomography (44.8%). When asked regarding “what AI would bring to the profession of radiology?”; 82 (53.2%) participants believed it would help in the detection of asymptomatic patients (screening), 74 (48.1%) believed it would help in detecting incidental findings, and 73 (47.4%) believed it would help in image post-processing.

On the expectation of AI’s impact on the number of job positions in the next 5-10 years; 67 (43.5%, *p*=0.742) participants anticipated that there will be no effect, 64 (41.6%, *p*=0.919) anticipated that job positions will be reduced, while 23 (14.9%, *p*=0.869) expected job positions to increase. In regards to AI’s impact on the total reporting workload in the next 5-10 years; 29 (18.8%, *p*=0.440) participants responded it will have no effect, 43 (27.9%, *p*=0.192) responded that the workload will increase, and 82 (53.2%, *p*=0.905) responded that the workload will decrease. Moreover, in the next 5-10 years, 79 (51.3%, *p*=0.810) participants expected that the use of AI-based applications will make the duties of radiologists more technical and clinical, and 36 (23.4%, *p*=0.065) participants believed that the rate of dedication to subspecialties will remain unchanged. A total of 79 (51.3%, *p*=0.489) participants believed that most patients would not accept a report from AI applications without the supervision and approval of a physician. When asked regarding the legal responsibility of AI-systems’ output, 105 (68.2%) participants believed that radiologists would take the legal responsibility, while 79 (51.3%) believed AI application developers should be responsible. A total of 35 (22.7%) believed it is the insurance companies responsibility and 9 (5.8%) participants believed the responsibility to be on other physicians. A total of 97 (63%) participants considered the role of radiologists in the development/validation of AI applications for medical imaging to be important and that their supervision is required to develop AI-based applications. Table 1 elaborates the responses in details.

View this table:
[Table 1](http://smj.org.sa/content/43/1/53/T1)

Table 1 
- Exposure assessment to AI in radiology corresponding to gender, training level and familiar with big data.

A total of 120 (77.9%) respondents were willing to learn and train on ML algorithm so it could perform some of the tasks they do as radiologists. When asked if they were involved in research projects on AI-based development, only 15 (6.5%) participants confirmed that they were involved. The aspects of AI that respondents considered important to learn were as follow: clinical applications of AI (76%), advantages and limitations of AI applications (74.7%), and technical methods such as ML or deep learning algorithms (40.9%). Top responses regarding the advantages of using AI were: speeding up processes in healthcare (79.2%; *p*=0.257) and helping reduce medical errors (47.4%; *p*=0.298). Regarding concerns on AI applications in medicine; 59 (38.3%; *p*=0.193) respondents were concerned that it cannot be used to provide opinions in unpredicted situations because of inadequate information. A total of 53 (34.4%; *p*=0.198) respondents were concerned that AI was not flexible enough to be applied to every patient. Detailed results are shown in Table 2.

View this table:
[Table 2](http://smj.org.sa/content/43/1/53/T2)

Table 2 
- Artificial intelligence applications in radiology corresponding to gender, training level and familiar with big data.

Approximately 63.4% of participants agreed that AI will augment the capabilities of radiologists, thereby making them more efficient. In addition, 111 (72.1%) participants agreed that these developments will make radiology more exciting. Interestingly, 117 (76%) participants wanted AI to be a part of residency training, while 49 (31.8%) participants were frightened by the development of AI. Furthermore, 126 (81.8%) participants agreed that AI would improve medicine in general and 135 (87.7%) participants wanted AI to be used in evaluating radiological images. In contrast, 81 (52.6%) responses did not believe that AI alone could be used to evaluate radiological images even if it achieved high diagnostic accuracy (Tables 3&4).

View this table:
[Table 3](http://smj.org.sa/content/43/1/53/T3)

Table 3 
- Evaluation of AI effect on radiology and medicine.

View this table:
[Table 4](http://smj.org.sa/content/43/1/53/T4)

Table 4 
- Perception of radiologist on AI in radiology.

## Discussion

The new field of AI received a great deal of interest from radiologists over the last few years and the medical imaging filed have been buzzing with talks and discussions regarding AI.22-24 Despite that, less than half of our participants were familiar with the concept of AI and big data. This is an important finding that might reflect scarcity of AI related dialogue in the local imaging community in Saudi Arabia, especially in residency training and day-to-day work. Reporting that journal articles, courses and research projects related to AI as the main source of information regarding AI confirms lack of formal training in the subject and can help direct efforts to the deficient areas.

Several participants in our study believed that AI would influence different radiological subspecialties, including breast imaging, molecular/nuclear imaging, thoracic imaging, and neuroradiology. These results are supported by a study conducted in European countries.17 Breast imaging, as a cancer screening tool, is performed frequently and to a large number of patients. Hence, it provides an enormous source of data, which is frequently needed to train AI algorithms. The large volume of cases, at the same time, is a burden that many radiologists would not mind having a helping hand in. Neuroradiology being among the top 3 responses was not surprising because of the high demand for accurate and early detection of critical clinical situations such as in acute stroke care or in tricky inconspicuous diagnosis such as cancer recurrence vs post-radiation treatment changes.25

Screening, incidental-finding detection and post-processing were the top 3 expected applications of AI. These findings indicate a need for specific aids in repetitive and routine work, and the need for a safety net to make sure nothing is missed. The large volume of normal exams (in the case of screening), the lack of targeted attentive search, the fear of missing an important abnormality (in the case of incidental findings), and the tedious repetitive work (in the case of post processing) appear to demonstrate the areas were participants needed most help to achieve a quick and accurate diagnosis.10,26

The advent of AI in radiology raises questions on how it might affect the jobs of radiologists. Interestingly, our results show a strife. Approximately 40% of our participants believed that there will be no effect, while another 40% percent anticipated that job positions will be reduced. The distance seen between AI developers lead by big corporate and the common practicing radiologist impose a fear of the unknown on the later group. Afterall, hearing regarding an impending disruptive technology without actually having a role in its development brings in ambiguity and misunderstanding. Added to that, is the limited applications of AI in everyday radiology workflow at present in most training centers. Such divided opinions identifies an important area for trainee education based on published evidence and opinions of field experts.27 A little more than half of the participants looked forward to a reduction in workload. Artificial intelligence techniques have been applied specifically to visual tasks such as automatic segmentation of regions of importance in imaging or analysis of images, which may reduce human errors, decrease cost, help in repetitive tedious tasks, and consequentially, reducing the workload.28-32 A similar percentage was reported in the literature in a study carried out among members of the European Society of Radiology.17

Approximately half of our participants believed that most patients will not accept a report from AI applications without the supervision of a physician and would require their approval. The literature supports this. Despite the reported accuracy of AI in diagnosis, it has been proven that many situations require a physician’s knowledge, examination skills, and experience for interpretation and discussion of the diagnosis with the patient.33

More than two-thirds of participants believed that the radiologists would take legal responsibility of the AI system. However, guidelines for the use of AI in healthcare are still under development.34,35 This is a highly important topic and opens up interesting ethical and legal discussions, as an example of how to protect patients’ privacy, where the institutions should make certain that their large datasets are properly utilized.27

The top 3 areas our participants recommended education on were: clinical applications of AI, advantages, limitations of AI applications, and technical methods used in AI. It is not surprising that our participants chose these areas since radiologists must consider the pitfalls, vulnerabilities, and possible errors that can arise when an AI product conducts a pattern recognition. Although AI algorithms are efficient, they are often fragile and can offer inappropriate answers when confronted with images outside their knowledge set. This can include images with technical issues such as motion or beam hardening, or images obtained with poor techniques.27

Approximately 64% of our participants agreed that AI would augment the capabilities of radiologists and make them more efficient and approximately 70% agreed that developments in AI make radiology more exciting. These positive views are consistent with previous studies.1,15,16 Artificial intelligence has the ability to detect early findings with higher accuracy, providing an early diagnosis that can lead to better patient’s outcome. Overall, image enhancement, quicker and more accurate reading with higher impact, easier reporting, and a decreased workload understandably makes the radiologists’ day more exciting.

Approximately 72% wanted to include AI as a part of their residency training. An early exposure to AI can make radiologists aware of its importance and its usefulness. Moreover, this early introduction can help in the development of new AI programs and open the way for more future applications in the field of Radiology. This is one important take-home point out of this survey.

### Study limitations

The sample size and response rate were lower than expected which could be a result of the questionnaire distribution during the peak of COVID-19 in Saudi Arabia. In addition, we have not included open ended questions that could allow our participants to specify their exact concerns and issues with AI.

In conclusion, radiology residents are inadequately exposed to existing resources on the science of AI and showed interest in learning more on it. Supplementing training curriculums with resources to promote knowledge and facilitate better implementation of AI applications in diagnostic radiology is an opportunity, if implemented, in promoting the development and use of AI in patient care. In addition, there are concerns regarding the impact of AI on radiology job opportunities while the nature of work is expected to change. Artificial intelligence is expected to reduce workload by speeding up processes of reporting, reducing medical errors, and attaining efficiency in repetitive and tedious tasks. These findings can help guide better decisions by radiology training programs, radiological societies, decision makers, AI developers, and researchers.

## Acknowledgment

*The authors gratefully acknowledge Editage ([www.editage.com](https://www.editage.com)) for English language editing.*

## Footnotes

*   **Disclosure.** Authors have no conflict of interests, and the work was not supported or funded by any drug company.

*   Received May 18, 2021.
*   Accepted November 23, 2021.


*   Copyright: © Saudi Medical Journal

This is an Open Access journal and articles published are distributed under the terms of the Creative Commons Attribution-NonCommercial License (CC BY-NC). Readers may copy, distribute, and display the work for non-commercial purposes with the proper citation of the original work.

## References

1.  1.Pinto Dos Santos D, Giese D, Brodehl S, Chon SH, Staab W, Kleinert R, et al. Medical students’ attitude towards artificial intelligence: a multicentre survey. Eur Radiol 2019; 29: 1640–1646.
    
    [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1007/s00330-018-5601-1&link_type=DOI) 
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

2.  2.Kagiyama N, Shrestha S, Farjo PD, Sengupta PP. Artificial intelligence: practical primer for clinical research in cardiovascular disease. J Am Heart Assoc 2019; 8: e012788.
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

3.  3.Kang J, Thompson RF, Aneja S, Lehman C, Trister A, Zou J, et al. National Cancer Institute workshop on artificial intelligence in radiation oncology: training the next generation. Pract Radiat Oncol 2021; 11: 74–83.
    
    

4.  4.Miller DD, Brown EW. Artificial intelligence in medical practice: the question to the answer? Am J Med 2018; 131: 129–133.
    
    [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1016/j.amjmed.2017.10.035&link_type=DOI) 
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

5.  5.Patel VL, Shortliffe EH, Stefanelli M, Szolovits P, Berthold MR, Bellazzi R, et al. The coming of age of artificial intelligence in medicine. Artif Intell Med 2009; 46: 5–17.
    
    [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1016/j.artmed.2008.07.017&link_type=DOI) 
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=18790621&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 
    
    [Web of Science](http://smj.org.sa/lookup/external-ref?access_num=000266132300002&link_type=ISI) 

6.  6.Stewart J, Sprivulis P, Dwivedi G. Artificial intelligence and machine learning in emergency medicine. Emerg Med Australas 2018; 30: 870–874.
    
    [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1111/1742-6723.13145&link_type=DOI) 
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

7.  7.Krittanawong C, Zhang H, Wang Z, Aydar M, Kitai T. Artificial intelligence in precision cardiovascular medicine. J Am Coll Cardiol 2017; 69: 2657–2664.
    
    [FREE Full Text](http://smj.org.sa/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6MzoiUERGIjtzOjExOiJqb3VybmFsQ29kZSI7czo0OiJhY2NqIjtzOjU6InJlc2lkIjtzOjEwOiI2OS8yMS8yNjU3IjtzOjQ6ImF0b20iO3M6MTc6Ii9zbWovNDMvMS81My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

8.  8.Alsharqi M, Woodward WJ, Mumith JA, Markham DC, Upton R, Leeson P. Artificial intelligence and echocardiography. Echo Res Pract 2018; 5: R115–R125.
    
    

9.  9.Shameer K, Johnson KW, Glicksberg BS, Dudley JT, Sengupta PP. Machine learning in cardiovascular medicine: are we there yet? Heart 2018; 104: 1156–1164.
    
    [Abstract/FREE Full Text](http://smj.org.sa/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiaGVhcnRqbmwiO3M6NToicmVzaWQiO3M6MTE6IjEwNC8xNC8xMTU2IjtzOjQ6ImF0b20iO3M6MTc6Ii9zbWovNDMvMS81My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

10. 10.Dreyer KJ, Geis JR. When machines think: radiology’s next frontier. Radiology 2017; 285: 713–718.
    
    [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1148/radiol.2017171183&link_type=DOI) 
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

11. 11.Nemati S, Holder A, Razmi F, Stanley MD, Clifford GD, Buchman TG. An interpretable machine learning model for accurate prediction of sepsis in the ICU. Crit Care Med 2018; 46: 547–553.
    
    [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1097/CCM.0000000000002936&link_type=DOI) 
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

12. 12.The Economist. Automation and anxiety: will smarter machines cause mass unemployment? [Updated 2016; 2020 Oct 25]. Available from: [https://www.economist.com/special-report/2016/06/23/automation-and-anxiety](https://www.economist.com/special-report/2016/06/23/automation-and-anxiety)
    
    

13. 13.Langlotz CP. Will artificial intelligence replace radiologists? Radiol Artif Intell 2019; 1: e190058.
    
    

14. 14.Sit C, Srinivasan R, Amlani A, Muthuswamy K, Azam A, Monzon L, et al. Attitudes and perceptions of UK medical students towards artificial intelligence and radiology: a multicentre survey. Insights Imaging 2020; 11: 14.
    
    [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1186/s13244-019-0830-7&link_type=DOI) 
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

15. 15.Gong B, Nugent JP, Guest W, Parker W, Chang PJ, Khosa F, et al. Influence of artificial intelligence on Canadian medical students’ preference for radiology specialty: a national survey study. Acad Radiol 2019; 26: 566–577.
    
    [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1016/j.acra.2018.10.007&link_type=DOI) 
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

16. 16.Collado-Mesa F, Alvarez E, Arheart K. The role of artificial intelligence in diagnostic radiology: a survey at a single radiology residency training program. J Am Coll Radiol 2018; 15: 1753–1757.
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

17. 17.Codari M, Melazzini L, Morozov SP, Van Kuijk CC, Sconfienza LM, Sardanelli F. Impact of artificial intelligence on radiology: a EuroAIM survey among members of the European Society of Radiology. Insights Imaging 2019; 10: 105.
    
    [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1186/s13244-019-0798-3&link_type=DOI) 
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=31673823&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

18. 18.Saudi Commission for Health Specialties. Saudi Board: medical imaging curriculum. [Updated 2015; 2020 Oct 25]. Available from: [https://www.scfhs.org.sa/MESPS/TrainingProgs/TrainingProgsStatement/Documents/Medical%20Imaging%20new.pdf](https://www.scfhs.org.sa/MESPS/TrainingProgs/TrainingProgsStatement/Documents/Medical%20Imaging%20new.pdf)
    
    

19. 19.Raosoft. Sample size calculator. [Updated 2004; 2020 Apr 10]. Available from: [http://www.raosoft.com/samplesize.html](https://www.raosoft.com/samplesize.html)
    
    

20. 20.Oh S, Kim JH, Choi SW, Lee HJ, Hong J, Kwon SH. Physician confidence in artificial intelligence: an online mobile survey. J Med Internet Res 2019; 21: e12422.
    
    [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.2196/12422&link_type=DOI) 
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

21. 21.Van Hoek J, Huber A, Leichtle A, Härmä K, Hilt D, von Tengg-Kobligk H, et al. A survey on the future of radiology among radiologists, medical students and surgeons: students and surgeons tend to be more skeptical about artificial intelligence and radiologists may fear that other disciplines take over. Eur J Radiol 2019; 121: 108742.
    
    [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1016/j.ejrad.2019.108742&link_type=DOI) 
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=31734640&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

22. 22.Carlos RC, Kahn CE, Halabi S. Data science: big data, machine learning, and artificial intelligence. J Am Coll Radiol 2018; 15: 497–498.
    
    

23. 23.Erickson BJ, Korfiatis P, Akkus Z, Kline T, Philbrick K. Toolkits and libraries for deep learning. J Digit Imaging 2017; 30: 400–405.
    
    [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1007/s10278-017-9965-6&link_type=DOI) 
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=28315069&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

24. 24.Kohli M, Prevedello LM, Filice RW, Geis JR. Implementing machine learning in radiology practice and research. AJR Am J Roentgenol 2017; 208: 754–760.
    
    [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.2214/AJR.16.17224&link_type=DOI) 
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

25. 25.Qiu W, Kuang H, Teleg E, Ospel JM, Sohn SI, Almekhlafi M, et al. Machine learning for detecting early infarction in acute stroke with non-contrast-enhanced CT. Radiology 2020; 294: 638–644.
    
    

26. 26.Yu KH, Beam AL, Kohane IS. Artificial intelligence in healthcare. Nat Biomed Eng 2018; 2: 719–731.
    
    

27. 27.Tang A, Tam R, Cadrin-Chênevert A, Guest W, Chong J, Barfett J, et al. Canadian Association of Radiologists white paper on artificial intelligence in radiology. Can Assoc Radiol J 2018; 69: 120–135.
    
    [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1016/j.carj.2018.02.002&link_type=DOI) 
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=29655580&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

28. 28.Guo J, Li B. The application of medical artificial intelligence technology in rural areas of developing countries. Health Equity 2018; 2: 174–181.
    
    

29. 29.Lakhani P, Sundaram B. Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology 2017; 284: 574–582.
    
    [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1148/radiol.2017162326&link_type=DOI) 
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=28436741&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

30. 30.American Medical Association. 3 ways medical AI can improve workflow for physicians. [Updated 2018; 2020 Oct 25]. Available from: [https://www.ama-assn.org/practice-management/digital/3-ways-medical-ai-can-improve-workflow-physicians](https://www.ama-assn.org/practice-management/digital/3-ways-medical-ai-can-improve-workflow-physicians)
    
    

31. 31.Wang Z, Majewicz Fey A. Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery. Int J Comput Assist Radiol Surg 2018; 13: 1959–1970.
    
    

32. 32.Weng SF, Vaz L, Qureshi N, Kai J. Prediction of premature all-cause mortality: a prospective general population cohort study comparing machine-learning and standard epidemiological approaches. PLoS One 2019; 14: e0214365.
    
    [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1371/journal.pone.0214365&link_type=DOI) 

33. 33.Krittanawong C. The rise of artificial intelligence and the uncertain future for physicians. Eur J Intern Med 2018; 48: e13–e14.
    
    [PubMed](http://smj.org.sa/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fsmj%2F43%2F1%2F53.atom) 

34. 34.Food and Drug Administration. Considerations for the practical impact of AI in healthcare. [Updated 2021; 2020 Oct 25]. Available from: [https://www.fda.gov/media/107792/download](https://www.fda.gov/media/107792/download)
    
    

35. 35.Rock Health. How should the FDA approach the regulation of AI and machine learning in healthcare? [Updated 2018; 2020 Oct 25] Available from: [https://rockhealth.com/how-should-the-fda-approach-the-regulation-of-ai-and-machine-learning-in-healthcare/](https://rockhealth.com/how-should-the-fda-approach-the-regulation-of-ai-and-machine-learning-in-healthcare/)