Abstract
Objectives: To assess psychometric properties of team-based learning student assessment instrument (TBL-SAI) and TBL acceptance by student academic year.
Methods: A cross-sectional study was conducted at College of Medicine, University of Bisha, Bisha, Saudi Arabia, between February and May 2018. Students (n=109) from second- to fourth-year were scored on TBL-SAI scale (33-items). Principal axis factoring with varimax rotation was conducted on the scale to identify the emerged factors. Cronbach’s alpha, item-total correlation were used to check the reliability and validity of the instrument.
Results: Cronbach’s alpha for the total scale was 0.798. The means scores of TBL-SAI scale and each subscale was favorable. On factoring analysis, 27-items of revised TBL-SAI scale were retained and 7 factors emerged. Factor 1 stands for TBL satisfaction; factors 2, 4, 5, 6 stand for TBL versus lecture preference, factors 3 and 7 stand for TBL accountability. The scale was positively correlated with each TBL favor subscale, ranged between 0.328 and 0.756. Year-4 students scored higher (p<0.001) on TBL-SAI compared to year-2 or year-3 students.
Conclusions: The revised TBL-SAI is valid and reliable tool to measure the favor of TBL. Longitudinal studies across all academic levels are recommended to provide a clearer picture of the students’ acceptance rate of TBL.
Team-based learning (TBL) has become an increasingly important educational method for the medical curriculum in the last few years.1 The increasing number of studies on TBL pedagogy with different opinions reinforces the need to develop a sound instrument to measure the acceptance of this learning approach.2 To date, there are limited validated instruments to assess students’ opinions about major areas of the TBL approach.2,3 The TBL-SAI originally developed for assessing TBL in baccalaureate nursing students,4 was found to be reliable among medical students5,6 and other health professions students.2,3 This instrument has been developed to measure accountability, learning preference, and students’ satisfaction with TBL.4 The psychometric properties of the TBL-SAI scale have been reported as satisfactory, and the construct validity of the scale has been examined in many countries.2-6 Such data are not available in Saudi Arabia, although a number of medical schools utilize TBL pedagogy as well as in the University of Bisha, College of Medicine (UBCOM), Bisha, Saudi Arabia. In addition, information regarding students’ acceptance of the TBL approach is limited, and many dimensions of the TBL have not been studied.7,8
This study designed to validate and examine the psychometric properties of TBL-SAI and the emerged dimensions based on a sample of medical students at the UBCOM. The study also aimed to assess TBL acceptance by student academic year.
Methods
A cross-sectional study was conducted among students at the UBCOM, Bisha, Saudi Arabia, between February and May 2018. The UBCOM admits approximately 40±5 medical students each year to a 6-year MBBS program. The medical curriculum at the UBCOM is arranged into modules that extend through pre-clinical and clinical phases. Team-based learning is one of main educational strategies that offered to deliver the contents of the curriculum.9
Medical students (n=121) from second- to fourth-year were included in the study. Students absent during the survey were excluded from the study. Participation was voluntary and approved by the ethical committee at the UBCOM.
The TBL-SAI is a self-administered instrument that contains 33 items (with total scores ranging from 33-165) designed to measure students’ attitudes towards TBL. Permission to use the instrument was granted by Heidi Mennenga. This instrument is composed of 3 subscales to measure TBL accountability (items 1 to 8, with scores ranging from 8 to 40), preference for lecture versus TBL (items 9 to 24, with scores ranging from 16 to 80) and student satisfaction (items 25 to 33, with scores ranging from 9 to 45). A positive perception of the use of TBL was represented by a score of >99 for the total scale, >24 for the accountability subscale, >48 for the preference for TBL versus lecture subscale and >27 for the student satisfaction subscale.4
The participants were asked to answer on a 5-point Likert scale that was scored from 1 to 5 (1=strongly disagree, 2=disagree, 3=neutral, 4=agree, or 5=strongly agree). Ten negatively worded items in the scale were reverse scored from 5 to 1.4
Statistical analysis
The Statistical Package for Social Sciences Version 22 (Armonk, NY: IBM Corp.) was used for data analysis. The internal consistency of TBL-SAI was estimated using Cronbach’s alpha.4 The sum and mean±SD of the instrument total and subscales was calculated. One-way analysis of variance (ANOVA) with Tukey’s post hoc test compared TBL preference by academic year. Factor analysis was conducted after the following steps: (i) Kaiser-Meyer-Olkin (KMO) criterion and Bartlett’s test of sphericity was performed for all instrument items to measure sampling adequacy. Values p>0.60 for the total items indicate that factor analysis could be conducted.2 (ii) Principal axis factoring was run to extract the number of components. (iii) Varimax rotation was conducted on retained components to identify the underlying factor structure. The extraction of factors was determined by considering eigen values >1.0. Items with factor-loading values >0.40 were considered valid within the instrument and were required to interpret the factor structure.2,4 Pearson’s correlation was performed on the final TBL-SAI model to determine the item-total correlation.
Results
Of the 121 students enrolled, the response rate was 90.1% (109). Of these participants, 100% (31/31) were from year-4, 87.8% (36/41) from year-3, and 87.5% (42/48) from year-2. All participants were 18 to 22 years old.
Reliability and validity of the instrument
Cronbach’s alpha was 0.798 for the total scale, 0.668 for accountability, 0.549 for TBL preference, and 0.709 for satisfaction subscales. The mean score for the total scale was 111.6 (SD=15.4). The mean subscale score was 28.4 (SD=5.5) for accountability, 51.7 (SD=7.3) for TBL preference and 31.5 (SD=6.3) for satisfaction.
Principal axis factoring of the total scale
The KMO measure of sampling adequacy yielded an index of 0.737, and Bartlett’s test of sphericity resulted in a Chi-squared value of (528) = 1701.425; p<0.001, indicating the appropriateness of the data for factoring analysis. The subsequent analysis showed that 6 items (1, 3, 5, 22, 23, 33) had factor loadings below 0.4 or loaded on 2 factors, and there were excluded. As a result, a 27-item instrument was retained and yielded acceptable scale content, with validity of 0.756. Based on the criteria of Kaiser’s eigenvalue being >1.0, 7 factors were retained that accounted for 53.8% of the total scale. The 7 subscales were named based on the relatedness of their items contents (Table 1).
Factor loading, eigenvalues, and percent of variance for revised TBL-SAI scale (27-items) emerging from principal axis factoring with varimax rotation.
Factor 1 labeled as TBL satisfaction; Factor 2 explained as preference of TBL than lecture; Factor 3 emerged as accountability to team’s learning; Factor 4 referred to as preference of lecture than TBL; Factor 5 labeled as TBL preference in information recall; Factor 6 explained as lecture preference in information recall; Factor 7 labeled as contribution to teamwork.
Convergent validity of the 7 extracted dimensions
Correlations between the total scores of the revised TBL-SAI scale (27-items) and each of the 7 dimensions were outlined in Table 2. Most of the correlation coefficients were significant. The correlation coefficient between the revised TBL-SAI scale and each generated subscale were positive and ranged between 0.328 and 0.756, with peak levels for: TBL satisfaction; preference of TBL than lecture, and accountability to team’s learning.
-Pearson’s correlation coefficients among revised TBL-SAI scale (n=27 items) and subscales obtained in the factor analysis.
Comparison of revised TBL-SAI scores between medical students
Table 3 summarizes the mean scores of the revised TBL-SAI scale and their related subscales by students’ academic level. The mean score for the total scale as well as the teamwork contribution subscale was significantly higher among year-4 students compared with year-2 students or year-3 students (p<0.001). Team-based learning satisfaction was significantly higher in year-2 students (p=0.030) or year-4 students (p<0.001) compared with year-3 students. Significant differences were observed between year-4 and year-3 students in preference of TBL than lecture (p=0.041) and between year 4 and year 2 students in lecture preference in information recall (p=0.015).
Comparison of total mean score of the revised TBL-SAI scale and the 7 emerged dimensions according to students’ academic year.
Discussion
The internal consistency of TBL-SAI and its subscales was supporting the reliability of this instrument to measure students’ acceptance of TBL in Saudi Arabian medical schools. Other authors have examined the appropriateness of the TBL-SAI scale and found it to be valid.2,6,10 Overall, the mean score of the TBL-SAI was favorable (111.6) compared with the reference values indicated by Mennenga.4 However, this was lower than 159.68 rated by medical students,5 117.9 rated by pharmacy students2 and 115.6 to 125.3 rated by physical therapy students.3 A possible reason for our findings could be attributed to the short duration of experience of TBL since the UBCOM was established in 2014. Thus, more experience with TBL could perhaps increase student acceptance of such educational methods.
In this work, the final factoring analysis indicated that the 27-items of TBL-SAI loaded on 7 underlying factors. The statistical analysis confirmed the good instruct validity of the emerged subscales. Previous works validate the original 33-items of TBL-SAI scale using principal axis factoring methods and result in 3 subscales of TBL accountability, preference of TBL versus lecture format and TBL satisfaction.2,3 However, the contents of the 7 subscales identified in this study was in consistent with the contents of the 3 original TBL-SAI subscales. For instance, factor one emerged from principal axis factoring was TBL satisfaction. This indicated that our students have positive feelings towards TBL activities. Similarly, the majority of items loaded on factor one have been identified on the original TBL-SAI subscale.4 Furthermore, 4 factors of the emerged dimensions, namely: factors 2, 4, 5, 6 were reflecting students’ preference for TBL versus lecture. This supported by the literature, where the loaded items of these identified dimensions found to stand for TBL versus lecture domain in the previous works.3,4 It has been widely accepted that aspects of effective TBL accountability should emphasis on improving students’ teamwork skills.6,11 However, 2 extracted dimensions of factors 3 and 7 were reflecting TBL accountability in our context. Elsewhere studies have confirmed the power of these items to measure students’ accountability.3,5
The present study found that there is a positive correlation between the revised TBL-SAI scores and each subscale of TBL favor. These findings were in consistent of research theory, where most of the students’ preferred TBL pedagogy than lecture format in performing higher-order tasks.12,13
In the revised TBL-SAI model, the mean score for the total scale as well as the Teamwork contribution subscale were significantly higher in year-4 students compared with year 2 or 3 students. These results fit with the fact that the long-term use of TBL in the education system improves the impact of this assay.6 Contrary to expectations, third-year students had the lowest mean scores on all the dimensions assessed TBL favors’. This finding does not reflect the theory that TBL becomes an enjoyable and valuable learning tool, and students respond positively after having experienced it.14 This phenomenon could be a result of motivational variance between year-2 and year-3 students. Evidence suggests that learning styles and personality types of students are significant factors influencing learning in a small-group TBL setting.15 Another possible explanation is that learning materials selected for TBL activities may be inappropriate to introduce via such pedagogy.
Study imitations
Firstly, although the study targeted all medical students at the UBCOM, the sample size is limited. Secondly, the study did not involve medical students in year 5 or year 6, since the UBCOM did not matriculate students in these years when the study was conducted. Thirdly, the study included only male students because the MBBS program for the females had not yet started. Fourthly, many of the TBL facilitators were novices in the implementation of TBL, which might have caused a negative attitude among students towards TBL.
In conclusion, this study provide evidence for the reliability of the revised TBL-SAI scale and support the validity of this instrument for medical students in Saudi Arabia. Overall, our experience with TBL was promising in terms of accountability, preference for learning mode, and students’ satisfaction. Medical students in year-4 have more admiration for the TBL method than second- and third-year students. Further longitudinal studies across all the academic levels are recommended to provide a clearer picture of the acceptance of TBL at the UBCOM.
Acknowledgments
I would like to acknowledge the contribution of the students at the University of Bisha, College of Medicine, Bisha, Saudi Arabia who participated in this study.
Footnotes
Disclosure. Authors have no conflict of interests, and the work was not supported or funded by any drug company.
- Received November 19, 2019.
- Accepted March 24, 2020.
- Copyright: © Saudi Medical Journal
This is an open-access article distributed under the terms of the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.