|
Meta-analytic studies of findings on computer-based instruction
By Kulik, James A.
University of Michigan
(1994)
Summary and Review | Profile and Features | Complete Study | Related Studies
 Print Version of Review
APA Reference:
Kulik, J. (1994). Meta-analytic studies of findings on computer-based instruction. In Baker, E. L. and O'Neil, H. F. Jr. (Eds.), Technology assessment in education and training. (pp. 9-33) Hillsdale, NJ: Lawrence Erlbaum.
Summary:
Kulik summarizes a large number of meta-analyses of computer-based instruction from the 1980's. He finds an average effect size of .32 standard deviation units for all computer uses. Effects vary with the type of research design, source of the study (dissertation or professional evaluation), duration of the study, type of computer use (tutorial, enhancement, management, simulations, programming), and the educational level of the intervention. Kulik notes that the average effect size of computer-based instruction compares favorably with a number of other innovations that have been implemented in schools.
Major implications for educators/decision makers:
The use of computer-based instruction typically has the following positive impacts:
*Students learn more in courses which involve computer-based instruction
*Students learn faster in courses which involve computer-based instruction
*Students have more positive attitudes towards instruction in courses which involve computer-based instruction
*Students have more positive attitudes towards computer as a result of courses which involve computer-based instruction
*Students attitude towards subject-matter is unaffected by the use of computer-based instruction
Kulik also points out that the benefits of computer-based instruction are typically much more pronounced when the computers are used in a tutorial fashion, as opposed to other types of usage (e.g., class management, simulations, enrichment, programming).
Finally, Kulik argues that the tutorial use of computers is an effective means to improve student achievement, as compared to grouping, learning packages, master learning, and programmed instruction. The tutorial use of computers is approximately as effective as peer and cross-age tutoring, but is not as effective as accelerated classes.
|
Major implications for educational researchers/evaluators:
The variables identified by Kulik, such as type of computer use and length of intervention, continue to be important in the assessment of educational technology. The same is true of the challenges he identified, in particular the appropriateness of outcome measures, controlling for teacher effect, and adequately specifying the alternative conditions to computer-enhanced instruction.
Major intervention(s) or variables studied:
This is a meta-analytic study that examines over 90 studies (in addition to a brief review of previously conducted meta-analyses), in an attempt to determine the effectiveness of computer-based instruction.
Major questions addressed:
Sources of evidence identified:
Meta-analysis of 97 studies that examine effectiveness of computer-based instruction.
Replicable strategies, practices, and/or products:
Kulik's meta-analysis of 1980's-vintage computer use provides a benchmark for later studies of educational technology. See the list of related studies below for more recent meta-analyses
Strengths and limitations of the study:
The strengths of this study are the standards by which articles are included in the study, the detailed analysis of the results, and the high degree of practical importance of the obtained results.
The weaknesses of this study are similar to those of other meta-analytic studies. Issues such as different groups of subjects, different experimental and analytic methodologies, different experimental contexts, and possible study selection biases are all sources of concern. As is the case with virtually all studies that are quasi-experimental in design, the results are potentially open to alternative interpretations that are not consistent with the researchers’ interpretation.
Probably the best way to interpret these types of studies is to treat them as exploratory in nature. They produce results that are interesting and useful, but their results need some sort of firmer empirical support to confirm their accuracy. That is, the results of this study need to be confirmed via studies that employ more rigorous experimental designs. Statistical manipulations such as meta-analysis are useful techniques to explain observed patterns in the data, and to establish predictive relationships. However, when it comes to establishing causative relationships, traditional experimental methodology is definitely to be preferred.
Suggested related studies or resources to consider:
* Blok, H., Oostdam, R., Otter, M., & Overmaat, M. (2002). Computer-assisted instruction in support of beginning reading instruction: A review. Review of Educational Research, 72(1), 101-130.
[go to CARET review]
Cavanaugh, C. S. (1999). The effectiveness of interactive distance education technologies in K-12 learning: A meta-analysis. ERIC: ED430547. Retrieved January 20, 2003 from http://www.ncrel.org/policy/pubs/html/pivol11/apr2002h.htm.
Christmann, E. P., Lucking, R. A., & Badgett, John L. (1997). The effectiveness of computer-assisted instruction on the academic achievement of secondary students: A meta-analytical comparison between urban, suburban, and rural educational settings, Computers in the Schools, 13(3-4), 31-41. ERIC: EJ552457.
Lee, J. (1999). Effectiveness of computer-based instructional simulation: A meta analysis. International Journal of Instructional Media, 26(1), 71–85. ERIC: EJ584288.
Liao, Y. (1999). Effects of hypermedia on students' achievement: A meta-analysis. Journal of Educational Multimedia and Hypermedia, 8(3), 255–277. Retrieved February 3, 2003, from http://www.medvet.umontreal.ca/techno/eta6785/articles/Effect_hypermedia.PDF.
Machtmes, K., Asher, J. W. (2000). A meta-analysis of the effectiveness of telecourses in distance education. American Journal of Distance Education, 14(1), 27–46. ERIC: EJ613342.
* Penuel, W. R., Kim, D. T., Michalchik, V., Lewis, S., Means, B., Murphy, R., Korbak, C., Whaley, A., & Allen, J. E. (2002). Using technology to enhance connections between home and school: A research synthesis. Planning and Evaluation Service, U. S. Department of Education, DHHS Contract #282-00-008-Task 1. Retrieved March 4, 2003 from http://www.sri.com/policy/ctl/html/synthesis1.html
[go to CARET review]
Soe, K., Koki, S., & Chang, J. M. (2000, June). Effect of computer-assisted instruction (CAI) on reading achievement: A meta-analysis. Honolulu, HI: PREL. Retrieved March 5, 2003 from http://www.prel.org/products/Products/effect-cai.htm.
Xin, Y. P., & Jitendra, A. K. (1999). The effects of instruction in solving mathematical word problems for students with learning problems: A meta-analysis. Journal of Special Education, 32(4), 207–225. ERIC: EJ581728.
Comments:
Numerous meta-analyses examining the effect of educational technology upon education have been conducted over the past several years. The reference section of this paper lists a number of well-designed and conducted studies that are relevant to Kulik’s meta-analysis.
* = Reviewed in CARET
|
