To print this review, choose "Print" from your browser's File menu.
A CARET REVIEW OF:
A Report on the Effect of the unitedstreaming(TM) Application on Educational Performance
|
Principal author(s):
|
Franklin J. Boster, Gary S. Meyer, Anthony J. Roberto, & Carole C. Inge
|
|
Institution or agency sponsor:
|
|
|
Publication date:
|
2002
|
|
Primary geographic location:
|
State - Virginia
Country - USA
|
|
Targeted settings:
|
Classroom
School Site
District
Single State
|
|
Type of study:
|
Formal Research Studies (4b)
|
|
Targeted population(s):
|
Elementary
Middle School
K-12 Teachers
K-12 Administrators
|
|
Subject area emphasized:
|
Math
Social Studies
3rd and 8th grade
|
|
Funding source:
|
|
Primary technology
applications addressed:
|
Internet
|
|
Indicators and/or standards used:
|
State Curriculum Standard
|
|
Where study can be accessed:
|
Boster, F. J., Meyer, G. S., Roberto, A. J., & Inge, C. C. (2002). A report on the effect of the unitedstreaming(TM) application on educational performance. Cometrika, Inc., Baseline Research, LLC., & Longwood University.
|
Major topics related
to technology:
|
Student Learning
Curriculum and Instruction
Professional Development
|
Additional relevant
background information:
|
|
|
Review posted:
|
November 27, 2002
|
FEATURES
A. The Study Addresses Important Educational Issues
- The study addresses at least one of the critical questions from the CARET Questions and Answers section
B. Objectives, Interventions and Evaluation Questions
- The objectives for the study are clearly stated and related to documented needs
- The objectives address local, state and/or national curriculum and/or performance standards
- The study defines specific products, services, policies, and/or practices to be studied
C. Context for the Study
D. Evaluation and Research Design
- Clear definition of problem and/or evaluation or research questions is present
- The study builds upon related prior research and evaluation studies
- The study is designed to directly answer research/evaluation questions presented
- The study provides an experimental design to statistically test a hypothesis
- Comparison groups to control for extraneous variables is incorporated in the study
E. Data Collection Strategies and Instrumentation
- Sources of evidence clearly described
- Instruments directly related the objectives and evaluation questions
F. The Findings are Adaptable to Multiple Contexts
- The study documentation justifies dissemination and replication in other settings
- The study has a strong potential for replication in another setting
G. Reporting of Findings and Analysis
- Recommendations are based on documented findings from the study
- The study identifies research and evaluation questions for future studies
- The study relates the specific interventions to the documented outcomes
- Reports on this study have previously been disseminated and/or published
NARRATIVE SUMMARY
Summary
This Type 4 B study provided a research-based approach to the evaluation of the impact of a United Learning product, unitedstreaming(TM), on student performance. Unitedstreaming(TM) offers a library of video clips available over the Internet. The video segments are accompanied by support materials, and can be aligned with academic content standards to support instruction in the classroom. Third- and eighth-grade students in Virginia were shown selected video clips related to state learning standards in science and social studies. Participating teachers received training in use of unitedstreaming(TM) prior to beginning instruction. Control groups covered similar content, but without exposure to the video clips.
All students were pre- and post-tested on a project-developed measure of the Virginia Standards of Learning. Differences between experimental group pre- and post-tests scores were compared with control group pre- and post-test differences. Experimental groups for 3rd grade science, 3rd grade social studies, and 8th grade social studies showed a significantly greater increase in scores on the post-tests over pre-tests than did the control groups. No significant difference existed between the 8th grade science group and the control group, a result possibly due to a lag time between the teacher training and implementation for that experimental condition. Overall, the students exposed to the video clips exceeded the control group post-test scores by 12.6%, an effect size that was about .66 of a standard deviation unit.
Major implications for educators/decision makers
A. This study showed that the use of unitedstreaming(TM) math and social studies video clips have a high probability of student achievement when:
1. video clips are selected to align with the state-adopted curriculum standards in Virginia
2. used during the time span allocated for instruction related to the standards for which the video clips are aligned
3. exam items correspond to each of the curriculum standards being taught
4. instructors are trained on ways to use and integrate the video clips with curriculum and instruction.
B. Educators should keep the above points in mind when utilizing videos to support instruction in core curriculum areas.
C. The use of video to supplement instruction can be an effective supplement that has a high probability of improving learning.
Major implications for educational researchers/evaluators
Reports of studies such as this, on the impact of technology applications on student learning would be more informational for educators if they were to include descriptions of:
- how the interventions were implemented and pedagogy used with the intervention
- how test items were developed and procedures used to determine the validity and reliability of test items to measure change
- the orientation and training of staff in the control and experimental groups for implementation of the intervention.
Information that would more easily enable the findings to be applied in other settings should include, but not be limited to details regarding the:
- instructional practices supported in the district, school, and classroom
- demographics of student and community populations
- level of teacher experience using the technology
- level of teacher familiarity with the standards of learning
- technology infrastructure, (i.e. wiring, hardware in the classroom)
Major intervention(s) or variables studied
The interventions analyzed were the use of video clips in the classroom combined with the related training of teachers in the experimental groups to use the video clips.
Critical questions addressed and specific answers found or suggested
- How can technology influence student academic performance?
- How can technology develop higher order thinking and problem solving?
- How can technology improve student motivation, attitude, and interest in learning?
- How can technology be infused into curriculum and instruction effectively?
- What strategies result in effective instructional applications of technology?
- How does technology facilitate teaching to national, state, and local academic content standards?
Sources of evidence identified
Pre- and post- tests based on the Virginia Standards of Learning were administered to experimental (video-clip) groups and control groups. The researchers created the multiple-choice tests (15 items for 3rd graders, 24 items for 8th graders) to focus on content covered by specific state standards. Items were reviewed by outside consultants for age-appropriate language and alignment with content.
Replicable strategies, practices, and/or products
The use of standards-aligned video clips to support core instruction was shown to be a replicable instructional approach. The video-streaming application is a commercial service available at http://www.unitedstreaming.com/. Although the teacher training is not described in detail, presumably the professional development component provided by the vendor could be replicated for other districts and other content areas.
Strengths and weaknesses of the study
Strengths:
The study followed traditional experimental design methods with the use of matched control groups with standards-matched pre- and post-tests to measure student learning. Besides identifying the significance and size of treatment effects, the authors attempted to statistically account for several possible confounding factors (spillover effects between subject areas and differences between districts, schools, and classrooms).
Limitations:
It would be useful for readers to have access to information regarding:
- whether control group teachers were provided equivalent familiarity with the content covered by the test items prior to instruction on the chosen Standards of Learning (SOL). On page 17, it is stated, “only after the completion of instruction on the three SOL” did they receive details about the study and instructions.
- a description of student, school, and community demographics, classroom learning environment, instructional strategies and pedagogy used by teachers in both experimental and control groups.
- procedures used to ensure that test questions were aligned with content covered by teachers in the control and experimental groups.
- how frequently, and within the context of what activities, students accessed the video clips.
Suggested related studies or resources to consider
Return to top