|
Glossary
This glossary is intended to help users of this web site as they read
CARET literature summaries and research reviews. It is not intended as a
comprehensive list of research terms and concepts. For more complete
references, see the Research
Terminology Links in the CARET Resources area.
Click on a letter below to jump to related glossary terms:
A B
C
D
E
F G H I J K L
M
N
O
P
Q
R
S
T U V W X Y Z
context
The instructional, organizational, and demographic setting in which a study takes place.
The extent to which a study includes information about context often determines how useful the findings will be to educational practice. Effective replication depends on knowing not only what technologies or techniques were used, but--among other details-- how the intervention was implemented, what alternatives were available, and what resources and constraints affected the outcomes.
educational technology
As used in CARET, educational technology refers to the full range of digital hardware and software used to support teaching and learning across the curriculum. That includes desktop, laptop, and handheld computers and applications; local networks and the Internet; and digital peripherals such as cameras, scanners, and adaptive devices. It generally does not include older analog media such as film and overhead projectors.
Note the difference from "technology education," which refers to specific training about technology itself, often as part of an industrial arts or vocational program.
effect size
A measure of how much one variable affects another.
Tests for significance can determine if an instructional practice makes a difference at all, but for purposes of comparing alternative approaches, it is important to know how much difference to expect. Effect size can be expressed in several ways, a common one being as a proportion of standard deviation. For example, if students take a test with a standard deviation of 100 and those who prepared using computer-assisted instruction (CAI) score an average of 30 points higher than those who studied using a conventional text, we would say the effect size of CAI was .3. Current research standards call for reporting effect sizes on any quantitative study. This allows readers to guage whether results have practical iimportance as well as statistical significance, and also allows other researchers to conduct a meta-analysis to compute an average effect size for similar studies. According to meta-analyses reviewed in CARET, average effect sizes of successful technology-based interventions range from around .2 to .6 standard deviation units.
evaluation
Research focused on assessing the merit and worth of particular programs or products.
Evaluation studies (Type 3 in CARET's taxonomy) may include both observational (Type 2) and experimental (Type 4) research. Evaluation is distinguished by its focus, which is often on how well particular interventions meet the needs of participating individuals and organizations, rather than on generalization to larger populations.
Of studies classified in the CARET Reading List, only about 7% are classified as Type 3. Unfortunately, many evauation studies are not published outside their programs, and consequently are not readily accessible.
experiment
The systematic manipulation of variables to test the effect on outcomes.
Ideally, in experimental studies, the individual subjects are chosen by random selection from the population and those subjects are randomly assigned to different treatment groups (e.g., classrooms with and without technology). This makes it more likely that results can be generalized to the population, and are not the result of any special characteristics of the particular participants. Experimental studies without random selection and assignment are sometimes referred to as "quasi-experimental."
In CARET, experimental and quasi-experimental studies are classified as Type 4 ("Formal Research"). About 12% of studies on the CARET Reading List fall in this category. Compare with Type 2, "Observational Studies."
mean
The sum of individual scores for a group divided by the number of cases or individuals. Much quantitative research in education involves comparing means on achievement, technology use, or other outcomes of interest.
The mean is a good predictor of performance when the distribution of scores is symmetrical on either side of the mean. The mean becomes harder to interpret when lots of individuals score above or below the mean (the distribution is skewed), or if there are large groups of individuals scoring at a particular level on either side of the mean. Careful quantitative research will mention any issues around meeting these assumptions. In reading less formal studies or reports, be cautious of mean comparisons when there is a wide range of scores, a large standard deviation, or a large percentage of individuals scoring above or below the mean. In some cases, it may be more useful to compare the most frequent score (the mode) or the score at which half the individuals scored higher and half lower (the median).
meta-analysis
A statistical technique for summarizing the results of multiple quantitative studies. A meta-analysis involves computing an effect size for the same variable in each of the studies and then calculating a mean effect for the variable.
An advantage of meta-analyses is that they have greater statistical power than individual studies to detect small but consistent effects. A disadvantage of meta-analyses is that they often do not specify the details of the instruction, technology use, or other context factors that will affect the observed outcomes.
In CARET, meta-analyses are currently classified under Type 2 with observational studies. This classification is under review, as by definition meta-analyses are made up exclusively of Type 4 (formal research) studies.
observational studies
Studies in which researchers collect data from existing situations or contexts, without manipulating variables.
Although these studies often employ surveys or qualitative rubrics, observational research can involve data that is quantitative or qualitative, empirical or subjective. CARET's Type 2 classification for observational or descriptive studies includes research that involves surveys, test scores, interviews, classroom observations, and portfolio assessment. The distinguishing characteristic is the investigation of conditions as the researcher finds them, as opposed to the creation of experiments.
About 27% of the articles on the CARET Reading List fall under Type 2, including meta-analyses of Type 4 experimental/quasi-experimental research.
power
A statistical term that refers to the ability of a comparison to identify real differences between groups. Power increases with sample size, one of the justifications for conducting large-scale studies and for pooling studies in meta-analyses.
random selection
The choosing of experimental subjects out of a population on the basis of chance.
Random selection is important because it makes it more likely that the subjects represent the population, that results of the experiment will generalize, and that any observed differences are the result of the experimental conditions and not some special characteristics of the particular individuals involved.
Random selection of student samples and random assignment of selected students to different treatments is often difficult in school settings because of logistical and equity concerns. Researchers often have to work around these limitations by using "quasi-experimental" designs in which pre-existing differences between groups are identified and factored out statistically.
significance
In technical research literature, the probability that an observed quantifiable difference between two groups could have occurred by chance. An effect is said to be significant if that probability is low—traditionally, less than .01 or .05.
The size of the difference necessary to achieve significance depends in large part on the number of cases or subjects (“N”) in the study: in general, the larger the N, the greater the power of a study to detect significant differences.
In reading education articles, it is important to distinguish statistical significance from educational significance. The latter term refers to the importance or relevance of a finding to educational decisions, and is the focus of the “Implications for Educators” section of CARET reviews.
standard deviation
Roughly, the average amount that individual scores vary from the mean.
The standard deviation is actually the square root of the variance (the sum of squared differences of all the individual scores from the mean divided by the degrees of freedom, N-1). Although sums of squares are the figures used in many advanced statistical formulas, the standard deviation, as defined above, provides a more intuitive indication of the extent to which scores vary. The standard deviation is also used in comparing results from different studies that use different measures. Any individual score or overall study effect size can be described in terms of how far it is from its mean in terms of proportion of standard deviation.
*
top
|
