A control group can solve some of these test/retest problems, because if the problems are methodological, they should show up in the assessment of both groups. However, one must realize that the pretest can itself influence the effectiveness of both the experimental and control group programs and thus influence the two groups in different ways. Fourth, several logistical problems often are encountered when a classic experimental design is employed. In all but the largest organizations there may not be a sufficient number of people for a control group. There also may not be enough time or money to conduct two assessments with both an experimental and control group.
Given these difficult problems with a classic experimental design, many entrepreneurs may have to adopt alternative designs that are less elegant but more practical. In some cases, entrepreneurs have restricted their assessment to outcome measures. They determine the level of performance achieved by a group of outpatients in a mental health clinic and use this information to determine the relative success of the program being evaluated. This type of information is subject to many misinterpretations and abuses, though it is the most common evaluation design being used in contemporary organizations.
The information is flawed even when a comparison is drawn with program units in other clinics. One doesn’t know if differences in performance of students or recovery rates for mental health patience can be attributed to the program being reviewed or to the entering characteristics of the students or patients. Did the students at the alpha charter school do better than students at the beta charter school Was it because alpha students were already better educated or working at a higher level than beta students before they even entered the classroom?
This confounding effect is prevalent in many of the current initiatives that call for students to perform at a certain level on standardized tests without any consideration being given to their level of performance upon entering the school. In order to be fair in the assessment of a school’s effectiveness, one must at the very least perform a “value-added” assessment (Astin, 1990; Bergquist, 1995). This type of assessment requires that a student’s performance be measured when they first enter a school and again when they graduate from the school to determine the “value” that has been added, or more specifically the improvement in performance that has been recorded, with regard to their test scores
