John Glenn School Corporation
Administrative Guidelines
 

2605 - EVALUATION OF PROGRAM

PURPOSE

Evaluation is essential to achieve a desired learning outcome. The staff needs to be able to monitor a given situation in such a way that the Corporation or a school has appropriate data about the current status on which to base decisions and actions. If the assessment is inadequate (for one of several reasons), it could lead to inappropriate actions and ultimately to lack of accomplishment of Corporation goals and staff responsibilities.

Sequence of Tasks in Program Evaluation

 

Task 1

Clarify the outcomes and elements of the program to be evaluated.

 
 

Task 2

Describe program outcomes and elements as they should be.

 
 

Task 3

Determine the procedures for gathering needed information about current status of program outcomes and elements.

 
 

Task 4

Gather the data.

 
 

Task 5

Verify the accuracy and sufficiency of the data.

 
 

Task 6

Assess the programs by:

 
 

1.

comparing the data (Task 4) with the standards (Task 2) to determine strengths and weaknesses;

 
 

2.

determining relative significance of both strengths and weaknesses.

 
 

Task 7

Evaluate the program by judging its value or worth based on the assessment (Task 6).

PROGRAM EVALUATION CHECKLIST

When conducting an evaluation of a Corporation program, use some or all of the following questions to 1.) determine what is to be evaluated (Task 1), 2.) determine standards (Task 2), and then 3.) obtain evidence about its effectiveness (Task 3):

 A.OUTCOMES

  1.How proficiently are the participating students learning to perform the tasks in each component of the program?

  2.How well are the participating students achieving each of the parts of the INTENDED LEARNING OUTCOME as demonstrated by their performance on the APPLICATION OF LEARNING TASKS?

  3.What specific secondary gains (additional benefits) are being realized for the participating students? for other students? for teachers? for the school/Corporation? etc.

  4.What specific side effects (negative consequences) are realized for the participating students? for other students? for teachers? for the school/Corporation? etc.

  5.How well does the program comply with local, State, and Federal regulations, guidelines, policies, constraints, etc.?

 B.ATTITUDES

  1.To what extent do those who are directly involved in the program think it is a good program and want to continue being involved in it?

  2.To what extent do those not directly involved in the program think it is a good program and should be continued?

 C.PROGRAM OPERATION

  1.How well do the needs assessment procedures work to efficiently identify the students who should be involved in the program?

  2.How well is the organization and the scheduling of the program working to provide the participating students the opportunity to achieve program goals and meet other needs without inhibiting other students from achieving their learning goals and objectives?

  3.How well is the staff able to fulfill the roles and responsibilities needed to develop and implement the program?

  4.How well are the program-planning procedures working to create a plan for learning that will achieve the Intended Learning Outcomes of the program?

  5.How effective are the instructional procedures being used working to implement the curriculum and to facilitate the students' achievement of the program goals?

  6.How adequate and appropriate are the facilities being used to implement the program effectively?

  7.How adequate, appropriate, and available are the instructional resources needed to implement the program effectively?

  8.How well is the information about the program and its participating students being communicated to those who need, want, and/or should have the information?

 

Criteria and standards for each selected program result and/or element need to be decided, based on a determination of the amount and quality of those characteristics that either are essential for the program to be effective or are needed to satisfy people in the particular situation.

ESTABLISHING CRITERIA AND STANDARDS

The following are suggested procedures for establishing a set of criteria and standards for evaluating any Corporation learning program.

 A.Using the Program Evaluation Checklist above, identify the outcomes and elements of the program which are of particular interest or concern.

 B.Arrange the outcomes and the elements in terms of relative importance and/or assign a weight to each.

 C.For each element, identify CRITERIA by which that outcome or element should be assessed and evaluated.
  (Criteria deal with qualities or characteristics, e.g. number of, adequacy of, etc. while standards indicate how much (many) is acceptable. For example, asking, "To what extent do those who are directly involved in the program think it is a good program and want to continue being involved in it?" would be a question related to the CRITERION of acceptability of the program. The STANDARD would be how many or what percentage of those in the program would want to continue in order for the acceptability or attitude toward the program to be "excellent", "good", or "poor".)

 D.For each CRITERION, decide the STANDARDS for determining what number or level or degree will be used for the MALQ (Minimum Acceptable Level of Quality), generally judged as "poor but acceptable", and what will be the DLQ (Desired Level of Quality), generally judged to be "excellent". Anything in-between can be rated by such terms as "fair", "good", "above-average," etc.

 E.For each CRITERION, develop a rating scale or some way of assigning a numerical value to varying degrees of that quality based on evidence or on opinion supported by evidence.

For examples of rating systems with criteria/standards, refer to Form 2252 F1 and Form 2520 F1.

EVALUATION GUIDELINES

Evaluation is a judgment about how "good" something is based on how well it meets particular standards.

If plans for evaluating a program are made when the program is planned, the task is made considerably easier than if the evaluation plan is not developed until after the program has been planned and/or implemented.

The following are some suggested guidelines regarding evaluation:

 A.Decide, well in advance of any reporting, what will be evaluated, who will be asked to evaluate, when the program will be evaluated, and on what basis.

 B.Be sure that everyone involved in the evaluation is clear about the criteria and standards and what will be considered evidence on which to base a rating on each criterion.

 C.Devise checklists, forms, etc. which make it easy for people to keep track of data related to each criterion.

 D.Do not formally assess and evaluate a program until there is evidence of results. In a learning program, nothing else - attitudes, teaching procedures, etc. - should be judged except in terms of how each helps to produce the learning results. Also, do not keep changing the program until there is evidence of results. Let the program produce results and then assess them and key program elements. After the assessment/evaluation is completed, changes can be made to:

  1.maintain and/or improve learning outcomes and other results;

  2.improve elements of the program;

  3.maintain/increase secondary gains;

  4.decrease or eliminate side-effects.