John Glenn School Corporation
Administrative Guidelines
 

3242C - SIX QUESTIONS FOR DETERMINING THE EFFECTIVENESSOF STAFF DEVELOPMENT PROGRAMS

The six (6) questions addressed in this guideline deal with the most important issues concerning any staff development program. The first relates to the outcome of the program - do staff members produce better results on the job? If that "outcome" question cannot be answered adequately, the effectiveness of any staff development program is very difficult to determine and the investment of valuable time and resources of doubtful justification.

Form 3242C F1 has been designed to provide a brief "No", "Somewhat", or "Yes" answer to each question. If other than Yes answers are obtained, further analysis and planning will be necessary if the program is to have much impact on job performance.

The remainder of the guideline provides background information on each question to help determine the "No", "Somewhat", or "Yes" answer. This will facilitate the rating of a program on the Form. Before using the guideline, therefore, it is important to review the form first.

This guideline can be used for staff development programs designed for either professional staff or support staff. For programs to be used with instructional staff, the end-result should be described in terms of the type and quality of student learning that should be forthcoming because of what the instructors can now do better or differently.

SIX QUESTIONS

1.

HAS THE RESULT THE INDIVIDUAL (TEAM) IS TO PRODUCE, UPON COMPLETION OF THE PROGRAM, BEEN DESCRIBED PRECISELY, COMPLETELY, AND AT DESIRED STANDARDS?

   

2.

DOES THE PROGRAM PROVIDE FOR ALL OF THE RELEVANT KNOWLEDGE, ATTITUDES, PROCEDURES, AND TEAM-WORK SKILLS THAT THE INDIVIDUALS (TEAM) HAVE TO LEARN IN ORDER TO CONSISTENTLY PRODUCE THE DESIRED END-RESULT?

   

3.

IS THE PROGRAM ORGANIZED/SEQUENCED TO BEST ENSURE THAT EACH INDIVIDUAL (TEAM) NOT ONLY LEARNS THE SPECIFIED CONTENT CUMULATIVELY, BUT ALSO LEARNS HOW TO APPLY IT PROPERLY AFTER COMPLETION OF THE TRAINING?

   

4.

CAN THE INSTRUCTORS DEMONSTRATE ALL OF THE KNOWLEDGE, ATTITUDES, AND/OR SKILLS WHICH ARE NEEDED TO HELP EACH INDIVIDUAL (TEAM) ACHIEVE THE STATED OUTCOME?

   

5.

ARE THE TRAINING MATERIALS/ENVIRONMENT DESIGNED AND ORGANIZED IN WAYS THAT ARE MOST CONDUCIVE FOR INDIVIDUALS (TEAMS) TO ACHIEVE THE NEEDED LEARNING?

   

6.

DOES THE END-OF-PROGRAM EVALUATION PROVIDE VALID EVIDENCE THAT INDIVIDUALS (TEAMS) HAVE FUSED THE KNOWLEDGE, ATTITUDES, PROCEDURES, AND/OR TEAM-WORK SKILLS, AS WELL AS LEARNING-TRANSFER STRATEGIES, WELL ENOUGH TO PRODUCE THE DESIRED END-RESULT?

REFERENCE FOR ANSWERING THE SIX QUESTIONS

1.

HAS THE RESULT THE INDIVIDUAL (TEAM) IS TO PRODUCE, UPON COMPLETION OF THE PROGRAM, BEEN DESCRIBED PRECISELY, COMPLETELY, AND AT DESIRED STANDARDS?

   
 

Why the Question Needs To be Answered

   
 

Any staff development program, if designed properly, should be based on a defined statement of intended result. For programs designed for instructional staff, that would mean a description of the intended learning result for students. For programs designed for other types of staff, there should be a description of the result that should be produced by proper application of the knowledge, attitudes, and/or skills developed by the program.

   
 

When leaders, program designers, instructors, and participants do not have a clear mental picture of the desired outcome of the program, it is difficult to justify, intelligently, the need for the training, or to know if the program is adequate, or to judge its effectiveness.

   
 

THE "NO" ANSWER

   
 

____

The end-result has not been defined or has not been justified.

     
 

THE "SOMEWHAT" ANSWER

     
 

____

The end-result has been defined in terms of staff member performance of selected tasks. These tasks are not the end-result of the responsibility, but represent means to an end. Knowledge of subject matter and/or skill in specific techniques does not confirm that the individual (team) can and will integrate and apply all that is necessary to consistently achieve the desired result.

     
 

____

The end result has been defined but only for one set of conditions. Since conditions usually change frequently, such a definition is inadequate for judging consistent performance over time.

     
 

THE "YES" ANSWER

     
 

The outcome contains each of the following parts. Each has been stated completely and accurately.

     
 

____

The end-result is described as the situation should be after all necessary tasks have been completed.

     
 

____

The criteria (what is to be judged) and the standards (how much quality) for determining the quality of the result the individual (team) is to produce upon completion of the program.

     
 

____

The significance of the result in terms of its necessary contribution to achieving broader Corporation goals.

     

2.

DOES THE PROGRAM PROVIDE FOR ALL OF THE RELEVANT KNOWLEDGE, ATTITUDES, PROCEDURES, AND TEAM-WORK SKILLS THAT THE INDIVIDUALS (TEAM) HAVE TO LEARN IN ORDER TO CONSISTENTLY PRODUCE THE DESIRED END-RESULT?

     
 

WHY THE QUESTION NEEDS TO BE ANSWERED

     
 

This question deals with the substance of the staff development program, that is, with the requisite concepts, skills, and attitudes that have to be utilized properly to achieve the intended results. If the question is not answered, there is no way of determining the adequacy of the program. The answer signifies the validity, relevance, and completeness of the program content.

     
 

THE "NO" ANSWER

     
 

____

The concepts, skills, and/or attitudes are not defined or are not directly related to producing the desired end-result.

     
 

THE "SOMEWHAT" ANSWER

     
 

____

The training specifies concepts and procedures but not attitudes or team-work skills.

     
 

____

The concepts, skills, and/or attitudes are inadequate, that is, if the individuals (teams) learned them all, they would still lack certain needed knowledge or skills to accomplish the end-result of the program.

     
 

____

There are some concepts, skills, and/or attitudes that are relevant and some that are not directly related to the end-result of the particular program.

     
 

THE "YES" ANSWER

     
 

The requisite concepts, attitudes, procedures, and/or teamwork skills, are:

     
 

____

described in a logical order;

     
 

____

explicitly interrelated (if the relationship is not apparent);

     
 

____

based on previously-learned instructional concepts and skills which have been confirmed or, if assumed, adequate remediation procedures (if needed) are described.

     

3.

IS THE PROGRAM ORGANIZED/SEQUENCED TO BEST ENSURE THAT EACH INDIVIDUAL (TEAM) NOT ONLY LEARNS THE SPECIFIED CONTENT CUMULATIVELY, BUT ALSO LEARNS HOW TO APPLY IT PROPERLY AFTER COMPLETION OF THE TRAINING?

     
 

WHY THE QUESTION NEEDS TO BE ANSWERED

     
 

Descriptions of what is to be learned do not confirm whether or not the content (knowledge, techniques, etc.) is organized/sequenced properly from the standpoint of the participant! Improper sequencing can create serious learning problems or "disconnects". Also, if the learning is "atomized" rather than cumulative, the individuals (teams) will experience great difficulty in "putting it all together" when it comes time to make applications. And, since dilution of learning and inability to apply is such a common phenomenon, learning how to transfer learning needs to be an ongoing, integral part of any staff development program.

     
 

THE "NO" ANSWER

     
 

____

There is no apparent logic to the organization or the sequencing and no attempt has been made to deal with learning-transfer.

     
 

THE "SOMEWHAT" ANSWER

     
 

____

The content is organized by discrete tasks or topics, but the sequence of learning is arbitrary or not clearly indicated.

     
 

____

The content is organized/sequenced properly for isolated tasks or topics but does not build cumulatively toward performance of all aspects needed to produce the desired end-result.

     
 

____

The content is organized/sequenced properly and builds cumulatively, but the strategies for applying the learning are missing or inadequate.

     
 

THE "YES" ANSWER

     
 

____

The content is organized logically/cumulatively for the end-result. For each unit or section of the program, the participants follow a learning sequence in which they:

     
   

____

develop a clear mental picture of what they will learn, why, and what they will do to confirm they have achieved the learning;

     
   

____

obtain needed information and process it through discussion, practice, problem-solving, etc.;

     
   

____

demonstrate the desired learning;

     
   

____

plan applications of the learning to their job responsibilities.

     

4.

CAN THE INSTRUCTORS DEMONSTRATE ALL OF THE KNOWLEDGE, ATTITUDES, AND/OR SKILLS WHICH ARE NEEDED TO HELP EACH INDIVIDUAL (TEAM) ACHIEVE THE STATED OUTCOME OF THE PROGRAM?

     
 

WHY THE QUESTION NEEDS TO BE ANSWERED

     
 

Regardless of how well designed the staff development program may be, it is only as good as its implementation. The key factor for effective implementation is quality instructors.

     
 

THE "NO" ANSWER

     
 

____

The program is to be instructor-free. That is, instruction is provided entirely by directions in materials (or equipment) which also provide any follow-up instruction.

     
 

____

The instructors will present the program primarily by lecture.

     
 

THE "SOMEWHAT" ANSWER

     
 

____

The instructors are to manage "self-instruction". That is, they are to organize facilities and materials, distribute materials, monitor individual (team) progress, correct tests, offer help when needed and/or requested. The actual instruction is provided by the materials. Instructors receive no formal training in such "management" tasks.

     
 

____

Same as above except instructors receive instructional management training which could range from inadequate to that which confirms that instructors can perform the instructional management tasks in actual staff development situations.

     
 

____

The instructors are to conduct the activities described in the program. Instructional procedures are not described or are so vague that different instructors could legitimately interpret them differently and thereby produce entirely different results. Instructors receive no formal training in conducting the activities.

     
 

____

Same as just above except instructors are given formal training, but the procedures they are taught are irrelevant, and/or inadequate and/or there is no confirmation they can and will apply them in the staff development situation.

     
 

____

The instructors are not required and/or have not been trained in how to model the behaviors expected of those who participate in the program.

     
 

THE "YES" ANSWER

     
 

____

All of the instructional procedures are provided or the plans are available for conducting each of the training activities. These include procedures or plans for:

     
   

1.

planning each activity;

       
   

2.

instructing individuals, small groups, and large groups;

       
   

3.

remediating "learning blocks" or deficiencies in prerequisites;

       
   

4.

confirming that what participants have learned can be applied.

       
     
 

____

The procedures make it possible for instructors to model expected behaviors and to conduct the program in a way that fosters self-confidence and the other attitudes needed for participants to become competent in using what they have learned.

     
 

____

The instructors have confirmed, through actual performance, that they are able to apply the instructional procedures as and when needed in their staff development situations.

     

5.

ARE THE TRAINING MATERIALS/ENVIRONMENT DESIGNED AND ORGANIZED IN WAYS THAT ARE MOST CONDUCIVE FOR INDIVIDUALS (TEAMS) TO ACHIEVE THE NEEDED LEARNING?

     
 

WHY THE QUESTION NEEDS TO BE ANSWERED

     
 

Materials and/or environment (facilities, etc.) can be over-emphasized or, just as bad, they can be given too little consideration in terms of their impact on learning. They do not produce the needed learning, but they do seriously affect how well and how fast an individual (team) reaches desired levels of understanding and/or performance.

     
 

THE "NO" ANSWER

     
 

____

The training environment does not make it possible for individuals (teams) to learn properly, because essential training materials, facilities, etc. are missing or in poor condition.

     
 

THE "SOMEWHAT" ANSWER

     
 

____

Some of the training materials are not essential but are included because they are appealing or reflect "conventional-wisdomitis". The training environment is such that it creates distractions that get participants off the focus of the needed learning.

     
 

____

Training materials are designed/organized as "instructors" rather than as information resources needed for proper learning and application. The training environment is not designed specifically for the staff development program and/or to allow the needed learning to occur efficiently.

     
 

THE "YES" ANSWER

     
 

____

The training program does not use materials/technology as a substitute for a trained instructor. Materials/technology are used, when needed, to provide the information individuals (teams) have to have in order for the instructor to guide their learning toward performance that can produce the desired end-result.

     
 

____

The training environment, to the extent possible, is designed to keep distractions to a minimum and to replicate as closely as possible the kind of environment the individuals(teams) will be working in.

     

6.

DOES THE END-OF-PROGRAM EVALUATION PROVIDE VALID EVIDENCE THAT INDIVIDUALS (TEAMS) HAVE FUSED THE KNOWLEDGE, ATTITUDES, PROCEDURES, AND/OR TEAM-WORK SKILLS, AS WELL AS LEARNING-TRANSFER STRATEGIES, WELL ENOUGH TO PRODUCE THE DESIRED END-RESULT?

     
 

WHY THE QUESTION NEEDS TO BE ANSWERED

     
 

The time, money, and effort involved in staff development are too great to leave evaluation of the program to unjustified assumptions about how well an individual (team) is prepared to properly apply what has been learned. Since critical decisions are often made on the assumption that people can and will transfer their training to their job situation, obtaining evidence which confirms such "readiness" should be an integral part of any staff development program.

     
 

THE "NO" ANSWER

     
 

____

Confirmation is missing and readiness is to be assumed from a description of program inputs (what activities consist of, time allocated, money to be spent, etc.)

     
 

THE "SOMEWHAT" ANSWER

     
 

____

Confirmation is of training inputs. The evidence consists of measuring such things as training time, use of resources, number of participants involved, popularity of activities, etc.

     
 

____

Confirmation is by tests (paper/pencil or otherwise) which measure only selected ideas, procedures, etc., learned during the program.

     
 

____

Confirmation is by a one-time performance test given under one (1) set of conditions which may or may not be typical of participants' job situations.

     
 

THE "YES" ANSWER

     
 

____

Before completion of the program and consistently thereafter, individuals (teams) are required to produce the end-result and to justify how their performance meets established standards.

     
 

____

If it is not possible for individuals (teams) to produce the desired end-result in an actual or simulated situation, they are given several different sets of conditions and asked to describe the desired end-results, how they would produce that result and justify how their performance would meet the established criteria at given standards.

PRIORITIES FOR REMEDIATION

Using these guidelines, it should be possible to determine relatively quickly whether or not any given staff development program can or has fulfilled its intended purpose. If the answers to all six (6) questions are YES, then it is very likely that the program will achieve its purpose. It does not necessarily follow, however, that the program should not be used (or continued) if some of the critical elements are not what they should be. Some are more critical than others. The following priorities indicate the order of importance for eliminating deficiencies.

TOP REMEDIATION PRIORITY - "NO" OR "SOMEWHAT" answers to Questions #1 & #6

Reason for Priority Rating

There is no way to achieve the unknown. Any staff development program is fundamentally flawed that lacks a clearly-defined, justified end-result from the effort. That end-result should be the one that signifies proper application of that which the individuals (teams) have learned from the program. There also has to be an effective way to confirm whether or not each participant has achieved the desired state of "readiness".

SECOND REMEDIATION PRIORITY - "NO" or "SOMEWHAT" answers to Question #4

Reason for Priority Rating

A highly competent instructor, given a clear, justified end-result can, if necessary, overcome all the other deficiencies and still make it possible for participants to reach "readiness" and to achieve the end-result with students.

THIRD REMEDIATION PRIORITY - "NO" or "SOMEWHAT" answers to Question #2

If any of the prerequisites (the training content) are missing or inadequate, it places a burden on the instructor (no matter how competent) which could make the staff development process too difficult for the time that is available to achieve the end result.

FOURTH REMEDIATION PRIORITY - "NO" OR "SOMEWHAT" answers to Questions #3 and #5

Reason for Priority

A competent instructor with the aid of adequate content can work around these program deficiencies even though it makes the training difficult, irritating, and inefficient.

Reprinted by NEOLA, Inc. with permission of the Institute for Curriculum and Instruction, copyright, 1978 and 1994.