Compare these properties to those of paper-and-pencil achievement tests. Although a paper-and-pencil test can be administered to many pupils simultaneously, it can be used only once with a class, while a performance assessment instrument can be used many times. Think of giving a class the same paper-and-pencil achievement test three of four times. The test results would be corrupted the second time it was given, because pupils would know what was going to be asked and would find and memorize the correct answers. Performance assessments are not corrupted by repeated use; in fact, since performance instruments contain descriptions of the performance that is being assessed, a teacher should be pleased if pupils master each of the behaviors on the assessment instrument. That would mean they could give a good performance.
Moreover, by identifying specific aspects of a performance and observing those aspects being performed, teachers are able to diagnose pupil strengths and weaknesses in terms of their actual behaviors. Compare this level of specificity and helpfulness to paper-and-pencil tests, which rarely provide direct evidence about the processes a pupil went through to arrive at an answer. As a consequence, paper-and-pencil tests can identify general areas where pupils have difficulty, but can provide little specific guidance to them about how to improve their performance.
Thus, while performance assessments can be time-consuming and difficult to carry out, most teachers make some effort to supplement their paper-and-pencil assessment with direct observation and rating of pupils' performance. Although available evidence suggests that present teacher efforts at performance assessment may be limited in many respects, it is encouraging that teachers attempt to gather information about important school outcomes in ways other than by paper-and-pencil tests. The following section describes procedures that can be used to develop performance assessment procedures that are manageable in light of classroom realities and useful for making decisions about pupil learning.
DEVELOPING PERFORMANCE ASSESSMENTS
An excellent and instructive example of performance assessment is a diving competition. Diving is a skill that must be demonstrated bv a performance; submitting a written essay describing how one would perform various dives is hardly an appropriate way to assess diving skill. It is necessary to see the diver perform—and to perform not just one dive, but a series of dives. Each dive is a separate performance that is assessed by a group of judges.
Recall watching a diving contest on television, perhaps during the Olympic Games. Remember what each diver did. The diver would slowly walk to the end of the board, stand still for a few seconds to compose him or herself, and finally launch into the selected dive. The diver would be off the board—twisting, somersaulting, or jack-knifing through the air for perhaps two seconds, and then splash into the pool below. If you blinked during the dive, you missed a good part of the performance. The crowd clapped, the camera followed the diver getting out of the pool to be greeted by his or her coach, and, about ten seconds after the dive, a group of judges would provide their scores of the diver's performance.
The judges use a scale that has twenty-one possible numerical scores that can be awarded to a dive (0.0, 0.5, 1.0,...5.5, 6.0, 6.5,...9.0, 9.5, 10.0). They observed a very complicated performance made up of various gyrations and body movements that took about two seconds to complete. They did not have the benefit of slow motion or instant replay to review the performance. They could not confer or discuss the dive with one another. If their attention strayed for even a second, they missed a significant portion of the performance. Yet, despite these difficulties, when the scores for the dive were flashed on the scoreboard, the judges inevitably were in very close agreement. Rarely did all the judges give a dive the exact same score, but rarely was there more than a one-point difference between any two judges' scores. This is amazing agreement among observers for such a short, complicated performance.
With this example in mind, we can determine the essential features of performance assessment. Structured performance assessments, such as the judging of a diving competition, a teacher's assessment of a pupil giving a five-minute speech, or a teacher's assessment of a bookcase built by a pupil in shop class, have four essential characteristics (Stiggins, 1987). Regardless of the setting in which performance assessment occurs—diving arena, classroom, gymnasium, or laboratory—these four characteristics should be present. Performance assessments should
• Have a clear purpose which identifies the decision to be made from the performance assessment
• Identify observable aspects of the performance that will be observed and judged
• Provide an exercise or setting for eliciting the performance
• Have predetermined scoring or rating procedures
In a diving competition, the purpose of the performance assessment is to rank the divers' performance in order to identify the best divers. This purpose is analogous to that of a performance assessment carried out in school for the purpose of assigning grades to pupils. The judges are given guides to help them focus on particular aspects of the divers' performances when they score a dive. Like most activities that are assessed by observation, a dive can be broken down into a series of steps or little performances. When diving judges observe and score performance, they are looking for specific behaviors or performance criteria to use in judging the quality of a dive. One manual for swim competitions provides the following directions for diving judges (National Collegiate Athletic Association, 1990, pp. 77-78):
a. The judges shall sit where the diving referee directs them.
b. Each judge shall assess each dive independently and give scores for each
dive from 10 to 0 in accordance with the following:
Very Good 8.5 9.0 9.5 10.0
Good 6.5 7.0 7.5 8.0
Satisfactory 5.0 5.5 6.0
Deficient 2.5 3.0 3.5 4.0 4.5
Unsatisfactory 0.5 1.0 1.5 2.0
Completely Failed 0
d. Each judge shall judge the dive on the following principles:
1. The dive shall be considered without regarding the approach to the starting position.
2. The dive shall be evaluated on—
(a) The forward approach when used.
(b) The takeoff.
(c) The technique and grade of the diver during the flight through the air.
(d) The height of the dive.
(e) The entry into the water without regard to movements under water.
3. Excessive Rocking or Crow-Hop—When executing the standing takeoff, the diver must not rock excessively or lift his or her feet from the board or platform before takeoff. For violation of either condition, each judge may deduct from zero to 1 1/2 points from his or her award, according to the judge's individual opinion.
4. Diving Safety—If the diver touches the end of the board or platform, or dives to the side of the direct line of flight, each judge shall exercise personal opinion regarding the deduction to be made
5. Split Tuck—If the diver's knees are spread while in the Tuck posit
the judges shall deduct one or two points_________
8. Arm Position on Water Entry—Deduct from one to three points from the
award if the arms are not in the correct prescribed position on entry into
the water.
These performance criteria provide guidance for the judges regarding what aspects of the diving performance they are to observe and score. Without any idea of what one should be looking for in a performance, it is all but impossible to judge it in a meaningful and consistent manner. Stiggins (1987) emphasizes the importance of such performance criteria in carrying out performance assessments of all kinds: "If you do not have a clear sense of the key dimensions of sound performance—a vision of poor and outstanding performance—you can neither teach students to perform nor evaluate their performance" (p. 37).
The third requisite for performance assessment is to provide a setting or exercise that allows the performance of interest to be demonstrated. In a diving competition, one needs a suitable board or platform to dive from and a pool to land in. In a performance assessment to determine pupils' skill in setting up and focusing a microscope, the teacher would have to create a situation in which there were microscopes, slides to be viewed, and a plan that allowed the pupils to be observed while performing the behavior. Obviously, it is difficult to conduct a performance assessment of diving or a microscope exercise without a swimming pool or microscopes.
The final step in the assessment of a dive is when the judges assign scores to the dive. First and foremost, the scores are based on the performance criteria; these are what are important in the performance, what the judges look for, and consequently, what the scores should describe. But the purpose of the performance assessment also influences scoring. In a diving competition, each judge provides one overall score to describe each dive made by a diver. The single score summarizes performance across all the performance criteria that make up the dive. Under different circumstances, a dive might not be scored in terms of a single number.
Suppose, for example, that a dive were being performed not in a competition, but during practice in preparation for a competition. The diver's coach would be the observer and judge for the practice dives. The coach's concern during practice would be less with the overall quality of the dive than with the smaller performances which make up the dive. These are the factors that could cause the diver to lose points during a competition. The coach would "score" the dive not with a single overall judgment, but with a judgment about each smaller aspect of the dive. Thus, a dive would produce
PERFORMANCE ASSESSMENT 267
not one judgment, but a series of judgments corresponding to the performance criteria diving judges would consider during a competition. Those areas in which the diver was weak would be emphasized in practice. Classroom teachers who wished to diagnose specific aspects of pupil performance also would give a series of scores corresponding to specific aspects of the performance. With this general overview of the four essential characteristics of performance assessment, we can turn to a more complete description of each as it is applied in the classroom.
Define the Purpose of the Performance Assessment
Teachers collect assessment information for a purpose, and the purpose influences how the assessment will be carried out. We just saw, for example, that the purpose of an assessment can influence the way in which a teacher observes and scores performance. If the purpose is diagnosis, what is observed and scored is different from scores obtained for the purpose of giving a single, overall rating of pupil performance. Thus, it is important to know why a performance assessment is being conducted. If teachers are asked why they are doing some assessment activity, their usual response is, "I am doing this in order to find out about this pupil characteristic so that I can make a decision about the pupil." Intended decisions are at the root of assessment activities.
Formal performance assessment is useful in providing information for a wide range of classroom assessment purposes (Jacobs, 1986; Stiggins, 1987). Many first and second grade teachers indicate that performance assessment of their pupils' oral reading skills at the start of the school year was an important piece of information used to determine the pupils' reading group placements. Industrial arts teachers told us that performance assessment of pupils' ability to follow safety rules was a major factor in deciding whether they would be permitted to work with power tools and live electricity. In general, most teachers indicated that they used structured performance assessments for two main purposes: grading pupils and diagnosing pupils. When asked about the usefulness of performance assessment in their classrooms, teachers make comments like the following.
It's not reasonable to grade reading without including the pupil's oral reading skills. I always spend some time when it's grading time listening to and rating my pupils' oral reading quality.
My kids know that a large part of their grade depends on how well they follow safety procedures and take proper care of the tools they use. They know I'm always on the lookout for times when they don't do these thing and that it will count against them if I see them.
Each day in my beginning typing class I observe one student's typing form. I have a fairly short list of things the students should do to show good form that I carry around in my head. You know, things like proper seating posture, position of fingers on the keyboard, hitting the correct key with the correct finger, and so on. I don't use these observations to grade the student, but to identify problems in his or her typing for them to correct. It's like a diagnosis to improve their form.
I wouldn't want anyone to assess my teaching competence solely on the basis of scores I got on some paper-and-pencil test. I would want to be seen interacting with the kids, teaching them, and attending to their needs. Why should I confine my assessment of my kids only to paper-and-pencil methods?
Structured performance assessments are particularly suited to diagnosis because they provide detailed information about how a pupil performs each of the behaviors that make up an overall performance. This behavior-by-behavior assessment makes it easy to identify the strong and weak points of a pupil's performance. The fact that the performance criteria are stated in terms of observable pupil behaviors also makes remediation easy. The teacher can say to the pupil, "If you want to improve, do this and this and this." Each suggestion for improvement can be described in terms of a specific behavior—e.g., "Make greater eye contact with your audience," "Enunciate your words more clearly," "Pause at the end of sentences." Whatever the purpose of performance assessment, it should be specified at the beginning of the assessment process so that proper kinds of performance criteria and scoring procedures can be set up.
Identify Performance Criteria
Performance criteria define those specific behaviors a pupil should perform to carry out an activity properly. When most people first begin to think about assessing performance, they tend to think in terms of overall performances such as oral reading, giving a speech, safety in the laboratory, handwriting, organizing ideas, zippering a zipper, or getting along with peers. In reality, such overall behaviors cannot be assessed until they are broken down into the smaller behaviors they are made up of. These smaller behaviors which make up an overall performance are described by the written performance criteria and are the behaviors that are actually observed and judged by the teacher.
In order to define performance criteria, the teacher must decide it type of performance will be observed. Will the pupil be observed actually performing a behavior (e.g., oral reading, giving a speech, mixing a cake) or will the product of the pupil's behavior be observed (e.g., a finished letter typed in five minutes, laboratory equipment set up, a handwriting sample)? In the former case, criteria are needed to judge the pupil's performance, while in the latter criteria are needed to judge the product of performance. It is also possible to observe and assess both performance and product. A first grade teacher observes both performance and product when he or she (1) watches a pupil writing in order to see how the pupil holds the pencil, positions the paper, and presses down with the pencil and (2) judges the finished handwritten product to assess how well the pupil formed the letters. Notice that in this example, the teacher would observe different things depending on whether he or she were interested in the pupil's handwriting process or product. It is for this reason that teachers must know what they want to observe before performance criteria can be identified.
For classroom performance assessment to be manageable and meaningful, a balance must be established between specificity and practicality. The key to attaining this balance is to identify the essential performance criteria associated with an overall performance. A teacher should not list every behavior that comes to mind.
For performance assessment of oral reading, it is clear that just making an undifferentiated judgment without any performance criteria would be inappropriate. On the other hand, a seventy-item list of behaviors involved in oral reading would be too cumbersome. Notice, however, that if a teacher referred a pupil for special assessment outside the classroom because of a reading problem, a seventy-item list of specific oral reading behaviors might be a useful tool for the clinician. The one-on-one situation in the clinician's office is quite different from the one-on-twenty-five situation in the classroom.
The teacher must identify the essential performance criteria for an overall performance in order for performance assessment to be carried out meaningfully in the classroom. The process for accomplishing this is an ongoing one. Teachers think about the performance they wish to observe and reflect on its key aspects. They may observe a few pupils carrying out the performance in order to check or add to their initial list of key behaviors. Further revisions may come from additional reflection and observation. The process does not have to stop once an initial list of essential performance criteria has been identified.
One classroom teacher developed the following set of performance criteria to assess the making of an oral presentation. The teacher first divided the overall performance of oral reading into three areas: physical expression, vocal expression, and verbal expression (Carey, 1988). Then the teacher identified a few important performance criteria within each of these areas and ended up with the set of performance criteria shown in Table 7.2.
The teacher's performance criteria have the following desirable characteristics:
1. They are specified in terms of observable actions the pupil should perform while making an oral presentation.
2. They include important aspects of the overall performance but are sufficiently limited that the teacher can reasonably observe and judge each criterion.
3. They are diagnostic and can be used to point out particular strong and weak points in a pupil's oral presentation.
4. They are capable of being used in most situations which require the teacher to assess pupils' performance while making an oral report.
TABLE 7.2 PERFORMANCE CRITERIA FOR AN ORAL PRESENTATION
I. Physical Expression
A. Stands straight and faces audience
B. Changes facial expression with changes in tone of the presentation
C. Maintains eye contact with the audience
II. Vocal Expression
A. Speaks in a steady, clear voice
B. Varies tone to emphasize points
C. Speaks loudly enough to be heard by the audience
D. Paces words in an even flow
E. Enunciates each word
III. Verbal Expression
A. Chooses precise words that convey meaning
B. Avoids unnecessary repetition
C. States sentences with complete thoughts or ideas
D. Organizes information logically
Another example of a manageable performance assessment involves typing of a standard, one-page business letter in a ten-minute time period. To assess this overall performance, the teacher would have to develop performance criteria. The teacher would probably start by looking at some completed business letters to identify their key aspects. Think of some of the performance criteria that would be useful in assessing a pupil's completed letter. Here we are concerned with the product the pupil has produced, not with the process that went into its production. Thus, we would not be looking at the pupil's posture or finger placement on the keys. What specific features of the pupil's typed letter will we observe to judge how well the pupil can perform the task? You may have identified some performance criteria that are different from the ones listed below. That is quite acceptable; different teachers have different emphases in their instruction. There is no one set of performance criteria for a task that is correct in all situations. What behaviors the teacher has taught and stressed will determine the main performance criteria in any given situation. Here are some of the important characteristics a teacher might examine in judging a pupil's ability to type a standard business letter in a ten-minute period. • The pupil completed the letter in the ten minutes allotted. • The format (standard business form) of the letter is correct. Heading Greeting/salutation |
E. Summarizes main points at conclusion
Paragraph structure Closing
• Typing mistakes are corrected so they don't show.
• There are no uncorrected spelling or typing errors.
Finally, a list of performance criteria for kindergarten or early elementary school pupils' mathematics proficiency might include the following behaviors, which could be used to assess performance at the end of the school year.
1. Counts to 10 aloud without error.
2. Writes the numbers 1 to 10 without error.
3. Counts to 30 by 2s.
4. Puts objects into sets of one to five.
5. Points to an empty set.
6. Points to paper cutouts of circles, squares, triangles, and rectangles when asked.
7. Points to pennies, nickels, dimes, and quarters when asked.
8. Compares objects to show which is bigger or smaller.
9. Walks on a number line as directed.
10. Can combine two attributes in selecting objects (e.g., "Find the red circle; find the big triangle; show me the blue triangle").
Performance assessment is dependent on the specification of performance criteria that can be observed and judged. To reiterate Stiggins' (1987) admonition: "If you do not have a clear sense of the key dimensions of sound performance—a vision of poor and outstanding performance—you can neither teach students to perform nor evaluate their performance" (p. 37). In order to accomplish this essential part of performance assessment in a practical manner, the following guidelines are offered.
1. Identify the overall performance or task to be assessed and either perform it yourself or imagine yourself performing it. Think to yourself, "What would I have to do in order to complete this task? What steps would I have to follow? What are the important aspects of the performance?" You may also observe pupils performing the task and try to identify the important elements in its performance, but usually it is better to imagine or actually carry out the performance yourself.
2. List the important aspects of the performance. What specific behaviors are most important to the successful completion of the task? What behaviors have been emphasized in instruction? The specific behaviors that are listed will become the performance criteria used to guide observation and assessment. Try to keep the number of performance criteria small enough that they can all be observed during a pupil's performance. This is less important when one is assessing a product, but even then, a limited number of kev criteria is better than a large number that vary widely in importance. Remember, a teacher will have to observe and judge performance on each of the criteria selected. A good rule of thumb to follow in the initial specification of performance criteria is to limit the number to about ten to fifteen.
A very useful practice when identifying performance criteria is to have a group of teachers think through the important behaviors included in a task. Since all first grade teachers assess oral reading in their classrooms and since the criteria for successful oral reading will not differ much from one first grade classroom to another, a group effort at defining performance criteria will likely save time and produce a more complete and universal set of criteria than one produced by a single teacher.
3. Express the performance criteria in terms of observable pupil
behaviors or product characteristics. The performance criteria
should direct the teacher's attention to something the pupil is doing
or some characteristic of the product. Be specific when stating the
performance criteria. Don't write, "The child jumps." Write in-
stead, "The child jumps forward at least two feet from a standing
start." In the examples provided previously, note that each performance criterion is expressed in terms of an observable pupil behavior or product characteristic.
Don't use words that cloud the meaning of the performance criteria. The worst offenders in this regard are adverbs that end in "ly." Criteria such as "organizes adequate/y," "speaks correct/y," "writes neat/y," and "performs graceful/y" leave the interpretation of the adverb up to the observer. The observer's interpretation may vary from time to time and pupil to pupil, diminishing the structure and fairness of the assessment. In short, avoid adverbs and state the desired behavior in observable terms. Instead of "organizes adequately," one might substitute "has an identifiable beginning, middle, and end" or "presents ideas in a logical order." Instead of "speaks correctly," one might substitute "enunciates each word," "can be heard in all parts of the room," or "does not run sentences together." The choice of criteria will depend on the teacher and the instruction, but the way the criteria are stated should be in terms of observable behaviors and product characteristics. Review and revise criteria as necessary based on experience observing students and using the criteria.
4. Check for existing performance criteria before defining your
own. The performance criteria associated with giving an oral
speech, reading aloud, using a microscope, serving a tennis ball, operating a drill press, observing safety rules in science laboratories, and the like have been thought about and listed by many people. No one who reads this book will be the first to try to assess these and other common school performances. The moral here is that one need not reinvent the wheel every time a wheel is needed. Textbooks in educational measurement contain examples of performance criteria for many skills, and these should be borrowed and used when appropriate (Carey, 1988; Cartwright and Cartwright, 1984; Ebel and Frisbie, 1986; Gronlund, 1985; Guerin and Maier, 1983; Nitko, 1983; Priestley, 1982; Sax, 1980).
It is interesting to note that performance assessment instruments are beginning to be provided by textbook publishers as part of their instructional packages. At least one elementary school reading series contains performance assessment tools for listening, speaking, and writing performances. In addition, attitudinal performance assessments are provided. Figures 7.2 through 7.4 are examples of performance assessment instruments that accompany a textbook series.
While these instruments are fairly rudimentary and contain some ambiguous performance criteria, they do remind the teacher that performance assessment of writing, speaking, and listening skills is important. They also try to focus the teacher on specific behaviors that make up these performances. As interest in performance assessment grows in response to dissatisfaction with the heavy reliance on paper-and-pencil tests (c.f., Educational Leadership, 1989), it is very likely that more textbook publishers will begin to provide performance assessment instruments as part of their instructional packages. Already many states have begun efforts to supplement traditional paper-and-pencil tests with performance assessments in their statewide assessment programs (c.f., Educational Leadership, 1989). In such instances, the criteria used to judge the usefulness of textbook paper-and-pencil tests are also suitable for judging the adequacy of textbook or state-developed performance assessment instruments.
Provide a Setting to Elicit and Observe the Performance
Once the performance or product to be assessed is defined, the teacher must set up a situation which will elicit the performance. The teacher can either observe targeted behaviors as they naturally occur in the classroom or arrange a specific exercise or situation under more controlled conditions. There are two considerations in deciding whether to observe naturally occurring behaviors or to set up
Дата добавления: 2015-11-04; просмотров: 22 | Нарушение авторских прав
| <== предыдущая лекция | | | следующая лекция ==> |