Creating and Scoring Essay Tests
FatCamera / Getty Images
- Tips & Strategies
- An Introduction to Teaching
- Policies & Discipline
- Community Involvement
- School Administration
- Technology in the Classroom
- Teaching Adult Learners
- Issues In Education
- Teaching Resources
- Becoming A Teacher
- Assessments & Tests
- Elementary Education
- Secondary Education
- Special Education
- Homeschooling
- M.Ed., Curriculum and Instruction, University of Florida
- B.A., History, University of Florida
Essay tests are useful for teachers when they want students to select, organize, analyze, synthesize, and/or evaluate information. In other words, they rely on the upper levels of Bloom's Taxonomy . There are two types of essay questions: restricted and extended response.
- Restricted Response - These essay questions limit what the student will discuss in the essay based on the wording of the question. For example, "State the main differences between John Adams' and Thomas Jefferson's beliefs about federalism," is a restricted response. What the student is to write about has been expressed to them within the question.
- Extended Response - These allow students to select what they wish to include in order to answer the question. For example, "In Of Mice and Men , was George's killing of Lennie justified? Explain your answer." The student is given the overall topic, but they are free to use their own judgment and integrate outside information to help support their opinion.
Student Skills Required for Essay Tests
Before expecting students to perform well on either type of essay question, we must make sure that they have the required skills to excel. Following are four skills that students should have learned and practiced before taking essay exams:
- The ability to select appropriate material from the information learned in order to best answer the question.
- The ability to organize that material in an effective manner.
- The ability to show how ideas relate and interact in a specific context.
- The ability to write effectively in both sentences and paragraphs.
Constructing an Effective Essay Question
Following are a few tips to help in the construction of effective essay questions:
- Begin with the lesson objectives in mind. Make sure to know what you wish the student to show by answering the essay question.
- Decide if your goal requires a restricted or extended response. In general, if you wish to see if the student can synthesize and organize the information that they learned, then restricted response is the way to go. However, if you wish them to judge or evaluate something using the information taught during class, then you will want to use the extended response.
- If you are including more than one essay, be cognizant of time constraints. You do not want to punish students because they ran out of time on the test.
- Write the question in a novel or interesting manner to help motivate the student.
- State the number of points that the essay is worth. You can also provide them with a time guideline to help them as they work through the exam.
- If your essay item is part of a larger objective test, make sure that it is the last item on the exam.
Scoring the Essay Item
One of the downfalls of essay tests is that they lack in reliability. Even when teachers grade essays with a well-constructed rubric, subjective decisions are made. Therefore, it is important to try and be as reliable as possible when scoring your essay items. Here are a few tips to help improve reliability in grading:
- Determine whether you will use a holistic or analytic scoring system before you write your rubric . With the holistic grading system, you evaluate the answer as a whole, rating papers against each other. With the analytic system, you list specific pieces of information and award points for their inclusion.
- Prepare the essay rubric in advance. Determine what you are looking for and how many points you will be assigning for each aspect of the question.
- Avoid looking at names. Some teachers have students put numbers on their essays to try and help with this.
- Score one item at a time. This helps ensure that you use the same thinking and standards for all students.
- Avoid interruptions when scoring a specific question. Again, consistency will be increased if you grade the same item on all the papers in one sitting.
- If an important decision like an award or scholarship is based on the score for the essay, obtain two or more independent readers.
- Beware of negative influences that can affect essay scoring. These include handwriting and writing style bias, the length of the response, and the inclusion of irrelevant material.
- Review papers that are on the borderline a second time before assigning a final grade.
- How to Study Using the Basketball Review Game
- Creating Effective Fill-in-the-Blank Questions
- Teacher Housekeeping Tasks
- 4 Tips for Effective Classroom Management
- Dealing With Trips to the Bathroom During Class
- Tips to Cut Writing Assignment Grading Time
- Collecting Homework in the Classroom
- Why Students Cheat and How to Stop Them
- Planning Classroom Instruction
- Using Cloze Tests to Determine Reading Comprehension
- 10 Strategies to Boost Reading Comprehension
- Taking Daily Attendance
- How Scaffolding Instruction Can Improve Comprehension
- Field Trips: Pros and Cons
- Assignment Biography: Student Criteria and Rubric for Writing
- 3 Poetry Activities for Middle School Students
Rubric Best Practices, Examples, and Templates
A rubric is a scoring tool that identifies the different criteria relevant to an assignment, assessment, or learning outcome and states the possible levels of achievement in a specific, clear, and objective way. Use rubrics to assess project-based student work including essays, group projects, creative endeavors, and oral presentations.
Rubrics can help instructors communicate expectations to students and assess student work fairly, consistently and efficiently. Rubrics can provide students with informative feedback on their strengths and weaknesses so that they can reflect on their performance and work on areas that need improvement.
How to Get Started
Best practices, moodle how-to guides.
- Workshop Recording (Spring 2024)
- Workshop Registration
Step 1: Analyze the assignment
The first step in the rubric creation process is to analyze the assignment or assessment for which you are creating a rubric. To do this, consider the following questions:
- What is the purpose of the assignment and your feedback? What do you want students to demonstrate through the completion of this assignment (i.e. what are the learning objectives measured by it)? Is it a summative assessment, or will students use the feedback to create an improved product?
- Does the assignment break down into different or smaller tasks? Are these tasks equally important as the main assignment?
- What would an “excellent” assignment look like? An “acceptable” assignment? One that still needs major work?
- How detailed do you want the feedback you give students to be? Do you want/need to give them a grade?
Step 2: Decide what kind of rubric you will use
Types of rubrics: holistic, analytic/descriptive, single-point
Holistic Rubric. A holistic rubric includes all the criteria (such as clarity, organization, mechanics, etc.) to be considered together and included in a single evaluation. With a holistic rubric, the rater or grader assigns a single score based on an overall judgment of the student’s work, using descriptions of each performance level to assign the score.
Advantages of holistic rubrics:
- Can p lace an emphasis on what learners can demonstrate rather than what they cannot
- Save grader time by minimizing the number of evaluations to be made for each student
- Can be used consistently across raters, provided they have all been trained
Disadvantages of holistic rubrics:
- Provide less specific feedback than analytic/descriptive rubrics
- Can be difficult to choose a score when a student’s work is at varying levels across the criteria
- Any weighting of c riteria cannot be indicated in the rubric
Analytic/Descriptive Rubric . An analytic or descriptive rubric often takes the form of a table with the criteria listed in the left column and with levels of performance listed across the top row. Each cell contains a description of what the specified criterion looks like at a given level of performance. Each of the criteria is scored individually.
Advantages of analytic rubrics:
- Provide detailed feedback on areas of strength or weakness
- Each criterion can be weighted to reflect its relative importance
Disadvantages of analytic rubrics:
- More time-consuming to create and use than a holistic rubric
- May not be used consistently across raters unless the cells are well defined
- May result in giving less personalized feedback
Single-Point Rubric . A single-point rubric is breaks down the components of an assignment into different criteria, but instead of describing different levels of performance, only the “proficient” level is described. Feedback space is provided for instructors to give individualized comments to help students improve and/or show where they excelled beyond the proficiency descriptors.
Advantages of single-point rubrics:
- Easier to create than an analytic/descriptive rubric
- Perhaps more likely that students will read the descriptors
- Areas of concern and excellence are open-ended
- May removes a focus on the grade/points
- May increase student creativity in project-based assignments
Disadvantage of analytic rubrics: Requires more work for instructors writing feedback
Step 3 (Optional): Look for templates and examples.
You might Google, “Rubric for persuasive essay at the college level” and see if there are any publicly available examples to start from. Ask your colleagues if they have used a rubric for a similar assignment. Some examples are also available at the end of this article. These rubrics can be a great starting point for you, but consider steps 3, 4, and 5 below to ensure that the rubric matches your assignment description, learning objectives and expectations.
Step 4: Define the assignment criteria
Make a list of the knowledge and skills are you measuring with the assignment/assessment Refer to your stated learning objectives, the assignment instructions, past examples of student work, etc. for help.
Helpful strategies for defining grading criteria:
- Collaborate with co-instructors, teaching assistants, and other colleagues
- Brainstorm and discuss with students
- Can they be observed and measured?
- Are they important and essential?
- Are they distinct from other criteria?
- Are they phrased in precise, unambiguous language?
- Revise the criteria as needed
- Consider whether some are more important than others, and how you will weight them.
Step 5: Design the rating scale
Most ratings scales include between 3 and 5 levels. Consider the following questions when designing your rating scale:
- Given what students are able to demonstrate in this assignment/assessment, what are the possible levels of achievement?
- How many levels would you like to include (more levels means more detailed descriptions)
- Will you use numbers and/or descriptive labels for each level of performance? (for example 5, 4, 3, 2, 1 and/or Exceeds expectations, Accomplished, Proficient, Developing, Beginning, etc.)
- Don’t use too many columns, and recognize that some criteria can have more columns that others . The rubric needs to be comprehensible and organized. Pick the right amount of columns so that the criteria flow logically and naturally across levels.
Step 6: Write descriptions for each level of the rating scale
Artificial Intelligence tools like Chat GPT have proven to be useful tools for creating a rubric. You will want to engineer your prompt that you provide the AI assistant to ensure you get what you want. For example, you might provide the assignment description, the criteria you feel are important, and the number of levels of performance you want in your prompt. Use the results as a starting point, and adjust the descriptions as needed.
Building a rubric from scratch
For a single-point rubric , describe what would be considered “proficient,” i.e. B-level work, and provide that description. You might also include suggestions for students outside of the actual rubric about how they might surpass proficient-level work.
For analytic and holistic rubrics , c reate statements of expected performance at each level of the rubric.
- Consider what descriptor is appropriate for each criteria, e.g., presence vs absence, complete vs incomplete, many vs none, major vs minor, consistent vs inconsistent, always vs never. If you have an indicator described in one level, it will need to be described in each level.
- You might start with the top/exemplary level. What does it look like when a student has achieved excellence for each/every criterion? Then, look at the “bottom” level. What does it look like when a student has not achieved the learning goals in any way? Then, complete the in-between levels.
- For an analytic rubric , do this for each particular criterion of the rubric so that every cell in the table is filled. These descriptions help students understand your expectations and their performance in regard to those expectations.
Well-written descriptions:
- Describe observable and measurable behavior
- Use parallel language across the scale
- Indicate the degree to which the standards are met
Step 7: Create your rubric
Create your rubric in a table or spreadsheet in Word, Google Docs, Sheets, etc., and then transfer it by typing it into Moodle. You can also use online tools to create the rubric, but you will still have to type the criteria, indicators, levels, etc., into Moodle. Rubric creators: Rubistar , iRubric
Step 8: Pilot-test your rubric
Prior to implementing your rubric on a live course, obtain feedback from:
- Teacher assistants
Try out your new rubric on a sample of student work. After you pilot-test your rubric, analyze the results to consider its effectiveness and revise accordingly.
- Limit the rubric to a single page for reading and grading ease
- Use parallel language . Use similar language and syntax/wording from column to column. Make sure that the rubric can be easily read from left to right or vice versa.
- Use student-friendly language . Make sure the language is learning-level appropriate. If you use academic language or concepts, you will need to teach those concepts.
- Share and discuss the rubric with your students . Students should understand that the rubric is there to help them learn, reflect, and self-assess. If students use a rubric, they will understand the expectations and their relevance to learning.
- Consider scalability and reusability of rubrics. Create rubric templates that you can alter as needed for multiple assignments.
- Maximize the descriptiveness of your language. Avoid words like “good” and “excellent.” For example, instead of saying, “uses excellent sources,” you might describe what makes a resource excellent so that students will know. You might also consider reducing the reliance on quantity, such as a number of allowable misspelled words. Focus instead, for example, on how distracting any spelling errors are.
Example of an analytic rubric for a final paper
| Above Average (4) | Sufficient (3) | Developing (2) | Needs improvement (1) | |
|---|---|---|---|---|
| (Thesis supported by relevant information and ideas | The central purpose of the student work is clear and supporting ideas always are always well-focused. Details are relevant, enrich the work. | The central purpose of the student work is clear and ideas are almost always focused in a way that supports the thesis. Relevant details illustrate the author’s ideas. | The central purpose of the student work is identified. Ideas are mostly focused in a way that supports the thesis. | The purpose of the student work is not well-defined. A number of central ideas do not support the thesis. Thoughts appear disconnected. |
| (Sequencing of elements/ ideas) | Information and ideas are presented in a logical sequence which flows naturally and is engaging to the audience. | Information and ideas are presented in a logical sequence which is followed by the reader with little or no difficulty. | Information and ideas are presented in an order that the audience can mostly follow. | Information and ideas are poorly sequenced. The audience has difficulty following the thread of thought. |
| (Correctness of grammar and spelling) | Minimal to no distracting errors in grammar and spelling. | The readability of the work is only slightly interrupted by spelling and/or grammatical errors. | Grammatical and/or spelling errors distract from the work. | The readability of the work is seriously hampered by spelling and/or grammatical errors. |
Example of a holistic rubric for a final paper
| The audience is able to easily identify the central message of the work and is engaged by the paper’s clear focus and relevant details. Information is presented logically and naturally. There are minimal to no distracting errors in grammar and spelling. : The audience is easily able to identify the focus of the student work which is supported by relevant ideas and supporting details. Information is presented in a logical manner that is easily followed. The readability of the work is only slightly interrupted by errors. : The audience can identify the central purpose of the student work without little difficulty and supporting ideas are present and clear. The information is presented in an orderly fashion that can be followed with little difficulty. Grammatical and spelling errors distract from the work. : The audience cannot clearly or easily identify the central ideas or purpose of the student work. Information is presented in a disorganized fashion causing the audience to have difficulty following the author’s ideas. The readability of the work is seriously hampered by errors. |
Single-Point Rubric
| Advanced (evidence of exceeding standards) | Criteria described a proficient level | Concerns (things that need work) |
|---|---|---|
| Criteria #1: Description reflecting achievement of proficient level of performance | ||
| Criteria #2: Description reflecting achievement of proficient level of performance | ||
| Criteria #3: Description reflecting achievement of proficient level of performance | ||
| Criteria #4: Description reflecting achievement of proficient level of performance | ||
| 90-100 points | 80-90 points | <80 points |
More examples:
- Single Point Rubric Template ( variation )
- Analytic Rubric Template make a copy to edit
- A Rubric for Rubrics
- Bank of Online Discussion Rubrics in different formats
- Mathematical Presentations Descriptive Rubric
- Math Proof Assessment Rubric
- Kansas State Sample Rubrics
- Design Single Point Rubric
Technology Tools: Rubrics in Moodle
- Moodle Docs: Rubrics
- Moodle Docs: Grading Guide (use for single-point rubrics)
Tools with rubrics (other than Moodle)
- Google Assignments
- Turnitin Assignments: Rubric or Grading Form
Other resources
- DePaul University (n.d.). Rubrics .
- Gonzalez, J. (2014). Know your terms: Holistic, Analytic, and Single-Point Rubrics . Cult of Pedagogy.
- Goodrich, H. (1996). Understanding rubrics . Teaching for Authentic Student Performance, 54 (4), 14-17. Retrieved from
- Miller, A. (2012). Tame the beast: tips for designing and using rubrics.
- Ragupathi, K., Lee, A. (2020). Beyond Fairness and Consistency in Grading: The Role of Rubrics in Higher Education. In: Sanger, C., Gleason, N. (eds) Diversity and Inclusion in Global Higher Education. Palgrave Macmillan, Singapore.
Structure and Scoring of the Assessment
The structure of the assessment.
You'll begin by reading a prose passage of 700-1,000 words. This passage will be about as difficult as the readings in first-year courses at UC Berkeley. You'll have up to two hours to read the passage carefully and write an essay in response to a single topic and related questions based on the passage's content. These questions will generally ask you to read thoughtfully and to provide reasoned, concrete, and developed presentations of a specific point of view. Your essay will be evaluated on the basis of your ability to develop your central idea, to express yourself clearly, and to use the conventions of written English.
Five Qualities of a Well-Written Essay
There is no "correct" response for the topic, but there are some things readers will look for in a strong, well-written essay.
- The writer demonstrates that they understood the passage.
- The writer maintains focus on the task assigned.
- The writer leads readers to understand a point of view, if not to accept it.
- The writer develops a central idea and provides specific examples.
- The writer evaluates the reading passage in light of personal experience, observations, or by testing the author's assumptions against their own.
Scoring is typically completed within three weeks after the assessment date. The readers are UC Berkeley faculty members, primarily from College Writing Programs, though faculty from other related departments, such as English or Comparative Literature might participate as well.
Your essay will be scored independently by two readers, who will not know your identity. They will measure your essay against a scoring guide. If the two readers have different opinions, then a third reader will assess your essay as well to help reach a final decision. Each reader will give your essay a score on a scale of 1 (lowest) to 6 (highest). When your two scores are added together, if they are 8 or higher, you satisfy the Entry Level Writing Requirement and may take any 4-unit "R_A" course (first half of the requirement, usually numbered R1A, though sometimes with a different number). If you receive a score less than 8, you should sign up for College Writing R1A, which satisfies both the Entry Level Writing Requirement and the first-semester ("A" part) of the Reading and Composition Requirement.
The Scoring Guide
The Scoring Guide outlines the characteristics typical of essays at six different levels of competence. Readers assign each essay a score according to its main qualities. Readers take into account the fact that the responses are written with two hours of reading and writing, without a longer period of time for drafting and revision.
An essay with a score of 6 may
- command attention because of its insightful development and mature style.
- present a cogent response to the text, elaborating that response with well-chosen examples and persuasive reasoning.
- present an organization that reinforces the development of the ideas which are aptly detailed.
- show that its writer can usually choose words well, use sophisticated sentences effectively, and observe the conventions of written English.
An essay with a score of 5 may
clearly demonstrate competent writing skill.
present a thoughtful response to the text, elaborating that response with appropriate examples and sensible reasoning.
present an organization that supports the writer’s ideas, which are developed with greater detail than is typical of an essay scored '4.'
have a less fluent and complex style than an essay scored '6,' but shows that the writer can usually choose words accurately, vary sentences effectively, and observe the conventions of written English.
An essay with a score of 4 may
be just 'satisfactory.'
present an adequate response to the text, elaborating that response with sufficient examples and acceptable reasoning.
demonstrate an organization that generally supports the writer’s ideas, which are developed with sufficient detail.
use examples and reasoning that are less developed than those in '5' essays.
show that its writer can usually choose words of sufficient precision, control sentences of reasonable variety, and observe the conventions of written English.
An essay with a score of 3 may
be unsatisfactory in one or more of the following ways:
It may respond to the text illogically
it may reflect an incomplete understanding of the text or the topic
it may provide insufficient reasoning or lack elaboration with examples, or the examples provided may not be sufficiently detailed to support claims
it may be inadequately organized
have prose characterized by at least one of the following:
frequently imprecise word choice
little sentence variety
occasional major errors in grammar and usage, or frequent minor errors
An essay with a score of 2 may
show weaknesses, ordinarily of several kinds.
present a simplistic or inappropriate response to the text, one that may suggest some significant misunderstanding of the text or the topic
use organizational strategies that detract from coherence or provide inappropriate or irrelevant detail.
simplistic or inaccurate word choice
monotonous or fragmented sentence structure
many repeated errors in grammar and usage
An essay with a score of 1 may
show serious weaknesses.
disregard the topic's demands, or it may lack structure or development.
Have an organization that fails to support the essay’s ideas.
be inappropriately brief.
have a pattern of errors in word choice, sentence structure, grammar, and usage.
- CRLT Consultation Services
- Consultation
- Midterm Student Feedback
- Classroom Observation
- Teaching Philosophy
- Upcoming Events and Seminars
- CRLT Calendar
- Orientations
- Teaching Academies
- Provost's Seminars
- For Faculty
- For Grad Students & Postdocs
- For Chairs, Deans & Directors
- Customized Workshops & Retreats
- Assessment, Curriculum, & Learning Analytics Services
- CRLT in Engineering
- CRLT Players
- Foundational Course Initiative
- CRLT Grants
- Other U-M Grants
- Provost's Teaching Innovation Prize
- U-M Teaching Awards
- Retired Grants
- Staff Directory
- Faculty Advisory Board
- Annual Report
- Equity-Focused Teaching
- Preparing to Teach
- Teaching Strategies
- Testing and Grading
- Teaching with Technology
- Teaching Philosophy & Statements
- Training GSIs
- Evaluation of Teaching
- Occasional Papers
Best Practices for Designing and Grading Exams
Adapted from crlt occasional paper #24: m.e. piontek (2008), center for research on learning and teaching.
The most obvious function of assessment methods (such as exams, quizzes, papers, and presentations) is to enable instructors to make judgments about the quality of student learning (i.e., assign grades). However, the method of assessment also can have a direct impact on the quality of student learning. Students assume that the focus of exams and assignments reflects the educational goals most valued by an instructor, and they direct their learning and studying accordingly (McKeachie & Svinicki, 2006). General grading systems can have an impact as well. For example, a strict bell curve (i.e., norm-reference grading) has the potential to dampen motivation and cooperation in a classroom, while a system that strictly rewards proficiency (i.e., criterion-referenced grading ) could be perceived as contributing to grade inflation. Given the importance of assessment for both faculty and student interactions about learning, how can instructors develop exams that provide useful and relevant data about their students' learning and also direct students to spend their time on the important aspects of a course or course unit? How do grading practices further influence this process?
Guidelines for Designing Valid and Reliable Exams
Ideally, effective exams have four characteristics. They are:
- Valid, (providing useful information about the concepts they were designed to test),
- Reliable (allowing consistent measurement and discriminating between different levels of performance),
- Recognizable (instruction has prepared students for the assessment), and
- Realistic (concerning time and effort required to complete the assignment) (Svinicki, 1999).
Most importantly, exams and assignments should f ocus on the most important content and behaviors emphasized during the course (or particular section of the course). What are the primary ideas, issues, and skills you hope students learn during a particular course/unit/module? These are the learning outcomes you wish to measure. For example, if your learning outcome involves memorization, then you should assess for memorization or classification; if you hope students will develop problem-solving capacities, your exams should focus on assessing students’ application and analysis skills. As a general rule, assessments that focus too heavily on details (e.g., isolated facts, figures, etc.) “will probably lead to better student retention of the footnotes at the cost of the main points" (Halpern & Hakel, 2003, p. 40). As noted in Table 1, each type of exam item may be better suited to measuring some learning outcomes than others, and each has its advantages and disadvantages in terms of ease of design, implementation, and scoring.
Table 1: Advantages and Disadvantages of Commonly Used Types of Achievement Test Items
|
|
|
|
|
| Many items can be administered in a relatively short time. Moderately easy to write; easily scored. | Limited primarily to testing knowledge of information. Easy to guess correctly on many items, even if material has not been mastered. |
|
| Can be used to assess broad range of content in a brief period. Skillfully written items can measure higher order cognitive skills. Can be scored quickly. | Difficult and time consuming to write good items. Possible to assess higher order cognitive skills, but most items assess only knowledge. Some correct answers can be guesses. |
|
| Items can be written quickly. A broad range of content can be assessed. Scoring can be done efficiently. | Higher order cognitive skills are difficult to assess. |
|
| Many can be administered in a brief amount of time. Relatively efficient to score. Moderately easy to write. | Difficult to identify defensible criteria for correct answers. Limited to questions that can be answered or completed in very few words. |
|
| Can be used to measure higher order cognitive skills. Relatively easy to write questions. Difficult for respondent to get correct answer by guessing. | Time consuming to administer and score. Difficult to identify reliable criteria for scoring. Only a limited range of content can be sampled during any one testing period. |
Adapted from Table 10.1 of Worthen, et al., 1993, p. 261.
General Guidelines for Developing Multiple-Choice and Essay Questions
The following sections highlight general guidelines for developing multiple-choice and essay questions, which are often used in college-level assessment because they readily lend themselves to measuring higher order thinking skills (e.g., application, justification, inference, analysis and evaluation). Yet instructors often struggle to create, implement, and score these types of questions (McMillan, 2001; Worthen, et al., 1993).
Multiple-choice questions have a number of advantages. First, they can measure various kinds of knowledge, including students' understanding of terminology, facts, principles, methods, and procedures, as well as their ability to apply, interpret, and justify. When carefully designed, multiple-choice items also can assess higher-order thinking skills.
Multiple-choice questions are less ambiguous than short-answer items, thereby providing a more focused assessment of student knowledge. Multiple-choice items are superior to true-false items in several ways: on true-false items, students can receive credit for knowing that a statement is incorrect, without knowing what is correct. Multiple-choice items offer greater reliability than true-false items as the opportunity for guessing is reduced with the larger number of options. Finally, an instructor can diagnose misunderstanding by analyzing the incorrect options chosen by students.
A disadvantage of multiple-choice items is that they require developing incorrect, yet plausible, options that can be difficult to create. In addition, multiple- choice questions do not allow instructors to measure students’ ability to organize and present ideas. Finally, because it is much easier to create multiple-choice items that test recall and recognition rather than higher order thinking, multiple-choice exams run the risk of not assessing the deep learning that many instructors consider important (Greenland & Linn, 1990; McMillan, 2001).
Guidelines for writing multiple-choice items include advice about stems, correct answers, and distractors (McMillan, 2001, p. 150; Piontek, 2008):
- S tems pose the problem or question.
- Is the stem stated as clearly, directly, and simply as possible?
- Is the problem described fully in the stem?
- Is the stem stated positively, to avoid the possibility that students will overlook terms like “no,” “not,” or “least”?
- Does the stem provide only information relevant to the problem?
Possible responses include the correct answer and distractors , or the incorrect choices. Multiple-choice questions usually have at least three distractors.
- Are the distractors plausible to students who do not know the correct answer?
- Is there only one correct answer?
- Are all the possible answers parallel with respect to grammatical structure, length, and complexity?
- Are the options short?
- Are complex options avoided? Are options placed in logical order?
- Are correct answers spread equally among all the choices? (For example, is answer “A” correct about the same number of times as options “B” or “C” or “D”)?
An example of good multiple-choice questions that assess higher-order thinking skills is the following test question from pharmacy (Park, 2008):
Patient WC was admitted for third-degree burns over 75% of his body. The attending physician asks you to start this patient on antibiotic therapy. Which one of the following is the best reason why WC would need antibiotic prophylaxis? a. His burn injuries have broken down the innate immunity that prevents microbial invasion. b. His injuries have inhibited his cellular immunity. c. His injuries have impaired antibody production. d. His injuries have induced the bone marrow, thus activated immune system
A second question builds on the first by describing the patient’s labs two days later, asking the students to develop an explanation for the subsequent lab results. (See Piontek, 2008 for the full question.)
Essay questions can tap complex thinking by requiring students to organize and integrate information, interpret information, construct arguments, give explanations, evaluate the merit of ideas, and carry out other types of reasoning (Cashin, 1987; Gronlund & Linn, 1990; McMillan, 2001; Thorndike, 1997; Worthen, et al., 1993). Restricted response essay questions are good for assessing basic knowledge and understanding and generally require a brief written response (e.g., “State two hypotheses about why birds migrate. Summarize the evidence supporting each hypothesis” [Worthen, et al., 1993, p. 277].) Extended response essay items allow students to construct a variety of strategies, processes, interpretations and explanations for a question, such as the following:
The framers of the Constitution strove to create an effective national government that balanced the tension between majority rule and the rights of minorities. What aspects of American politics favor majority rule? What aspects protect the rights of those not in the majority? Drawing upon material from your readings and the lectures, did the framers successfully balance this tension? Why or why not? (Shipan, 2008).
In addition to measuring complex thinking and reasoning, advantages of essays include the potential for motivating better study habits and providing the students flexibility in their responses. Instructors can evaluate how well students are able to communicate their reasoning with essay items, and they are usually less time consuming to construct than multiple-choice items that measure reasoning.
The major disadvantages of essays include the amount of time instructors must devote to reading and scoring student responses, and the importance of developing and using carefully constructed criteria/rubrics to insure reliability of scoring. Essays can assess only a limited amount of content in one testing period/exam due to the length of time required for students to respond to each essay item. As a result, essays do not provide a good sampling of content knowledge across a curriculum (Gronlund & Linn, 1990; McMillan, 2001).
Guidelines for writing essay questions include the following (Gronlund & Linn, 1990; McMillan, 2001; Worthen, et al., 1993):
- Restrict the use of essay questions to educational outcomes that are difficult to measure using other formats. For example, to test recall knowledge, true-false, fill-in-the-blank, or multiple-choice questions are better measures.
- Generalizations : State a set of principles that can explain the following events.
- Synthesis : Write a well-organized report that shows…
- Evaluation : Describe the strengths and weaknesses of…
- Write the question clearly so that students do not feel that they are guessing at “what the instructor wants me to do.”
- Indicate the amount of time and effort students should spend on each essay item.
- Avoid giving students options for which essay questions they should answer. This choice decreases the validity and reliability of the test because each student is essentially taking a different exam.
- Consider using several narrowly focused questions (rather than one broad question) that elicit different aspects of students’ skills and knowledge.
- Make sure there is enough time to answer the questions.
Guidelines for scoring essay questions include the following (Gronlund & Linn, 1990; McMillan, 2001; Wiggins, 1998; Worthen, et al., 1993; Writing and grading essay questions , 1990):
- Outline what constitutes an expected answer.
- Select an appropriate scoring method based on the criteria. A rubric is a scoring key that indicates the criteria for scoring and the amount of points to be assigned for each criterion. A sample rubric for a take-home history exam question might look like the following:
|
| |||
|
|
|
| |
| Number of references to class reading sources | 0-2 references | 3-5 references | 6+ references |
| Historical accuracy | Lots of inaccuracies | Few inaccuracies | No apparent inaccuracies |
| Historical Argument | No argument made; little evidence for argument | Argument is vague and unevenly supported by evidence | Argument is clear and well-supported by evidence |
| Proof reading | Many grammar and spelling errors | Few (1-2) grammar or spelling errors | No grammar or spelling errors |
|
| |||
For other examples of rubrics, see CRLT Occasional Paper #24 (Piontek, 2008).
- Clarify the role of writing mechanics and other factors independent of the educational outcomes being measured. For example, how does grammar or use of scientific notation figure into your scoring criteria?
- Create anonymity for students’ responses while scoring and create a random order in which tests are graded (e.g., shuffle the pile) to increase accuracy of the scoring.
- Use a systematic process for scoring each essay item. Assessment guidelines suggest scoring all answers for an individual essay question in one continuous process, rather than scoring all answers to all questions for an individual student. This system makes it easier to remember the criteria for scoring each answer.
You can also use these guidelines for scoring essay items to create grading processes and rubrics for students’ papers, oral presentations, course projects, and websites. For other grading strategies, see Responding to Student Writing – Principles & Practices and Commenting Effectively on Student Writing .
Cashin, W. E. (1987). Improving essay tests . Idea Paper, No. 17. Manhattan, KS: Center for Faculty Evaluation and Development, Kansas State University.
Gronlund, N. E., & Linn, R. L. (1990). Measurement and evaluation in teaching (6th ed.). New York: Macmillan Publishing Company.
Halpern, D. H., & Hakel, M. D. (2003). Applying the science of learning to the university and beyond. Change, 35 (4), 37-41.
McKeachie, W. J., & Svinicki, M. D. (2006). Assessing, testing, and evaluating: Grading is not the most important function. In McKeachie's Teaching tips: Strategies, research, and theory for college and university teachers (12th ed., pp. 74-86). Boston: Houghton Mifflin Company.
McMillan, J. H. (2001). Classroom assessment: Principles and practice for effective instruction. Boston: Allyn and Bacon.
Park, J. (2008, February 4). Personal communication. University of Michigan College of Pharmacy.
Piontek, M. (2008). Best practices for designing and grading exams. CRLT Occasional Paper No. 24 . Ann Arbor, MI. Center for Research on Learning and Teaching.>
Shipan, C. (2008, February 4). Personal communication. University of Michigan Department of Political Science.
Svinicki, M. D. (1999a). Evaluating and grading students. In Teachers and students: A sourcebook for UT- Austin faculty (pp. 1-14). Austin, TX: Center for Teaching Effectiveness, University of Texas at Austin.
Thorndike, R. M. (1997). Measurement and evaluation in psychology and education. Upper Saddle River, NJ: Prentice-Hall, Inc.
Wiggins, G. P. (1998). Educative assessment: Designing assessments to inform and improve student performance . San Francisco: Jossey-Bass Publishers.
Worthen, B. R., Borg, W. R., & White, K. R. (1993). Measurement and evaluation in the schools . New York: Longman.
Writing and grading essay questions. (1990, September). For Your Consideration , No. 7. Chapel Hill, NC: Center for Teaching and Learning, University of North Carolina at Chapel Hill.
back to top
Contact CRLT
location_on University of Michigan 1071 Palmer Commons 100 Washtenaw Ave. Ann Arbor, MI 48109-2218
phone Phone: (734) 764-0505
description Fax: (734) 647-3600
email Email: [email protected]
Connect with CRLT
directions Directions to CRLT
group Staff Directory
markunread_mailbox Subscribe to our Blog
- All topics A-Z
- Grammar
- Vocabulary
- Speaking
- Reading
- Listening
- Writing
- Pronunciation
- Virtual Classroom
- Worksheets by season
- 600 Creative Writing Prompts
- Warmers, fillers & ice-breakers
- Coloring pages to print
- Flashcards
- Classroom management worksheets
- Emergency worksheets
- Revision worksheets
- Resources we recommend
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
IMAGES
COMMENTS
Here's a look at essay tests as a whole with advice about creating and scoring essay tests.
Rubric Best Practices, Examples, and Templates A rubric is a scoring tool that identifies the different criteria relevant to an assignment, assessment, or learning outcome and states the possible levels of achievement in a specific, clear, and objective way. Use rubrics to assess project-based student work including essays, group projects, creative endeavors, and oral presentations.
Rules for Scoring Essay and Short-Answer Items Because of their subjective nature, essay and short-answer items are difficult to grade, particularly if the score scale contains many points. The same items that are easy to grade on a 3-point scale may be very hard to grade on a 5- or 10-point scale.
The Scoring Guide The Scoring Guide outlines the characteristics typical of essays at six different levels of competence. Readers assign each essay a score according to its main qualities.
Abstract This encyclopedic entry on essay items provides a general definition of this item type, scoring procedures, and challenges to gathering validity evidence.
2.2 Writing Essay Test Items Essay items are useful when examinees have to show how they arrived at an answer. A test of writing ability is a good example of the kind of test that should be given in an essay response format. This type of item, however, is difficult to score reliably and can require a significant amount of time to be graded. Grading is often affected by the verbal fluency in ...
You can also use these guidelines for scoring essay items to create grading processes and rubrics for students' papers, oral presentations, course projects, and websites.
A rubric is a chart used in grading essays, special projects and other more items which can be more subjective. It lists each of the grading criteria separately and defines the different performance levels within those criteria. Standardized tests like the SAT's use rubrics to score writing samples, and designing one for your own use is easy if you take it step by step. Keep in mind that ...
Constructed-response items (often known as performance tasks) — A task that requires test takers to construct answers rather than select from predetermined multiple-choice options; examples include essays, works of art or speeches. Rubric — The set of scoring standards that describes the criteria for each score level.
Outline what constitutes an acceptable answer (criteria for knowledge and skills). Select an appropriate scoring method based on the criteria. Clarify the role of writing mechanics and other factors independent of the learning aims being measured. each essay item. For instance, score all responses to a single question in one setting.
An excellent paper: Argument: The paper knows what it wants to say and why it wants to say it. It goes beyond pointing out comparisons to using them to change the reader?s vision. Organization: Every paragraph supports the main argument in a coherent way, and clear transitions point out why each new paragraph follows the previous one. Evidence ...
Objective items include multiple-choice, true-false, matching and completion, while subjective items include short-answer essay, extended-response essay, problem solving and performance test items. For some instructional purposes one or the other item types may prove more efficient and appropriate.
C. Scoring Essay Items Although essay questions are powerful assessment tools, they can be difficult to score. With essays, there isn't a single, correct answer and it is almost impossible to use an automatic scantron or computer-based system. In order to minimize the subjectivity and bias that may occur in the assessment, teachers should prepare a list of criteria prior to scoring the essays ...
Evaluating Written Items A key point in scoring supply-type written responses, such as essay items, is that the criteria for scoring must be determined in advance! Typically this involves two areas - content as well as the structure and style of the respondent's writing. This discussion will focus on content.
Since essay questions typically sample a limited range of content, are time consuming to score, and involve greater subjectivity in scoring than objectively scored items, the use of essay questions should be reserved for learning outcomes that cannot be better assessed by some other means.
Learn best practices for grading bar exam essays to ensure that they serve as reliable and valid indicators of competence to practice law.
This study tries to explain various methods of essay tests scoring in different published papers. Materials & Methods: In this study, first the different papers and articles published in national and international journals were selected by using keywords of essay test, scoring, and student assessment. Later, they were studied and analyzed.
Evaluation and Grading Criteria for Essays IVCC's online Style Book presents the Grading Criteria for Writing Assignments. This page explains some of the major aspects of an essay that are given special attention when the essay is evaluated.
As a result, automated essay scoring systems generate a single score or detailed evaluation of predefined assessment features. This chapter describes the evolution and features of automated scoring systems, discusses their limitations, and concludes with future directions for research and practice.
Abstract. In this paper, we propose an architecture of automated essay scoring system based on rubric, which combines automated scoring with human scoring. Rubrics are valid criteria for grading students' essays. Our proposed rubric has five evaluation viewpoints "Contents, Structure, Evidence, Style, and Skill" and 25 evaluation items ...
You can also use these guidelines for scoring essay items to create grading processes and rubric s for students' papers, oral presentations, course projects, and websites.
A scoring rubric is a standard of performance for a defined population. It is a pre-determined set of goals and objectives on which to base an evaluation. In the Higher Education Report, S.M. Brookhart describes a scoring rubric as, "Descriptive scoring schemes that are developed by teachers or other evaluators to guide the analysis of the products or processes of students' efforts."
However, the assessment (scoring) of these writing compositions or essays is a very challenging process in terms of reliability and time. The need for objective and quick scores has raised the need for a computer system that can automatically grade essay questions targeting specific prompts.