Writing Effective Short Answer Questions

Short answer itemsare an effective measure of a student's ability to accurately recall specific,target information. Short answer items require students to either complete astatement (fill-in-the-blank or completion items) or answer a direct questionusing a single word or brief phrase. Within the eLearning context, the short-answer question type is designed for a one-to-three word response. While a longer response can be required and accommodated, the grading of longer responses increases the likelihood of instructor involvement in the grading process.

As opposed to traditional objective measures (true-false, matching, multiple-choice, etc.) that assess the "recognition" of correct information, short answer items require students to independently generate their own response. Therefore, short answer assessments may decrease the likelihood of student misconduct.

While this type of recall assessment is more cognitively demanding, the independent nature of the responses makes scoring more subjective. Due to the subjective interpretation of short answer items and the increased challenges in assessing the accuracy of responses, instructors may wish to examine the utility of short answer items in relationship to their instructional objectives.

Tips for Writing Short Answer Items:

• Questions must be clearly worded so that students understand the nature of the information being requested.
  • In order to facilitate understanding, phrase the item so that the required answer is brief and specific.
    • Faculty may consider providing clues to indicate the expected nature of the response.
      • I.e., the distance from the earth to the sun is ______.
        • How is the student to answer this question, miles, kilometers, furlongs?
        
        
        
• To ensure that a short answer question is an effective measure of student knowledge, require students to fill in important terms or phrases.
  • For example, when assessing understanding of definitions, have students supply the term.
  
  
  
• When utilizing short answer questions that require a numerical response, specify the degree of precision that is expected and the relevant units of measurement.



• To prevent confusion and make scoring more precise, phrase question so there is only one answer or a limited range of answers possible.
  • If multiple answers will correctly complete the item, provide a pre-established scoring rubric to deal with variations in response.
  
  
  
• Leave information to be filled in at or near the end of the question.
  • This type of arrangement allows for ease of reading and enhances the efficiency of scoring. For example:
    • The animal with the highest blood pressure is the ___________. (Good)
    • The animal with the highest ____________ is the giraffe. (Not so good)
    
    
    
• Utilize clear, explicit instructions that specify the format of the target answer (one word, multiple words, etc.) as well as the amount of acceptable variation (spelling, synonyms, etc.).



• To prevent confusion and ensure requested information is clear, limit the number of blanks within each short answer question.
  • In addition, ensure that blanks are the same physical length to prevent context clues to the correct answer.
  
  
  
• Limit the influence of extraneous clues to the correct answer by utilizing correct, neutral grammar.
  • Avoid providing grammatical clues to the correct answer (plurals, "a" versus "an," specific modifiers, etc.) and make certain that all correct responses can fit grammatically in the blank.
  • Another thought along this line: the response blank in all questions should be the same size. Again, this is done to avoid providing clues to the correct answer.
  
  
  
• To reduce the emphasis on rote memorization of trivial information, do not use direct quotes from the text or lecture.
  • Rather, phrase short answer items using unique or novel wording.

• To ensure an accurate measure of target information, use direct questions rather than fill-in-the-blank or incomplete statements.
  • This type of wording reduces confusion or ambiguity concerning the request information and directs students toward the relevant information.
  
  
  
• One of the benefits of short answer items is that they often encourage more intensive study of information due to the increased cognitive demands of recall over recognition.
  • GOOD IDEA: To promote this type of invested studying, award more credit for short answer items than for lower level recognition items (true-false, matching, etc.).
  
  
  
• While short answer items often target knowledge or comprehension understanding, effectively developed completion items can also be utilized to assess application, synthesis, analysis, and evaluation levels.
  • One means of measuring this type of higher-order understanding is to utilize combinations of short answer statements within a given paragraph.
  • When implementing the paragraph format, be sure that desired knowledge is clearly specified.

Advantages of short answer questions:

• Easy to write.
• Reduces possibility of guessing.
• Can discriminate achievement effectively.
• Can have a lengthy stem such as a paragraph.

Disadvantages of short answer questions:

• Essentially a measure of memorization ability.
• Grading can be subjective.
• Exact answers are not likely because of misspellings, order of words, etc.
  • Caution: You should generally not expect an exact word-for-word, character-for-character answer. Consider possible alternatives that you would be willing to accept.

Resource Links:

• Teaching Tips: Test Item Writing (University of Alabama at Birmingham)
• Introduction to Short Answer and Completion Items (Alabama Professional Development Modules)
• Selecting Test Items (Virginia Polytechnic Institute and State University)

Used with permission from the
The Center for Excellence in Teaching and Learning at Park University (http://www.park.edu/)

If you have any questions about online assessment, please feel free to contact John Crane (jcrane@uwf.edu, 473-7239).

Have a great day!