Questions to Avoid
From SurveyWiki
Jump to navigationJump to searchAs you form questions for a text, avoid:
- Yes-no questions – Such questions allow the subject to achieve a unnaturally high score simply by guessing.
- Questions that could be answered with common sense – Questions that don't necessarily require an understanding of the text enable a subject to score correctly even if he or she did not comprehend that text section.
- Questions that ask for implied information, as opposed to information explicitly stated – It's likely that there will be a variety of answers to such questions, because subjects may provide opinions or guesses. This problem may arise especially when questioning "purpose." For example:
- "My mother picked up the axe and ran toward the highlander."
- "Why did his mother pick up the axe?"
- Questioning elements that are or use loanwords in the text – You are testing comprehension of the test language, not the language(s) from which it borrows words. Additionally, those words may be limited to particular domains, and their recognition will depend on subjects' familiarity with the domain in which the word is used. Another reason to avoid loanwords is that two related varieties may have borrowed words from different languages.
- Questions which ask for a long series of events – The RTT tests comprehension, not memory. Though they understand the text equally, some people will score better than others because they remember more of the text.
- Questions which may be considered ambiguous – Unclear questions are likely to be discarded in the pilot testing process, but avoid them to save your subjects some confusion and increase the number of good questions on the test.
- Writing questions that are difficult to translate – A simple question eases translation and avoids the chance that a subject won't understand it because its construction is too complex.
