Testing Participants

Recorded Text Testing
Background Research
Intelligibility Interviews
Choose Kind of Test
Choose Test Points
Materials
Obtain a Text
Develop Questions
Create Introduction Text
Assemble Pilot Test
Administer Pilot Test
Select Final Questions
Build Reference Tests
Develop Post-RTT Questions
Administer Text Test Set
Process Scores

Administer the Pilot Test
Before Testing Participants
Testing Participants

How to Administer a Pilot Text Test

To use this procedural guide, you will need to have the complete list of questions and answers for the pilot test written out, with the track numbers corresponding to each question's location written beside the question.

1. Follow the procedural guides for translating and recording the introduction text. Record the introduction text in the speech variety of the pilot test on the same MiniDisc (disc) as the 30-question pilot test. Note (in your data management notebook) the track number assigned to the introduction text. Only one track is needed for this text.
2. Find a helper who can translate your instructions to the test subjects from the LWC into the local language. Explain the steps involved in test administration so that the translator understands what you will be doing.
3. Ask for help in finding potential test subjects.
4. When a test subject arrives, sit him or her down in a comfortable setting and briefly explain what the test is like. You may need to explain how the voices were recorded and that they can be heard through the headphones.
5. Double check that he or she meets the basic qualifications needed to participate in this pilot test.
6. Explain that the first speech he or she will hear is an announcement.
7. Play the introduction text and pause at the end of the text.
8. Check the introduction by having the first few test subjects explain it in their own words. If they don't understand it, then you need to write, record, or translate the introduction text again.
9. Cue the MD player to the beginning of the test text and put the list of questions and answers in front of you so that you can follow along. If you set your MD player's counter to count down the time remaining in each track, it will help you know when to pause the test.
10. Play the first section of text and the first question track and then pause the MD player.
11. If the test subject does not answer automatically after hearing the question, then ask him or her to answer the question. If the test subject seems unable to answer the question, you may play the question track again. Remember - do not repeat the text before the question and give them an unfair chance to hear it again!
12. If he or she is still unable to answer the question, place a mark next to its track number. This question may be skipped with the subjects that follow, because no question that was missed in the pilot test may be used in the final test. When such questions come the next time, simply "jump" or "skip" to the next track.
13. If a pilot test subject is missing nearly every question or is not answering them, it may be that he or she is having a hard time with the test methodology itself. Graciously dismiss him while thanking him for his time, and then call the next participant. In this case, you should continue to play the questions that the dismissed subject missed.
14. If the test subject answers the question correctly, there is no need to make any mark.
15. Play the next text segment and question, and then pause the MD player.
16. Repeat steps 4 - 14 until the entire text is finished. Repeat the process for each of the additional subjects required for the pilot test.

Tips from the Field

• When you play the test without any disruptions or distractions due to operator error, you keep the subject's focus on listening to the text and questions.
• You are going to use the pilot test subjects' answers to choose the questions for the final test. Writing down their responses correctly helps ensure that good questions aren't discarded or bad questions retained.
• Responses can be different from what you expect but still be correct. To score the pilot or final test accurately, you need to be able to judge the answers quickly and fairly.
• Errors in forming the test or in analyzing the final results can occur if the correct notation is not used for a response, even when that response has been evaluated correctly. Practicing your local method of marking scores will help you develop consistency and familiarity.
• If more than one person administers a RTT, it is possible for similar answers to be scored differently. Making the scores the same when possible will allow you to keep questions that otherwise might need to be deleted or disregarded.