Build Reference Tests

Recorded Text Testing
Background Research
Intelligibility Interviews
Choose Kind of Test
Choose Test Points
Materials
Obtain a Text
Develop Questions
Create Introduction Text
Assemble Pilot Test
Administer Pilot Test
Select Final Questions
Build Reference Tests
Develop Post-RTT Questions
Administer Text Test Set
Process Scores

The reference tests will provide the data you use to draw conclusions about comprehension. When each test is assembled properly, variation due to error in the test will be reduced. The more you can attribute variation in the results to differences in comprehension, the better your test will be.

Adjusting the Introduction

The basic information that a subject needs to understand about the RTT will probably not change from test to test, but it's likely that there are other facts peculiar to each survey. Since each people differs from the next, considering the context of each may improve the impact of the introduction text. Transforming technical or academic information into simple, informative content is difficult, especially when it will also be translated into another language. The introduction text is intended to inform and instruct the subjects - and put them more at ease. If it is translated inaccurately or recorded badly, then the subjects may have problems agreeing to and taking the test.

Building the Reference Test

1. Set both MiniDisc machines side by side on a table and attach the patch cord to the "Line Out" jack on the PLAYER and the "Line In" on the RECORDER. Be sure that the RECORDER is set to record from the line and not the microphone.
2. Place the disc that contains the pilot text test with the 12 inserted questions in the PLAYER.
3. Cue the PLAYER to begin playing the first track of the text of this pilot test. Scroll to the beginning of the first track of the text and then pause the machine.
4. Set up the countdown feature on the PLAYER, so that you will see how much time is left on any given track. This will help you know when you need to be ready to pause the machine so that you can insert a question in the reference speech variety.
5. Place the target disc onto which you wish to record the new reference text test in the RECORDER.
6. Cue the machine to begin recording, but pause it in the recording mode.
7. In your bound notebook, set up a chart to show the contents of this new reference text test. Include the name of the area where the text test will be played (the name of the speech variety into which the questions have been translated). Include the name of the text itself, since you will likely be constructing more than one text test for that location. Write down each track number of the text test and include the content of each track. Your translation of the text is tied to the track numbers and if you mismanage those numbers, you may have to retranslate the text.
8. Some have found it helpful to design the score sheet at the same time the text test is built. A good score sheet can be a big help during the test administration in the field. The score sheet will contain all the necessary information for managing the tracks of the reference text test. Click here for a sample.
9. You are now ready to begin recording. On the score sheet, write down the track number for the beginning of the new reference text test.
10. Release the pause buttons on both machines and record all of the text tracks that lead to the first question. The PLAYER should be set to count down to the end of each track to help you know when to stop the machines and insert the new questions.
11. After recording the first section of text, place both machines on pause.
12. Scroll ahead on the PLAYER to the location of the first question that you have translated into the speech variety of the test point.
13. On the RECORDER, add a track mark to create a place for the question.
14. On the score sheet, write down the track of the question to be inserted.
15. Release the pause on both machines and record the question, leaving a second or two of silence at the end of the question track on the RECORDER.
16. Place both machines on pause and, turning back to the first machine, scroll back to the next portion of text that is to be recorded.
17. Repeat steps 9 - 16 until all of the reference text test is recorded.
18. Listen to the test to be sure that the track numbers are accurate and that there is enough silence at the end of each question. You do not need enough time for someone to give you the answer. You only need enough space to allow yourself time to pause the machine before the next portion of text begins.