https://surveywiki.info/api.php?action=feedcontributions&user=Katie+Crystal&feedformat=atomSurveyWiki - User contributions [en]2024-03-28T14:50:17ZUser contributionsMediaWiki 1.33.0https://surveywiki.info/index.php?title=Template:Recorded_Text_Testing&diff=648Template:Recorded Text Testing2011-04-19T16:15:57Z<p>Katie Crystal: Added link to RTT page</p>
<hr />
<div>{{Infobox<br />
|above = [[Recorded Text Testing]] <br />
|abovestyle= background:#DFB8DF;<br />
|header1 = [[Background Research]]<br />
|header2 = [[Intelligibility Interviews]]<br />
|header3 = [[Decision Guide for RTT Administration Protocols|Choose Kind of Test]]<br />
|header4 = [[Choosing RTT Test Points|Choose Test Points]]<br />
|header5 = [[Materials]]<br />
|header6 = [[Obtaining a Text|Obtain a Text]]<br />
|header7 = [[Develop Questions]]<br />
|header8 = [[Create Introduction Text]]<br />
|header9 = [[Assemble Pilot Test]]<br />
|header10 = [[Administer Pilot Test]]<br />
|header11 = [[Select Final Questions]]<br />
|header12 = [[Build Reference Tests]]<br />
|header13 = [[Develop Post-RTT Questions]]<br />
|header14 = [[Administer Text Test Set]]<br />
|header15 = [[Process Scores]]<br />
<br />
}}<br />
<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=SurveyWiki_Work&diff=647SurveyWiki Work2011-04-19T16:14:20Z<p>Katie Crystal: Added RTT suggestion</p>
<hr />
<div>'''Note for Editors: please add to these lists at your discretion. Please also remove items from these lists if content is added (by you or whoever) so that this is kept up to date.'''<br />
<br />
The following content needs to be added to SurveyWiki. <br />
<br />
<br />
==[[Recorded Text Tests]]==<br />
* I think all the basic information on methodology is on the site (at least what was in the "Procedures of Language Survey" course). I'm sure someone who has actually conducted an RTT before could fill in some of the gaps and tie things together better.<br />
* expand introduction page<br />
* see if we can use Radloff 1993 on "Develop Post-RTT Questions" page<br />
* make more links between the pages<br />
* perhaps put the charts from some of the PDF documents into the SurveyWiki pages themselves<br />
* add references (or links, if possible) to other resources on each topic<br />
* add info about RTT-Rs<br />
* add info about RTT-Qs<br />
* add info about RTT-RAs<br />
* '''update recording info for the digital age'''<br />
* upload O’Leary, C. F. (1994). The role of Recorded Text Tests in Intelligibility Assessment and Language Program Decisions. Notes on Literature in Use and Language Programs, 48-72<br />
* upload Landweer, M. L. (2009). Use of the Recorded Text Test in Papua New Guinea: Some Initial Thoughts. unpublished paper.<br />
<br />
==[[Sentence Repetition Tests]]==<br />
* add methodology<br />
<br />
==Survey mailing list==<br />
* Need to go through list from now backwards and strip out useful content and paste in wiki<br />
* if anyone's personal information is used, permission must be given<br />
* encourage the original writer to join SurveyWiki and add content themselves<br />
<br />
==[[Interviews]]==<br />
* non existant<br />
<br />
==[[Questionnaires]]==<br />
* incomplete<br />
<br />
==[[Matched Guise]]==<br />
* find out if is this used by survey teams?<br />
* add content/delete as necessary<br />
<br />
==[[Social Networks]]==<br />
* expand on practical use of SNA in surveying (if we are allowed to use content from Rueck 2006, there are some good matrices and diagrams we could post as examples of community-level networks for defining language/dialect areas and for language program planning)<br />
* if possible, add PDF of Rueck 2006<br />
<br />
==[[Language Vitality]]==<br />
* add some discussion of issues<br />
* link to EGIDS Malaysia/Indonesia page<br />
<br />
==[[Glossary]]==<br />
* make a redirect to [[Category:Help Glossary]] page<br />
* Africa Area Glossary<br />
** find out who maintains Africa area definitions<br />
** ask them to register and update glossary<br />
* if they won't, add in Africa area definitions<br />
<br />
==[[Statistics]]==<br />
* make a short list of terms/methods that are useful for survey<br />
* work through each of these terms/methods defining and describing them<br />
<br />
==[[Data Analysis]]==<br />
* add in a section on tips for analysis of qualitative data<br />
<br />
==[[Phonemic Analysis]]==<br />
* add in links to phonology editor/Praat and other sound tools<br />
* add in work with formants for vowel analysis<br />
<br />
==[[Word Lists]]==<br />
* add in detailed description of analysing word lists using WordSurv<br />
<br />
==[[Academic Writing]]==<br />
* SILESR report style template<br />
** get permission to add SILESR template doc<br />
** add SILESR template doc<br />
* write guidelines for report writing<br />
<br />
==[[Library]]==<br />
* create library page<br />
* populate it with resources organised by topic area and suborganised by author<br />
* upload files as necessary e.g. pdfs of articles and create necessary links</div>Katie Crystalhttps://surveywiki.info/index.php?title=Choosing_RTT_Test_Points&diff=646Choosing RTT Test Points2011-04-19T15:40:45Z<p>Katie Crystal: Added links</p>
<hr />
<div>{{Recorded Text Testing}}<br />
__NOTOC__<br />
In each survey, the choice of test points for administering [[Recorded Text Testing]] (RTT) will be influenced by the purpose of testing and the knowledge you have when planning the data collection trips. This knowledge can come from written sources, pre-trip [[interviews]], local community interviews, and [[word lists|word list]] analysis.<br />
<br />
People are different from place to place, even if they speak the same lect. So the places you choose will shape the data you collect - and possibly the conclusions you draw. When choosing test points, taking into account everything you know about the people and lects will make the test more useful.<br />
<br />
==Sources to Consult when Choosing RTT Test Points==<br />
<br />
The sources below are the usual contributors to the first list of places where you plan to develop tests and administer them. The list may change as testing progresses, because you should always review it when you receive more information. The number of places you choose will depend on the time you have for the survey and the number of lects you know about.<br />
<br />
===1. The purpose of intelligibility testing in the survey=== <br />
Is it to provide a broad description of intercomprehension? <br />
:*This purpose will usually require the greatest number of [[Build Reference Tests|reference tests]], because each lect will need to be tested with each of the others.<br />
Is it to identify the reference lect(s)?<br />
:*This purpose will require fewer reference tests, because [[Background Research|background research]] and [[interviews]] (both pre-trip and within the community) will be used to reveal the most likely candidates.<br />
Is it to identify the peripheral lects that understand a reference lect?<br />
:*In this case, it is clear that one lect will serve as a reference, but it is unclear which of the communities that speak related lects understand it.<br />
<br />
===2. [[Background Research|Pre-trip library research]] and [[Intelligibility Interviews|background interviews]]=== <br />
What have knowledgeable people outside the community said? <br />
:*Linguists, emigrant community members, NGO workers, and other people who have spent time in the community may have useful information on the names, locations, relationships, and comprehension of lects.<br />
What do earlier surveys or censuses report? <br />
:*These sources of information may be useful especially for the names, locations, and perhaps population figures for the lects. You may be able to identify the boundaries beyond which testing is unlikely to produce helpful results.<br />
What are the results of earlier intelligibility or comparative studies? <br />
:*These studies may reveal which communities ought to be included in testing solely on the basis of linguistic relatedness.<br />
Where are the centers of commerce or government? <br />
:*These locations are highly influential and are easy to identify from maps and informal questioning at government offices and in the local community. They should always be considered for reference tests.<br />
What geographic or political boundaries exist in the area? <br />
:*Such features can act as obstacles to contact and communication for closely related lects. Alternatively, by "fencing" communities in, they can act as catalysts for contact and communication for distantly related or unrelated lects. If boundaries seem to have an influence in the area, you should discover their effect on comprehension by including points within them in the tests.<br />
<br />
===3. Sociolinguistic community interviews conducted with speakers of the lects=== <br />
In what places do they say the lects are spoken? <br />
:*The answers to this question, together with the information gathered before the trip, will provide a thorough master list from which to select places in which to develop reference tests and administer them.<br />
Which lects or peoples do they associate or identify with? <br />
:*The answers to this question may help narrow the choices of reference tests.<br />
How similar do they say the lects are to one another? <br />
:*The answers to this question will group communities and lects together and may help reduce the number of test points by lowering the possibility of doing more testing than is really necessary.<br />
How well do the people say they understand the other lects? <br />
:*The answers to this question go beyond those of the one above and speak directly to the issue of comprehension. They may reveal that speakers of one lect understand another lect even though the two are not closely related, which is information that would not come out of a word list analysis.<br />
<br />
===4. Comparative analysis of word lists from all the lects=== <br />
If [[word lists]] have been collected in each of the lects, then hopefully there are results from community interviews to accompany them that will expose the sociolinguistic information discussed above. <br />
<br />
Which lects seem to group closely? <br />
:*Comprehension should always be tested among lects that share a large number of innovations. If a "parent" language still exists, it should be considered for a reference test.<br />
Which sets of lects are not grouped together? <br />
:*Unless sociolinguistic information indicates otherwise, it is probably not useful to do comprehension testing between groups.<br />
Which lects seem not to fall in any group? <br />
:*Use the results from community interviews to determine whether a lect is isolated or rather is associated with a lect in one of the groupings. If there is an association, then test speakers of the first lect with a reference test from the second lect.<br />
<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Talk:RTT_Administration_Protocols&diff=645Talk:RTT Administration Protocols2011-04-19T15:33:09Z<p>Katie Crystal: </p>
<hr />
<div>== Adding RTT menu? -- [[User:Katie Crystal|Katie Crystal]] 14:56, 19 April 2011 (UTC) ==<br />
<br />
It would be nice to add the [[Template:Recorded Text Testing|RTT menu]] so that it lines up with the introduction text but doesn't overlap with table.<br />
:[[User:Katie Crystal|Katie Crystal]] 14:56, 19 April 2011 (UTC)<br />
We should also add links to any pages that are created about these specific types of RTTs.<br />
:[[User:Katie Crystal|Katie Crystal]] 15:33, 19 April 2011 (UTC)</div>Katie Crystalhttps://surveywiki.info/index.php?title=Decision_Guide_for_RTT_Administration_Protocols&diff=644Decision Guide for RTT Administration Protocols2011-04-19T15:31:23Z<p>Katie Crystal: Added links</p>
<hr />
<div>{{Recorded Text Testing}}<br />
How do I know what kind of [[Recorded Text Testing]] (RTT) is right for my situation? Follow instructions and links in this decision guide to determine the best [[RTT Administration Protocols|administration protocols]] (developed by Douglas Boone, 2004).<br />
<br />
==Decision Guide for RTT Administration Protocols==<br />
For any research method, it is important to choose appropriate protocols to ensure the reliability of the test and to enhance its validity. When researching comprehension, the RTT is no exception. The following questions will help you choose the optimal protocols for a given situation.<br />
<br />
1. '''Is this survey limited by time or other resources?'''<br />
::Time and other resources are often insufficient for the full application of the Classical RTT protocols. The Rapid Appraisal RTT (RA RTT) is an attempt to apply the recorded text method under straitened circumstances. It builds upon the use of group [[Interviews|interviews]] and [[Observation|observation]] that characterizes Rapid Appraisal surveys (Bergman 1991). The RA RTT is generally suggested when the alternative is doing no comprehension testing at all.<br />
::*If the survey is limited, proceed to Step 3a in order to choose between [[RTT Administration Protocols|Options 1 and 2]].<br />
::*If it is possible to conduct in-depth testing, proceed to Step 2 to help you choose from among [[RTT Administration Protocols|Options 7-13]].<br />
<br />
2. '''Is the question and answer format natural?'''<br />
::Many people are unaccustomed to answering specific questions about something that has just been heard or read. Individuals or groups may feel so awkward about it that they score poorly or refuse the test. The retelling format avoids this problem. It is most commonly used in Rapid Appraisal surveys, but it may also be recommended for other surveys when it is more appropriate than the question and answer format.<br />
::*If the question and answer format is natural, proceed to Step 3c to continue with [[RTT Administration Protocols|Options 9-13]].<br />
::*If it is not natural, proceed to Step 3b so you can choose between [[RTT Administration Protocols|Options 7 and 8]].<br />
<br />
3. '''Is there a common language?'''<br />
::'''3a. RA RTT.''' If the researchers and the subjects share a common language, it is best to use this language to prompt the retelling ([[RTT Administration Protocols|Option 1]]). Otherwise, use an interpreter to prompt the retelling through the local language ([[RTT Administration Protocols|Option 2]]). In either case, the subjects may reply in whatever language they prefer.<br />
<br />
::'''3b. Quantitative scoring of Individual Retelling.''' If the researchers and the subjects share a common language, it is optimal to use this language to prompt the retelling ([[RTT Administration Protocols|Option 7]]). Otherwise, use an interpreter to prompt the retelling through the vernacular, or include the prompt in recorded instructions that accompany the text ([[RTT Administration Protocols|Option 8]]). In either case, the subject may reply in whatever language they prefer.<br />
<br />
::'''3c. Quantitative scoring of Individual Answers.''' If the researchers and the subjects share a common language, take advantage of that situation and use that language. Proceed to Step 4 to narrow the choice from among [[RTT Administration Protocols|Options 9-11]].<br />
<br />
::*If there is no common language, or if you cannot be assured that each subject knows it, translate the questions into the local language with the intention of eliciting responses in the vernacular. Proceed to Step 5 to help you choose between [[RTT Administration Protocols|Options 12 and 13]].<br />
<br />
4. '''Can the subjects be expected to be able to translate?'''<br />
<br />
::*If there is no doubt that subjects are bilingual to the point of being able to hear a question and answer it in the common language, then the best choice is not to use the vernacular at all ([[RTT Administration Protocols|Option 11]]).<br />
<br />
::*If there is any doubt, they should be left to respond in the language of their choice. If subjects are able to comprehend the common language, then the questions may be posed in the common language ([[RTT Administration Protocols|Option 9]]). Otherwise, it is more prudent to translate the questions into the vernacular ([[RTT Administration Protocols|Option 10]]).<br />
<br />
::Since you can expect the language of stimulus to bias the choice of language of response, the convenience of the subject should dictate the choice among these three options.<br />
<br />
5. '''Will people respond to a recorded stimulus?'''<br />
<br />
::*If people will probably realize that they are supposed to answer the recorded questions, then the Classical RTT protocols can be used ([[RTT Administration Protocols|Option 13]]).<br />
<br />
::*Some people have a hard time recognizing that the questions are separate from the story and that they are expected to answer them. You may get more reliable results if you have someone with you to ask the questions in the subjects' language. In this case, the questions are translated into the vernacular and recorded, but a local assistant repeats each recorded question for the convenience of the subject ([[RTT Administration Protocols|Option 12]]).<br />
<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=SurveyWiki_Work&diff=643SurveyWiki Work2011-04-19T15:18:45Z<p>Katie Crystal: Update on RTT pages</p>
<hr />
<div>'''Note for Editors: please add to these lists at your discretion. Please also remove items from these lists if content is added (by you or whoever) so that this is kept up to date.'''<br />
<br />
The following content needs to be added to SurveyWiki. <br />
<br />
<br />
==[[Recorded Text Tests]]==<br />
* I think all the basic information on methodology is on the site (at least what was in the "Procedures of Language Survey" course). I'm sure someone who has actually conducted an RTT before could fill in some of the gaps and tie things together better.<br />
* expand introduction page<br />
* see if we can use Radloff 1993 on "Develop Post-RTT Questions" page<br />
* make more links between the pages<br />
* perhaps put the charts from some of the PDF documents into the SurveyWiki pages themselves<br />
* add info about RTT-Rs<br />
* add info about RTT-Qs<br />
* add info about RTT-RAs<br />
* '''update recording info for the digital age'''<br />
* upload O’Leary, C. F. (1994). The role of Recorded Text Tests in Intelligibility Assessment and Language Program Decisions. Notes on Literature in Use and Language Programs, 48-72<br />
* upload Landweer, M. L. (2009). Use of the Recorded Text Test in Papua New Guinea: Some Initial Thoughts. unpublished paper.<br />
<br />
==[[Sentence Repetition Tests]]==<br />
* add methodology<br />
<br />
==Survey mailing list==<br />
* Need to go through list from now backwards and strip out useful content and paste in wiki<br />
* if anyone's personal information is used, permission must be given<br />
* encourage the original writer to join SurveyWiki and add content themselves<br />
<br />
==[[Interviews]]==<br />
* non existant<br />
<br />
==[[Questionnaires]]==<br />
* incomplete<br />
<br />
==[[Matched Guise]]==<br />
* find out if is this used by survey teams?<br />
* add content/delete as necessary<br />
<br />
==[[Social Networks]]==<br />
* expand on practical use of SNA in surveying (if we are allowed to use content from Rueck 2006, there are some good matrices and diagrams we could post as examples of community-level networks for defining language/dialect areas and for language program planning)<br />
* if possible, add PDF of Rueck 2006<br />
<br />
==[[Language Vitality]]==<br />
* add some discussion of issues<br />
* link to EGIDS Malaysia/Indonesia page<br />
<br />
==[[Glossary]]==<br />
* make a redirect to [[Category:Help Glossary]] page<br />
* Africa Area Glossary<br />
** find out who maintains Africa area definitions<br />
** ask them to register and update glossary<br />
* if they won't, add in Africa area definitions<br />
<br />
==[[Statistics]]==<br />
* make a short list of terms/methods that are useful for survey<br />
* work through each of these terms/methods defining and describing them<br />
<br />
==[[Data Analysis]]==<br />
* add in a section on tips for analysis of qualitative data<br />
<br />
==[[Phonemic Analysis]]==<br />
* add in links to phonology editor/Praat and other sound tools<br />
* add in work with formants for vowel analysis<br />
<br />
==[[Word Lists]]==<br />
* add in detailed description of analysing word lists using WordSurv<br />
<br />
==[[Academic Writing]]==<br />
* SILESR report style template<br />
** get permission to add SILESR template doc<br />
** add SILESR template doc<br />
* write guidelines for report writing<br />
<br />
==[[Library]]==<br />
* create library page<br />
* populate it with resources organised by topic area and suborganised by author<br />
* upload files as necessary e.g. pdfs of articles and create necessary links</div>Katie Crystalhttps://surveywiki.info/index.php?title=Process_Scores&diff=642Process Scores2011-04-19T15:13:04Z<p>Katie Crystal: Took out outline category</p>
<hr />
<div>{{Recorded Text Testing}}<br />
<br />
The analysis of the RTT scores will provide the basis for your estimate of how well the test subjects and the people they represent comprehend the test language. Ensuring that scoring has been applied consistently will make that analysis and estimate more accurate. When all of the scores from at least one location are available, the analysis process begins through the computation of figures such as the mean and standard deviation. These measures show you what the typical score was and how much the scores differed from one another.<br />
<br />
==Standard Deviation==<br />
The standard deviation gives an indication of how the scores vary for each group of subjects. For example, a high measure hints that the subjects did not understand the text equally. If comprehension is not close to equal, then it will be necessary to investigate the possible reasons for the differences.<br />
<br />
The standard deviation of the comprehension scores at a given location, for a given text, is a measure of the variability of the scores. If every subject has the same score, the standard deviation will be zero. The more variability in the scores, the greater the standard deviation will be. If this measure is higher than the value associated with your team's analysis protocol, it suggests that some subjects are considerably more familiar with the speech variety than others.<br />
<br />
If your intention in testing is to infer inherent intelligibility, your sampling protocol will have included safeguards to screen for language contact. Sometimes, however, these are insufficient.<br />
<br />
Generally, people with considerably more prior exposure to another speech variety than their fellows will score higher on a comprehension test. Occasionally, though, greater exposure will mean a heightened sensitivity to the features of a speech variety spoken by a despised people. This could produce LOWER comprehension scores as the subject feigns incomprehension to reflect an unwillingness to accept the speech variety in which the story is told.<br />
<br />
Any time that variability in test scores is observed, whether through a high standard deviation, widely differing first and third quartiles, or a visual inspection of the raw scores, one should look at the biographical data collected in the pre-RTT interview. (See the appendix in Radloff 1993 for sample pre-RTT questions.)<br />
<br />
Usually there are too few respondents to permit a conclusive statistical analysis of the influence of social variables (sex, age, travel patterns, habits of radio listening, etc.). However, when there are signs of variable comprehension, the researcher should look for striking patterns.<br />
<br />
Are there, for example, three scores in your sample that are at least 20 percentage points higher than the others? Were these attained by people who share one or more social characteristics? Did the people with unusual scores also give unusual answers to any of the post-RTT questions?<br />
<br />
Does the fact of a high standard deviation point instead to a few very low scores? If so, do these scores seem really to indicate that the subjects did not understand the story, or is there a more likely explanation – the subject's unwillingness to proceed with the test, a distraction or some other incapacity? And don't forget to look at individual answers to the post-RTT (and pre-RTT) questions.<br />
<br />
Sometimes the survey team suspects variability in the scores before the team leaves the area. If time allows, the team should consider testing more than the customary ten subjects, to ensure a sample of sufficient size to suggest the reasons why scores are not more uniform.<br />
<br />
In any case, one should not be content to report a high standard deviation – in such cases, one may want to report the distribution of scores, without, of course, compromising confidentiality. Whenever possible, there should be appropriate commentary on the sample, and even speculation on the reasons for the variation.<br />
<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Administer_Text_Test_Set&diff=641Administer Text Test Set2011-04-19T15:11:47Z<p>Katie Crystal: Number formatting</p>
<hr />
<div>{{Recorded Text Testing}}<br />
<br />
The administration of the final recorded text tests differs from that of the pilot tests mainly in the sampling procedure. The process also becomes more complicated and fatiguing when there is more than one test given. When you familiarize yourself with the steps in this process and practice them, you will be better prepared to administer an RTT properly and prevent or reduce errors.<br />
<br />
First, follow these [[Before Testing Participants|suggestions]] to prepare for testing participants.<br />
<br />
==How to Administer a Recorded Text Test==<br />
<br />
To use this procedural guide, you will need to have the complete list of questions and answers for the test written out, with the track numbers corresponding to each question's location written beside the question.<br />
<br />
#Follow the procedural guide for translating and recording the introduction text. Record the introduction text in the speech variety of the test site on the same MiniDisc (disc) as the hometown text for that variety. Note (in your data management notebook) the track number assigned to the introduction text. Only one track is needed for this text.<br />
#Find a helper who can translate your instructions to the test subjects from the LWC into the local language. Explain the steps involved in test administration so that the translator understands what you will be doing.<br />
#Ask for help in finding potential test subjects.<br />
#When a test subject arrives, sit him or her down in a comfortable setting and briefly explain what the test is like. You may need to explain how the voices were recorded and that they can be heard through the headphones.<br />
#Double check that he or she meets the basic qualifications needed to participate in this test.<br />
#Explain that the first speech he or she will hear is an announcement.<br />
#Play the introduction text and pause at the end of the text.<br />
#Check the introduction by having the first few test subjects explain it in their own words. If they don't understand it, then you need to write, record, or translate the introduction text again.<br />
#Cue the MD player to the beginning of the test text and put the list of questions and answers in front of you so that you can follow along. If you set your MD player's counter to count down the time remaining in each track, it will help you know when to pause the test.<br />
#Play the first section of text and the first question track and then pause the MD player.<br />
#If the test subject does not answer automatically after hearing the question, then ask him or her to answer the question. If the test subject seems unable to answer the question, you may play the question track again. Remember – do not repeat the text before the question!<br />
#If he or she is still unable to answer the question, mark the score sheet appropriately and continue the test.<br />
#If a test subject is missing nearly every question or is not answering them, it may be that he or she is having a hard time with the test methodology itself. Graciously dismiss him while thanking him for his time, and then call the next participant.<br />
#If the test subject answers the question correctly, write the appropriate score on the score sheet and continue the test.<br />
#Play the next text segment and question, and then pause the MD player.<br />
#Repeat steps 4 - 15 until the entire text is finished. Repeat the process for each of the additional subjects required for the test.<br />
<br />
==Rotate Texts==<br />
If a test subject has to listen to several tests, he or she may get tired toward the end and not listen as closely. Rotating the texts equalizes the effect fatigue has on the scores. Sometimes it is not possible or appropriate to give the test in private, and you must do it in a public place. Rotation also prevents bystanders from memorizing the answers that they think they should use, because they will be listening to a different test at that point. Click [[Media:Rotate.pdf|here]] for more details on rotating texts.<br />
<br />
<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Administer_Text_Test_Set&diff=640Administer Text Test Set2011-04-19T15:10:42Z<p>Katie Crystal: Took out outline category</p>
<hr />
<div>{{Recorded Text Testing}}<br />
<br />
The administration of the final recorded text tests differs from that of the pilot tests mainly in the sampling procedure. The process also becomes more complicated and fatiguing when there is more than one test given. When you familiarize yourself with the steps in this process and practice them, you will be better prepared to administer an RTT properly and prevent or reduce errors.<br />
<br />
First, follow these [[Before Testing Participants|suggestions]] to prepare for testing participants.<br />
<br />
==How to Administer a Recorded Text Test==<br />
<br />
To use this procedural guide, you will need to have the complete list of questions and answers for the test written out, with the track numbers corresponding to each question's location written beside the question.<br />
<br />
#Follow the procedural guide for translating and recording the introduction text. Record the introduction text in the speech variety of the test site on the same MiniDisc (disc) as the hometown text for that variety. Note (in your data management notebook) the track number assigned to the introduction text. Only one track is needed for this text.<br />
#Find a helper who can translate your instructions to the test subjects from the LWC into the local language. Explain the steps involved in test administration so that the translator understands what you will be doing.<br />
#Ask for help in finding potential test subjects.<br />
#When a test subject arrives, sit him or her down in a comfortable setting and briefly explain what the test is like. You may need to explain how the voices were recorded and that they can be heard through the headphones.<br />
#Double check that he or she meets the basic qualifications needed to participate in this test.<br />
#Explain that the first speech he or she will hear is an announcement.<br />
#Play the introduction text and pause at the end of the text.<br />
#Check the introduction by having the first few test subjects explain it in their own words. If they don't understand it, then you need to write, record, or translate the introduction text again.<br />
#Cue the MD player to the beginning of the test text and put the list of questions and answers in front of you so that you can follow along. If you set your MD player's counter to count down the time remaining in each track, it will help you know when to pause the test.<br />
#Play the first section of text and the first question track and then pause the MD player.<br />
#If the test subject does not answer automatically after hearing the question, then ask him or her to answer the question. If the test subject seems unable to answer the question, you may play the question track again. Remember – do not repeat the text before the question!<br />
#If he or she is still unable to answer the question, mark the score sheet appropriately and continue the test.<br />
#If a test subject is missing nearly every question or is not answering them, it may be that he or she is having a hard time with the test methodology itself. Graciously dismiss him while thanking him for his time, and then call the next participant.<br />
#If the test subject answers the question correctly, write the appropriate score on the score sheet and continue the test.<br />
#Play the next text segment and question, and then pause the MD player.<br />
#Repeat steps 4 - 14 until the entire text is finished. Repeat the process for each of the additional subjects required for the test.<br />
<br />
==Rotate Texts==<br />
If a test subject has to listen to several tests, he or she may get tired toward the end and not listen as closely. Rotating the texts equalizes the effect fatigue has on the scores. Sometimes it is not possible or appropriate to give the test in private, and you must do it in a public place. Rotation also prevents bystanders from memorizing the answers that they think they should use, because they will be listening to a different test at that point. Click [[Media:Rotate.pdf|here]] for more details on rotating texts.<br />
<br />
<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Build_Reference_Tests&diff=639Build Reference Tests2011-04-19T15:09:28Z<p>Katie Crystal: Removed outline category</p>
<hr />
<div>{{Recorded Text Testing}}<br />
<br />
The reference tests will provide the data you use to draw conclusions about comprehension. When each test is assembled properly, variation due to error in the test will be reduced. The more you can attribute variation in the results to differences in comprehension, the better your test will be.<br />
<br />
==Adjusting the Introduction==<br />
The basic information that a subject needs to understand about the RTT will probably not change from test to test, but it's likely that there are other facts peculiar to each survey. Since each people differs from the next, considering the context of each may improve the impact of the introduction text. Transforming technical or academic information into simple, informative content is difficult, especially when it will also be translated into another language. The introduction text is intended to inform and instruct the subjects - and put them more at ease. If it is translated inaccurately or recorded badly, then the subjects may have problems agreeing to and taking the test.<br />
<br />
==Building the Reference Test==<br />
<br />
#Set both MiniDisc machines side by side on a table and attach the patch cord to the "Line Out" jack on the PLAYER and the "Line In" on the RECORDER. Be sure that the RECORDER is set to record from the line and not the microphone.<br />
#Place the disc that contains the pilot text test with the 12 inserted questions in the PLAYER.<br />
#Cue the PLAYER to begin playing the first track of the text of this pilot test. Scroll to the beginning of the first track of the text and then pause the machine.<br />
#Set up the countdown feature on the PLAYER, so that you will see how much time is left on any given track. This will help you know when you need to be ready to pause the machine so that you can insert a question in the reference speech variety.<br />
#Place the target disc onto which you wish to record the new reference text test in the RECORDER.<br />
#Cue the machine to begin recording, but pause it in the recording mode.<br />
#In your bound notebook, set up a chart to show the contents of this new reference text test. Include the name of the area where the text test will be played (the name of the speech variety into which the questions have been translated). Include the name of the text itself, since you will likely be constructing more than one text test for that location. Write down each track number of the text test and include the content of each track. Your translation of the text is tied to the track numbers and if you mismanage those numbers, you may have to retranslate the text.<br />
#Some have found it helpful to design the score sheet at the same time the text test is built. A good score sheet can be a big help during the test administration in the field. The score sheet will contain all the necessary information for managing the tracks of the reference text test. Click [[Media:Rtt_score_chart.pdf|here]] for a sample.<br />
#You are now ready to begin recording. On the score sheet, write down the track number for the beginning of the new reference text test.<br />
#Release the pause buttons on both machines and record all of the text tracks that lead to the first question. The PLAYER should be set to count down to the end of each track to help you know when to stop the machines and insert the new questions.<br />
#After recording the first section of text, place both machines on pause.<br />
#Scroll ahead on the PLAYER to the location of the first question that you have translated into the speech variety of the test point.<br />
#On the RECORDER, add a track mark to create a place for the question.<br />
#On the score sheet, write down the track of the question to be inserted.<br />
#Release the pause on both machines and record the question, leaving a second or two of silence at the end of the question track on the RECORDER.<br />
#Place both machines on pause and, turning back to the first machine, scroll back to the next portion of text that is to be recorded.<br />
#Repeat steps 9 - 16 until all of the reference text test is recorded.<br />
#Listen to the test to be sure that the track numbers are accurate and that there is enough silence at the end of each question. You do not need enough time for someone to give you the answer. You only need enough space to allow yourself time to pause the machine before the next portion of text begins.<br />
<br />
<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Select_Final_Questions&diff=638Select Final Questions2011-04-19T15:08:52Z<p>Katie Crystal: Removed outline category</p>
<hr />
<div>{{Recorded Text Testing}}<br />
For one reason or another, not all of the questions created for the pilot test will be good ones or be understood clearly by the subjects. Taking these poor questions out of the test will leave only the ones that were understood and answered correctly by everyone, which will make the instrument more reliable. If the speakers of the test language were able to answer every question, then you can reasonably expect any person who understands the text to provide the correct answers also.<br />
<br />
Once standardization is applied to the pilot test scores, you need to mark for deletion the ones that will not be included in the final tests. When deleting question tracks from the pilot test to form the hometown test, it is possible to delete the wrong track or reassign track numbers through mismanagement. Practicing this procedure will help you keep all the right tracks and maintain a correct track table.<br />
<br />
Open the following document to learn [[Media:How to Evaluate and Choose Final Questions.pdf|how to evaluate and choose final questions]]. <br />
<br />
==Suggestions for removing and deleting questions==<br />
<br />
These are the questions that should be eliminated from the final test:<br />
<br />
:*Questions that were answered incorrectly<br />
<br />
:*Questions that were answered incompletely – if you find that some subjects provide less detail in their answers than other subjects, this may be an indication that the question was not understood equally or that hometown subjects have differing ideas about what to include in the answer. You do not want to risk either situation in the final test.<br />
<br />
:*Questions that were unclear to the pilot test subjects<br />
<br />
:*Questions that confused the subjects<br />
<br />
:*Questions that were answered only after a long delay – if subjects had to stop and think about the meaning of the question or the way in which they should respond, then you don't want to put final test subjects in the same position with potentially less comprehension.<br />
<br />
You also need to remove from the test:<br />
<br />
:*Questions that repeat a type of focus or semantic category (Casad 1974). Keep the question that creates the most even distribution of questions throughout the text.<br />
<br />
Do not eliminate:<br />
<br />
:*Any question answered correctly that has a unique focus on a particular component of the text.<br />
<br />
In short, you want the largest possible inventory of questions that have never been answered incorrectly by the hometown population and which each have a unique focus.<br />
<br />
When deleting questions tracks from the MiniDisc itself, remember to:<br />
<br />
:*Create a new track change table – the new track table will be the guide for the insertion of questions in the speech of each test point.<br />
<br />
:*Backup the disc that contains the whole pilot test – if you make a mistake, this backup could save you hours of work.<br />
<br />
:*Note the changes in tracks – remember that the numbers of the tracks after the one deleted will move up.<br />
<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Administer_Pilot_Test&diff=637Administer Pilot Test2011-04-19T15:05:53Z<p>Katie Crystal: Removed outline category</p>
<hr />
<div>{{Recorded Text Testing}}<br />
{{Administer the Pilot Test}}<br />
Administering the pilot text test uses many skills, each of which must be applied properly to ensure the validity and reliability of the test. Subjects must be screened in order to establish that they are, in fact, speakers of the language you are testing. Consistent and correct application of the test procedure will make certain that each subject has had an equal chance to hear the text and answer the questions. Clear scoring criteria and processes will guarantee that answers are marked appropriately and without variation.<br />
<br />
===[[Before Testing Participants]]===<br />
Click [[Before Testing Participants|here]] to learn about explaining your testing purposes and selecting test participants.<br />
<br />
===[[Testing Participants]]===<br />
[[Testing Participants|Here]] are some instructions and tips on the testing process.<br />
<br />
<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Create_Scoring_Chart&diff=636Create Scoring Chart2011-04-19T15:05:16Z<p>Katie Crystal: Took out outline category</p>
<hr />
<div>{{Recorded Text Testing}}<br />
{{Assemble Pilot Test}}<br />
Scoring charts will help you remember to gather background information from each test subject and write it down. During test administration, it is helpful to see the questions written and the corresponding question track numbers listed for each of the reference tests. Each sheet also provides space for writing the test subjects' responses, scores, and answers to the post-RTT questions.<br />
<br />
[[Media:rtt_score_chart.pdf|This document]] shows an example scoring chart.<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Use_Track_Change_Tables&diff=635Use Track Change Tables2011-04-19T15:04:25Z<p>Katie Crystal: Took out outline category</p>
<hr />
<div>{{Recorded Text Testing}}<br />
{{Assemble Pilot Test}}<br />
When you create a test disc to use in a village, the track assignments on that disc will be different from those on the source disc. A track change table is a good way to note the changes so that you know where to find each test and question set. You and other people will need to refer to and use the contents of each MiniDisc. Noting the changes consistently and accurately will ensure that no mistakes are made when transferring or playing tracks.<br />
<br />
Open [[Media:Using_Track_Change_Tables.pdf|this document]] for instructions and examples on using track change tables.<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Assemble_Pilot_Test&diff=634Assemble Pilot Test2011-04-19T15:03:40Z<p>Katie Crystal: Removed outline category</p>
<hr />
<div>{{Recorded Text Testing}}<br />
{{Assemble Pilot Test}}<br />
The pilot text test will determine which questions are the most effective and ought to be used in the final test. Care needs to be taken that each question is inserted after the section of text it refers to, so that the subjects do not become confused. Keeping a written record of track numbers and their contents will help you do this. Click on the headings below to learn the assembly process step by step.<br />
<br />
==[[Steps to Assemble Pilot Test]]==<br />
To preserve your original recording of the text, both as a backup and as the source for the LWC translation in your notebook, you will need a second MiniDisc (MD) recorder. This method for assembling the pilot test is the same as the cassette method. Follow these steps to assemble the test.<br />
<br />
==[[Use Track Change Tables]]==<br />
When you create a test disc to use in a village, the track assignments on that disc will be different from those on the source disc. A track change table is a good way to note the changes so that you know where to find each test and question set.<br />
<br />
==[[Create Scoring Chart]]==<br />
Scoring charts will help you remember to gather background information from each test subject and write it down. During test administration, it is helpful to see the questions written and the corresponding question track numbers listed for each of the reference tests. Each sheet also provides space for writing the test subjects' responses, scores, and answers to the post-RTT questions.<br />
<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Talk:Develop_Questions&diff=633Talk:Develop Questions2011-04-19T15:02:09Z<p>Katie Crystal: Menu -- ~~~~</p>
<hr />
<div>== Menu -- [[User:Katie Crystal|Katie Crystal]] 15:02, 19 April 2011 (UTC) ==<br />
<br />
It might be nice to add a menu for this one, but I'm not sure there are enough pages to justify it.<br />
:[[User:Katie Crystal|Katie Crystal]] 15:02, 19 April 2011 (UTC)</div>Katie Crystalhttps://surveywiki.info/index.php?title=Talk:RTT_Administration_Protocols&diff=632Talk:RTT Administration Protocols2011-04-19T14:56:52Z<p>Katie Crystal: Adding RTT menu? -- ~~~~</p>
<hr />
<div>== Adding RTT menu? -- [[User:Katie Crystal|Katie Crystal]] 14:56, 19 April 2011 (UTC) ==<br />
<br />
It would be nice to add the [[Template:Recorded Text Testing|RTT menu]] so that it lines up with the introduction text but doesn't overlap with table.<br />
:[[User:Katie Crystal|Katie Crystal]] 14:56, 19 April 2011 (UTC)</div>Katie Crystalhttps://surveywiki.info/index.php?title=Recorded_Text_Testing&diff=631Recorded Text Testing2011-04-19T14:50:22Z<p>Katie Crystal: Added short intro</p>
<hr />
<div>{{Recorded Text Testing}}<br />
Recorded Text Testing (RTT) can help surveyors understand how well people in certain areas understand particular speech varieties. The pages linked in the table to the right can help walk you through the RTT process.<br />
[[Category:Outline]]<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Category:Recorded_Text_Testing&diff=630Category:Recorded Text Testing2011-04-19T14:38:36Z<p>Katie Crystal: Deleted unnecessary text</p>
<hr />
<div></div>Katie Crystalhttps://surveywiki.info/index.php?title=Template:Recorded_Text_Testing&diff=629Template:Recorded Text Testing2011-04-19T14:36:59Z<p>Katie Crystal: Moved up materials link</p>
<hr />
<div>{{Infobox<br />
|above = Recorded Text Testing <br />
|abovestyle= background:#DFB8DF;<br />
|header1 = [[Background Research]]<br />
|header2 = [[Intelligibility Interviews]]<br />
|header3 = [[Decision Guide for RTT Administration Protocols|Choose Kind of Test]]<br />
|header4 = [[Choosing RTT Test Points|Choose Test Points]]<br />
|header5 = [[Materials]]<br />
|header6 = [[Obtaining a Text|Obtain a Text]]<br />
|header7 = [[Develop Questions]]<br />
|header8 = [[Create Introduction Text]]<br />
|header9 = [[Assemble Pilot Test]]<br />
|header10 = [[Administer Pilot Test]]<br />
|header11 = [[Select Final Questions]]<br />
|header12 = [[Build Reference Tests]]<br />
|header13 = [[Develop Post-RTT Questions]]<br />
|header14 = [[Administer Text Test Set]]<br />
|header15 = [[Process Scores]]<br />
<br />
}}<br />
<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Materials&diff=628Materials2011-04-19T14:34:13Z<p>Katie Crystal: Added content</p>
<hr />
<div>{{Recorded Text Testing}}<br />
<br />
The following list will help you assemble the equipment and other supplies you need to do recorded text testing using one team. To have two or more teams testing at the same time, you will need more equipment.<br />
*RTT pre- and post-test interview schedules<br />
*1 bound, ruled notebook<br />
*2 or more pens<br />
*2 MiniDisc recorders<br />
*2 patch cords, to connect the two recorders (one is a backup)<br />
*2 unidirectional digital microphones (one is a backup)<br />
*2-4 sets of headphones<br />
*1-2 Y-adapters, for using 2-4 headphones from one machine<br />
*10 or more 74-minute MiniDiscs<br />
*spare batteries<br />
Optional:<br />
*external speakers (not optional if doing group testing)<br />
*battery charger<br />
*12V inverter, to power charger<br />
*notebook computer<br />
*spreadsheet software (e.g. Microsoft Excel, Lotus 1-2-3)<br />
*printer<br />
*printer paper<br />
<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Process_Scores&diff=627Process Scores2011-04-19T14:27:14Z<p>Katie Crystal: Added content</p>
<hr />
<div>{{Recorded Text Testing}}<br />
<br />
The analysis of the RTT scores will provide the basis for your estimate of how well the test subjects and the people they represent comprehend the test language. Ensuring that scoring has been applied consistently will make that analysis and estimate more accurate. When all of the scores from at least one location are available, the analysis process begins through the computation of figures such as the mean and standard deviation. These measures show you what the typical score was and how much the scores differed from one another.<br />
<br />
==Standard Deviation==<br />
The standard deviation gives an indication of how the scores vary for each group of subjects. For example, a high measure hints that the subjects did not understand the text equally. If comprehension is not close to equal, then it will be necessary to investigate the possible reasons for the differences.<br />
<br />
The standard deviation of the comprehension scores at a given location, for a given text, is a measure of the variability of the scores. If every subject has the same score, the standard deviation will be zero. The more variability in the scores, the greater the standard deviation will be. If this measure is higher than the value associated with your team's analysis protocol, it suggests that some subjects are considerably more familiar with the speech variety than others.<br />
<br />
If your intention in testing is to infer inherent intelligibility, your sampling protocol will have included safeguards to screen for language contact. Sometimes, however, these are insufficient.<br />
<br />
Generally, people with considerably more prior exposure to another speech variety than their fellows will score higher on a comprehension test. Occasionally, though, greater exposure will mean a heightened sensitivity to the features of a speech variety spoken by a despised people. This could produce LOWER comprehension scores as the subject feigns incomprehension to reflect an unwillingness to accept the speech variety in which the story is told.<br />
<br />
Any time that variability in test scores is observed, whether through a high standard deviation, widely differing first and third quartiles, or a visual inspection of the raw scores, one should look at the biographical data collected in the pre-RTT interview. (See the appendix in Radloff 1993 for sample pre-RTT questions.)<br />
<br />
Usually there are too few respondents to permit a conclusive statistical analysis of the influence of social variables (sex, age, travel patterns, habits of radio listening, etc.). However, when there are signs of variable comprehension, the researcher should look for striking patterns.<br />
<br />
Are there, for example, three scores in your sample that are at least 20 percentage points higher than the others? Were these attained by people who share one or more social characteristics? Did the people with unusual scores also give unusual answers to any of the post-RTT questions?<br />
<br />
Does the fact of a high standard deviation point instead to a few very low scores? If so, do these scores seem really to indicate that the subjects did not understand the story, or is there a more likely explanation – the subject's unwillingness to proceed with the test, a distraction or some other incapacity? And don't forget to look at individual answers to the post-RTT (and pre-RTT) questions.<br />
<br />
Sometimes the survey team suspects variability in the scores before the team leaves the area. If time allows, the team should consider testing more than the customary ten subjects, to ensure a sample of sufficient size to suggest the reasons why scores are not more uniform.<br />
<br />
In any case, one should not be content to report a high standard deviation – in such cases, one may want to report the distribution of scores, without, of course, compromising confidentiality. Whenever possible, there should be appropriate commentary on the sample, and even speculation on the reasons for the variation.<br />
<br />
[[Category:Recorded Text Testing]]<br />
[[Category:Outline]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Talk:Administer_Text_Test_Set&diff=626Talk:Administer Text Test Set2011-04-19T14:18:51Z<p>Katie Crystal: </p>
<hr />
<div>== Plans -- [[User:Katie Crystal|Katie Crystal]] 21:26, 18 April 2011 (UTC) ==<br />
<br />
(To be continued...<br />
*add link to "before testing" page from pilot test<br />
*add rtt_administer.htm, which is different from the one on the pilot test page<br />
*add rtt_rotate.htm)<br />
<br />
:*Done! [[User:Katie Crystal|Katie Crystal]] 14:18, 19 April 2011 (UTC)</div>Katie Crystalhttps://surveywiki.info/index.php?title=File:Rotate.pdf&diff=625File:Rotate.pdf2011-04-19T14:16:15Z<p>Katie Crystal: Added RTT category</p>
<hr />
<div>[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=File:Rotate.pdf&diff=624File:Rotate.pdf2011-04-19T14:15:21Z<p>Katie Crystal: </p>
<hr />
<div></div>Katie Crystalhttps://surveywiki.info/index.php?title=Administer_Text_Test_Set&diff=623Administer Text Test Set2011-04-19T14:14:30Z<p>Katie Crystal: Added instructions</p>
<hr />
<div>{{Recorded Text Testing}}<br />
<br />
The administration of the final recorded text tests differs from that of the pilot tests mainly in the sampling procedure. The process also becomes more complicated and fatiguing when there is more than one test given. When you familiarize yourself with the steps in this process and practice them, you will be better prepared to administer an RTT properly and prevent or reduce errors.<br />
<br />
First, follow these [[Before Testing Participants|suggestions]] to prepare for testing participants.<br />
<br />
==How to Administer a Recorded Text Test==<br />
<br />
To use this procedural guide, you will need to have the complete list of questions and answers for the test written out, with the track numbers corresponding to each question's location written beside the question.<br />
<br />
#Follow the procedural guide for translating and recording the introduction text. Record the introduction text in the speech variety of the test site on the same MiniDisc (disc) as the hometown text for that variety. Note (in your data management notebook) the track number assigned to the introduction text. Only one track is needed for this text.<br />
#Find a helper who can translate your instructions to the test subjects from the LWC into the local language. Explain the steps involved in test administration so that the translator understands what you will be doing.<br />
#Ask for help in finding potential test subjects.<br />
#When a test subject arrives, sit him or her down in a comfortable setting and briefly explain what the test is like. You may need to explain how the voices were recorded and that they can be heard through the headphones.<br />
#Double check that he or she meets the basic qualifications needed to participate in this test.<br />
#Explain that the first speech he or she will hear is an announcement.<br />
#Play the introduction text and pause at the end of the text.<br />
#Check the introduction by having the first few test subjects explain it in their own words. If they don't understand it, then you need to write, record, or translate the introduction text again.<br />
#Cue the MD player to the beginning of the test text and put the list of questions and answers in front of you so that you can follow along. If you set your MD player's counter to count down the time remaining in each track, it will help you know when to pause the test.<br />
#Play the first section of text and the first question track and then pause the MD player.<br />
#If the test subject does not answer automatically after hearing the question, then ask him or her to answer the question. If the test subject seems unable to answer the question, you may play the question track again. Remember – do not repeat the text before the question!<br />
#If he or she is still unable to answer the question, mark the score sheet appropriately and continue the test.<br />
#If a test subject is missing nearly every question or is not answering them, it may be that he or she is having a hard time with the test methodology itself. Graciously dismiss him while thanking him for his time, and then call the next participant.<br />
#If the test subject answers the question correctly, write the appropriate score on the score sheet and continue the test.<br />
#Play the next text segment and question, and then pause the MD player.<br />
#Repeat steps 4 - 14 until the entire text is finished. Repeat the process for each of the additional subjects required for the test.<br />
<br />
==Rotate Texts==<br />
If a test subject has to listen to several tests, he or she may get tired toward the end and not listen as closely. Rotating the texts equalizes the effect fatigue has on the scores. Sometimes it is not possible or appropriate to give the test in private, and you must do it in a public place. Rotation also prevents bystanders from memorizing the answers that they think they should use, because they will be listening to a different test at that point. Click [[Media:Rotate.pdf|here]] for more details on rotating texts.<br />
<br />
<br />
[[Category:Recorded Text Testing]]<br />
[[Category:Outline]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Talk:Administer_Text_Test_Set&diff=622Talk:Administer Text Test Set2011-04-18T21:26:29Z<p>Katie Crystal: Plans -- ~~~~</p>
<hr />
<div>== Plans -- [[User:Katie Crystal|Katie Crystal]] 21:26, 18 April 2011 (UTC) ==<br />
<br />
(To be continued...<br />
*add link to "before testing" page from pilot test<br />
*add rtt_administer.htm, which is different from the one on the pilot test page<br />
*add rtt_rotate.htm</div>Katie Crystalhttps://surveywiki.info/index.php?title=Administer_Text_Test_Set&diff=621Administer Text Test Set2011-04-18T21:22:55Z<p>Katie Crystal: Added first part of page (see discussion for next steps)</p>
<hr />
<div>{{Recorded Text Testing}}<br />
<br />
The administration of the final recorded text tests differs from that of the pilot tests mainly in the sampling procedure. The process also becomes more complicated and fatiguing when there is more than one test given. When you familiarize yourself with the steps in this process and practice them, you will be better prepared to administer an RTT properly and prevent or reduce errors.<br />
<br />
Rotate Texts<br />
If a test subject has to listen to several tests, he or she may get tired toward the end and not listen as closely. Rotating the texts equalizes the effect fatigue has on the scores. Sometimes it is not possible or appropriate to give the test in private, and you must do it in a public place. Rotation also prevents bystanders from memorizing the answers that they think they should use, because they will be listening to a different test at that point.<br />
<br />
<br />
[[Category:Recorded Text Testing]]<br />
[[Category:Outline]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Develop_Post-RTT_Questions&diff=620Develop Post-RTT Questions2011-04-18T21:17:05Z<p>Katie Crystal: Intro</p>
<hr />
<div>{{Recorded Text Testing}}<br />
<br />
Asking test subjects questions about their understanding and opinion of the test language provides information that is useful when interpreting RTT data. It may confirm the resulting estimate of comprehension or help explain why the expected result did not occur. Such questions may also reveal attitudes about the test language that affect decisions about language development.<br />
<br />
[[Category:Recorded Text Testing]]<br />
[[Category:Outline]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Develop_Post-RTT_Questions&diff=619Develop Post-RTT Questions2011-04-18T21:14:48Z<p>Katie Crystal: Undo revision 617 by Katie Crystal (Talk)</p>
<hr />
<div>{{Recorded Text Testing}}<br />
<br />
Content coming...<br />
<br />
[[Category:Recorded Text Testing]]<br />
[[Category:Outline]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Talk:Develop_Post-RTT_Questions&diff=618Talk:Develop Post-RTT Questions2011-04-18T21:14:20Z<p>Katie Crystal: Content -- ~~~~</p>
<hr />
<div>== Content -- [[User:Katie Crystal|Katie Crystal]] 21:14, 18 April 2011 (UTC) ==<br />
<br />
Radloff 1993 (Post-RTT Questions for Interpreting RTT Scores) belongs here but I don't know yet if we have permission to post it. If not, maybe we can use a properly cited summary or reference.</div>Katie Crystalhttps://surveywiki.info/index.php?title=Develop_Post-RTT_Questions&diff=617Develop Post-RTT Questions2011-04-18T20:53:25Z<p>Katie Crystal: Plans</p>
<hr />
<div>{{Recorded Text Testing}}<br />
<br />
Radloff 1993 (''Post-RTT Questions for Interpreting RTT Scores'') belongs here but I don't know yet if we have permission to post it. If not, maybe we can use a properly cited summary or reference.<br />
<br />
[[Category:Recorded Text Testing]]<br />
[[Category:Outline]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Build_Reference_Tests&diff=616Build Reference Tests2011-04-18T20:31:05Z<p>Katie Crystal: Added content</p>
<hr />
<div>{{Recorded Text Testing}}<br />
<br />
The reference tests will provide the data you use to draw conclusions about comprehension. When each test is assembled properly, variation due to error in the test will be reduced. The more you can attribute variation in the results to differences in comprehension, the better your test will be.<br />
<br />
==Adjusting the Introduction==<br />
The basic information that a subject needs to understand about the RTT will probably not change from test to test, but it's likely that there are other facts peculiar to each survey. Since each people differs from the next, considering the context of each may improve the impact of the introduction text. Transforming technical or academic information into simple, informative content is difficult, especially when it will also be translated into another language. The introduction text is intended to inform and instruct the subjects - and put them more at ease. If it is translated inaccurately or recorded badly, then the subjects may have problems agreeing to and taking the test.<br />
<br />
==Building the Reference Test==<br />
<br />
#Set both MiniDisc machines side by side on a table and attach the patch cord to the "Line Out" jack on the PLAYER and the "Line In" on the RECORDER. Be sure that the RECORDER is set to record from the line and not the microphone.<br />
#Place the disc that contains the pilot text test with the 12 inserted questions in the PLAYER.<br />
#Cue the PLAYER to begin playing the first track of the text of this pilot test. Scroll to the beginning of the first track of the text and then pause the machine.<br />
#Set up the countdown feature on the PLAYER, so that you will see how much time is left on any given track. This will help you know when you need to be ready to pause the machine so that you can insert a question in the reference speech variety.<br />
#Place the target disc onto which you wish to record the new reference text test in the RECORDER.<br />
#Cue the machine to begin recording, but pause it in the recording mode.<br />
#In your bound notebook, set up a chart to show the contents of this new reference text test. Include the name of the area where the text test will be played (the name of the speech variety into which the questions have been translated). Include the name of the text itself, since you will likely be constructing more than one text test for that location. Write down each track number of the text test and include the content of each track. Your translation of the text is tied to the track numbers and if you mismanage those numbers, you may have to retranslate the text.<br />
#Some have found it helpful to design the score sheet at the same time the text test is built. A good score sheet can be a big help during the test administration in the field. The score sheet will contain all the necessary information for managing the tracks of the reference text test. Click [[Media:Rtt_score_chart.pdf|here]] for a sample.<br />
#You are now ready to begin recording. On the score sheet, write down the track number for the beginning of the new reference text test.<br />
#Release the pause buttons on both machines and record all of the text tracks that lead to the first question. The PLAYER should be set to count down to the end of each track to help you know when to stop the machines and insert the new questions.<br />
#After recording the first section of text, place both machines on pause.<br />
#Scroll ahead on the PLAYER to the location of the first question that you have translated into the speech variety of the test point.<br />
#On the RECORDER, add a track mark to create a place for the question.<br />
#On the score sheet, write down the track of the question to be inserted.<br />
#Release the pause on both machines and record the question, leaving a second or two of silence at the end of the question track on the RECORDER.<br />
#Place both machines on pause and, turning back to the first machine, scroll back to the next portion of text that is to be recorded.<br />
#Repeat steps 9 - 16 until all of the reference text test is recorded.<br />
#Listen to the test to be sure that the track numbers are accurate and that there is enough silence at the end of each question. You do not need enough time for someone to give you the answer. You only need enough space to allow yourself time to pause the machine before the next portion of text begins.<br />
<br />
<br />
[[Category:Recorded Text Testing]]<br />
[[Category:Outline]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Select_Final_Questions&diff=615Select Final Questions2011-04-18T20:06:13Z<p>Katie Crystal: Formatting</p>
<hr />
<div>{{Recorded Text Testing}}<br />
For one reason or another, not all of the questions created for the pilot test will be good ones or be understood clearly by the subjects. Taking these poor questions out of the test will leave only the ones that were understood and answered correctly by everyone, which will make the instrument more reliable. If the speakers of the test language were able to answer every question, then you can reasonably expect any person who understands the text to provide the correct answers also.<br />
<br />
Once standardization is applied to the pilot test scores, you need to mark for deletion the ones that will not be included in the final tests. When deleting question tracks from the pilot test to form the hometown test, it is possible to delete the wrong track or reassign track numbers through mismanagement. Practicing this procedure will help you keep all the right tracks and maintain a correct track table.<br />
<br />
Open the following document to learn [[Media:How to Evaluate and Choose Final Questions.pdf|how to evaluate and choose final questions]]. <br />
<br />
==Suggestions for removing and deleting questions==<br />
<br />
These are the questions that should be eliminated from the final test:<br />
<br />
:*Questions that were answered incorrectly<br />
<br />
:*Questions that were answered incompletely – if you find that some subjects provide less detail in their answers than other subjects, this may be an indication that the question was not understood equally or that hometown subjects have differing ideas about what to include in the answer. You do not want to risk either situation in the final test.<br />
<br />
:*Questions that were unclear to the pilot test subjects<br />
<br />
:*Questions that confused the subjects<br />
<br />
:*Questions that were answered only after a long delay – if subjects had to stop and think about the meaning of the question or the way in which they should respond, then you don't want to put final test subjects in the same position with potentially less comprehension.<br />
<br />
You also need to remove from the test:<br />
<br />
:*Questions that repeat a type of focus or semantic category (Casad 1974). Keep the question that creates the most even distribution of questions throughout the text.<br />
<br />
Do not eliminate:<br />
<br />
:*Any question answered correctly that has a unique focus on a particular component of the text.<br />
<br />
In short, you want the largest possible inventory of questions that have never been answered incorrectly by the hometown population and which each have a unique focus.<br />
<br />
When deleting questions tracks from the MiniDisc itself, remember to:<br />
<br />
:*Create a new track change table – the new track table will be the guide for the insertion of questions in the speech of each test point.<br />
<br />
:*Backup the disc that contains the whole pilot test – if you make a mistake, this backup could save you hours of work.<br />
<br />
:*Note the changes in tracks – remember that the numbers of the tracks after the one deleted will move up.<br />
<br />
[[Category:Recorded Text Testing]]<br />
[[Category:Outline]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=File:How_to_Evaluate_and_Choose_Final_Questions.pdf&diff=614File:How to Evaluate and Choose Final Questions.pdf2011-04-18T20:04:53Z<p>Katie Crystal: Add RTT category</p>
<hr />
<div>[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=File:How_to_Evaluate_and_Choose_Final_Questions.pdf&diff=613File:How to Evaluate and Choose Final Questions.pdf2011-04-18T20:04:17Z<p>Katie Crystal: </p>
<hr />
<div></div>Katie Crystalhttps://surveywiki.info/index.php?title=Select_Final_Questions&diff=612Select Final Questions2011-04-18T20:03:59Z<p>Katie Crystal: Fixed link</p>
<hr />
<div>{{Recorded Text Testing}}<br />
For one reason or another, not all of the questions created for the pilot test will be good ones or be understood clearly by the subjects. Taking these poor questions out of the test will leave only the ones that were understood and answered correctly by everyone, which will make the instrument more reliable. If the speakers of the test language were able to answer every question, then you can reasonably expect any person who understands the text to provide the correct answers also.<br />
<br />
Once standardization is applied to the pilot test scores, you need to mark for deletion the ones that will not be included in the final tests. When deleting question tracks from the pilot test to form the hometown test, it is possible to delete the wrong track or reassign track numbers through mismanagement. Practicing this procedure will help you keep all the right tracks and maintain a correct track table.<br />
<br />
Open the following document to learn [[Media:How to Evaluate and Choose Final Questions.pdf|how to evaluate and choose final questions]]. Then use the following suggestions for removing and deleting questions.<br />
<br />
These are the questions that should be eliminated from the final test:<br />
<br />
:*Questions that were answered incorrectly<br />
<br />
:*Questions that were answered incompletely – if you find that some subjects provide less detail in their answers than other subjects, this may be an indication that the question was not understood equally or that hometown subjects have differing ideas about what to include in the answer. You do not want to risk either situation in the final test.<br />
<br />
:*Questions that were unclear to the pilot test subjects<br />
<br />
:*Questions that confused the subjects<br />
<br />
:*Questions that were answered only after a long delay – if subjects had to stop and think about the meaning of the question or the way in which they should respond, then you don't want to put final test subjects in the same position with potentially less comprehension.<br />
<br />
You also need to remove from the test:<br />
<br />
:*Questions that repeat a type of focus or semantic category (Casad 1974). Keep the question that creates the most even distribution of questions throughout the text.<br />
<br />
Do not eliminate:<br />
<br />
:*Any question answered correctly that has a unique focus on a particular component of the text.<br />
<br />
In short, you want the largest possible inventory of questions that have never been answered incorrectly by the hometown population and which each have a unique focus.<br />
<br />
When deleting questions tracks from the MiniDisc itself, remember to:<br />
<br />
:*Create a new track change table – the new track table will be the guide for the insertion of questions in the speech of each test point.<br />
<br />
:*Backup the disc that contains the whole pilot test – if you make a mistake, this backup could save you hours of work.<br />
<br />
:*Note the changes in tracks – remember that the numbers of the tracks after the one deleted will move up.<br />
<br />
[[Category:Recorded Text Testing]]<br />
[[Category:Outline]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Select_Final_Questions&diff=611Select Final Questions2011-04-18T19:44:24Z<p>Katie Crystal: Added content</p>
<hr />
<div>{{Recorded Text Testing}}<br />
For one reason or another, not all of the questions created for the pilot test will be good ones or be understood clearly by the subjects. Taking these poor questions out of the test will leave only the ones that were understood and answered correctly by everyone, which will make the instrument more reliable. If the speakers of the test language were able to answer every question, then you can reasonably expect any person who understands the text to provide the correct answers also.<br />
<br />
Once standardization is applied to the pilot test scores, you need to mark for deletion the ones that will not be included in the final tests. When deleting question tracks from the pilot test to form the hometown test, it is possible to delete the wrong track or reassign track numbers through mismanagement. Practicing this procedure will help you keep all the right tracks and maintain a correct track table.<br />
<br />
Open the following document to learn [[How to Evaluate and Choose Final Questions.pdf|how to evaluate and choose final questions]]. Then use the following suggestions for removing and deleting questions.<br />
<br />
These are the questions that should be eliminated from the final test:<br />
<br />
:*Questions that were answered incorrectly<br />
<br />
:*Questions that were answered incompletely – if you find that some subjects provide less detail in their answers than other subjects, this may be an indication that the question was not understood equally or that hometown subjects have differing ideas about what to include in the answer. You do not want to risk either situation in the final test.<br />
<br />
:*Questions that were unclear to the pilot test subjects<br />
<br />
:*Questions that confused the subjects<br />
<br />
:*Questions that were answered only after a long delay – if subjects had to stop and think about the meaning of the question or the way in which they should respond, then you don't want to put final test subjects in the same position with potentially less comprehension.<br />
<br />
You also need to remove from the test:<br />
<br />
:*Questions that repeat a type of focus or semantic category (Casad 1974). Keep the question that creates the most even distribution of questions throughout the text.<br />
<br />
Do not eliminate:<br />
<br />
:*Any question answered correctly that has a unique focus on a particular component of the text.<br />
<br />
In short, you want the largest possible inventory of questions that have never been answered incorrectly by the hometown population and which each have a unique focus.<br />
<br />
When deleting questions tracks from the MiniDisc itself, remember to:<br />
<br />
:*Create a new track change table – the new track table will be the guide for the insertion of questions in the speech of each test point.<br />
<br />
:*Backup the disc that contains the whole pilot test – if you make a mistake, this backup could save you hours of work.<br />
<br />
:*Note the changes in tracks – remember that the numbers of the tracks after the one deleted will move up.<br />
<br />
[[Category:Recorded Text Testing]]<br />
[[Category:Outline]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Before_Testing_Participants&diff=610Before Testing Participants2011-04-18T19:10:33Z<p>Katie Crystal: fixed link</p>
<hr />
<div>{{Recorded Text Testing}}<br />
{{Administer the Pilot Test}}<br />
===Before Beginning the Test===<br />
When talking to local authorities to obtain permission to do research, or when explaining to subjects what they'll be doing, you often must describe the RTT procedure. Usually you do this in another language. If you practice talking about the test in advance, you will be more confident of doing it with test subjects and officials.<br />
<br />
===[[Selecting Test Participants]]===<br />
Allowing just anyone to participate in the formation or testing of an RTT can reduce the validity of the test. When you screen the people who will be pilot testing an RTT, you are making sure that they really know the test language and that their speech hasn't been influenced much by other, possibly related languages. Screening subjects for the final test makes sure that they're really part of the group you want to test and that they're not familiar with the test language. Click [[Selecting Test Participants|here]] for some suggestions for selecting RTT participants. <br />
<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Testing_Participants&diff=609Testing Participants2011-04-18T19:06:36Z<p>Katie Crystal: Added content</p>
<hr />
<div>{{Recorded Text Testing}}<br />
{{Administer the Pilot Test}}<br />
==How to Administer a Pilot Text Test==<br />
To use this procedural guide, you will need to have the complete list of questions and answers for the pilot test written out, with the track numbers corresponding to each question's location written beside the question.<br />
<br />
#Follow the procedural guides for translating and recording the [[Before Testing Participants|introduction text]]. Record the introduction text in the speech variety of the pilot test on the same MiniDisc (disc) as the 30-question pilot test. Note (in your data management notebook) the track number assigned to the introduction text. Only one track is needed for this text.<br />
#Find a helper who can translate your instructions to the test subjects from the LWC into the local language. Explain the steps involved in test administration so that the translator understands what you will be doing.<br />
#Ask for help in finding potential test subjects.<br />
#When a test subject arrives, sit him or her down in a comfortable setting and briefly explain what the test is like. You may need to explain how the voices were recorded and that they can be heard through the headphones.<br />
#Double check that he or she meets the [[Before Testing Participants|basic qualifications]] needed to participate in this pilot test.<br />
#Explain that the first speech he or she will hear is an announcement.<br />
#Play the introduction text and pause at the end of the text.<br />
#Check the introduction by having the first few test subjects explain it in their own words. If they don't understand it, then you need to write, record, or translate the introduction text again.<br />
#Cue the MD player to the beginning of the test text and put the list of questions and answers in front of you so that you can follow along. If you set your MD player's counter to count down the time remaining in each track, it will help you know when to pause the test.<br />
#Play the first section of text and the first question track and then pause the MD player.<br />
#If the test subject does not answer automatically after hearing the question, then ask him or her to answer the question. If the test subject seems unable to answer the question, you may play the question track again. Remember - do not repeat the text before the question and give them an unfair chance to hear it again!<br />
#If he or she is still unable to answer the question, place a mark next to its track number. This question may be skipped with the subjects that follow, because no question that was missed in the pilot test may be used in the final test. When such questions come the next time, simply "jump" or "skip" to the next track.<br />
#If a pilot test subject is missing nearly every question or is not answering them, it may be that he or she is having a hard time with the test methodology itself. Graciously dismiss him while thanking him for his time, and then call the next participant. In this case, you should continue to play the questions that the dismissed subject missed.<br />
#If the test subject answers the question correctly, there is no need to make any mark.<br />
#Play the next text segment and question, and then pause the MD player.<br />
#Repeat steps 4 - 14 until the entire text is finished. Repeat the process for each of the additional subjects required for the pilot test.<br />
<br />
==Tips from the Field==<br />
*When you play the test without any disruptions or distractions due to operator error, you keep the subject's focus on listening to the text and questions.<br />
*You are going to use the pilot test subjects' answers to choose the questions for the final test. Writing down their responses correctly helps ensure that good questions aren't discarded or bad questions retained.<br />
*Responses can be different from what you expect but still be correct. To score the pilot or final test accurately, you need to be able to judge the answers quickly and fairly.<br />
*Errors in forming the test or in analyzing the final results can occur if the correct notation is not used for a response, even when that response has been evaluated correctly. Practicing your local method of marking scores will help you develop consistency and familiarity.<br />
*If more than one person administers a RTT, it is possible for similar answers to be scored differently. Making the scores the same when possible will allow you to keep questions that otherwise might need to be deleted or disregarded.<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Selecting_Test_Participants&diff=608Selecting Test Participants2011-04-18T18:48:04Z<p>Katie Crystal: Added content</p>
<hr />
<div>{{Recorded Text Testing}}<br />
{{Administer the Pilot Test}}<br />
There are some differences between the selection criteria for participants in the pilot test and those in the final test. The suggestions below are not exhaustive, and the rationale for each may not be the only one.<br />
<br />
In both cases, you will need:<br />
<br />
:*'''At least ten (10) people''' – this number is the minimum necessary to give your results meaning, that is, to establish statistical significance. This number may be reduced for the pilot test if ten people are not available.<br />
<br />
:*'''Both men and women''' – this practice ensures that if a sex-based, in-group variety has inadvertently been used for the text, the results will show it. Unless you have valid reasons for focusing on one sex, there should be equal numbers of both.<br />
<br />
When selecting subjects for the pilot test, look for people who:<br />
<br />
:*'''Speak, as their first language, the speech variety of the text used in the pilot test''' – this restriction is a general confirmation that the person can verify the text and questions.<br />
:*'''Were born in the local community''' – being native to the area means that a person will be a good judge of a text that is supposed to have come from that area.<br />
:*'''Lived nearly their entire lives in the local community''' – this condition limits the amount of exposure to other varieties the subjects have had through travel.<br />
<br />
When selecting subjects for the final tests, look for people who:<br />
<br />
:*'''Have not lived in other areas for any extended time''' – a long time away may have changed the person's ability in the test point language and therefore his or her ability to understand related varieties.<br />
:*'''Have had little or no contact with the speech varieties you will be testing at that particular test point''' – such contact gives a person an undesirable "advantage" in being able to understand the texts.<br />
<br />
You may also need:<br />
<br />
:*'''People chosen at random''' – usually, statistical random sampling is only needed when a RTT is used to test bilingualism.<br />
:*Other criteria that your entity or the team leader has deemed necessary.<br />
<br />
A bit of additional advice that was obtained the hard way:<br />
<br />
:*'''Limit the number of texts''' – if you have several texts to play in one place, there may be too many for each person to sit through. Choose more subjects than the usual ten so you can spread out the tests. For example, instead of playing six stories to each of ten people, choose another five people and have each of the fifteen listen to four stories.<br />
:*'''Investigate gender roles''' – there may be a correlation between gender and exposure to other lects. For example, if the women in a group do not typically travel, then they will tend to have less exposure to and understanding of other lects. The opposite will be true of the men. On the other hand, if the sellers in area markets are mostly women, then they will have more exposure.<br />
:*'''Check the sociological factors that may influence sampling.''' For example, if you find that a group practices clan exogamy, then you would know not to test the married women, since they come from another area. (However, such women may provide other useful information related to language use, bidialectalism, perceived inherent intelligibility, etc.)<br />
:*'''Question the frequency and duration of travel and be careful about how the terms you use are defined locally.''' In Tanzania, people reported that they had always "lived" in their hometown, even though they had "traveled" to another place and "stayed" there for five years!<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Before_Testing_Participants&diff=607Before Testing Participants2011-04-18T16:53:26Z<p>Katie Crystal: Added content</p>
<hr />
<div>{{Recorded Text Testing}}<br />
{{Administer the Pilot Test}}<br />
===Before Beginning the Test===<br />
When talking to local authorities to obtain permission to do research, or when explaining to subjects what they'll be doing, you often must describe the RTT procedure. Usually you do this in another language. If you practice talking about the test in advance, you will be more confident of doing it with test subjects and officials.<br />
<br />
===[[Selecting Test Participants]]===<br />
Allowing just anyone to participate in the formation or testing of an RTT can reduce the validity of the test. When you screen the people who will be pilot testing an RTT, you are making sure that they really know the test language and that their speech hasn't been influenced much by other, possibly related languages. Screening subjects for the final test makes sure that they're really part of the group you want to test and that they're not familiar with the test language. Click [[Selecting RTT Participants|here]] for some suggestions for selecting RTT participants. <br />
<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Template:Administer_the_Pilot_Test&diff=606Template:Administer the Pilot Test2011-04-18T16:52:19Z<p>Katie Crystal: Trying to get link to work to template</p>
<hr />
<div>{{Infobox<br />
|above = Administer the Pilot Test<br />
|abovestyle= background:#FFDEAD;<br />
|header1 = [[Before Testing Participants]]<br />
|header2 = [[Testing Participants]]<br />
}}<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Template:Administer_the_Pilot_Test&diff=605Template:Administer the Pilot Test2011-04-18T16:44:26Z<p>Katie Crystal: Created template</p>
<hr />
<div>{{Infobox<br />
|above = Administer Pilot Test<br />
|abovestyle= background:#FFDEAD;<br />
|header1 = [[Before Testing Participants]]<br />
|header2 = [[Testing Participants]]<br />
}}<br />
[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Administer_Pilot_Test&diff=604Administer Pilot Test2011-04-18T16:30:27Z<p>Katie Crystal: Added content</p>
<hr />
<div>{{Recorded Text Testing}}<br />
{{Administer the Pilot Test}}<br />
Administering the pilot text test uses many skills, each of which must be applied properly to ensure the validity and reliability of the test. Subjects must be screened in order to establish that they are, in fact, speakers of the language you are testing. Consistent and correct application of the test procedure will make certain that each subject has had an equal chance to hear the text and answer the questions. Clear scoring criteria and processes will guarantee that answers are marked appropriately and without variation.<br />
<br />
===[[Before Testing Participants]]===<br />
Click [[Before Testing Participants|here]] to learn about explaining your testing purposes and selecting test participants.<br />
<br />
===[[Testing Participants]]===<br />
[[Testing Participants|Here]] are some instructions and tips on the testing process.<br />
<br />
<br />
[[Category:Recorded Text Testing]]<br />
[[Category:Outline]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=Talk:Create_Scoring_Chart&diff=603Talk:Create Scoring Chart2011-04-18T15:35:53Z<p>Katie Crystal: Charts -- ~~~~</p>
<hr />
<div>== Charts -- [[User:Katie Crystal|Katie Crystal]] 15:35, 18 April 2011 (UTC) ==<br />
<br />
I'm not sure whether it would be easier for people to save this document to use or to see it laid out in tables on the website itself. It was faster to just upload the document but when we start running out of things to do on the site (ha, ha) we could maybe come back and enter the charts.</div>Katie Crystalhttps://surveywiki.info/index.php?title=Talk:Use_Track_Change_Tables&diff=602Talk:Use Track Change Tables2011-04-18T15:35:21Z<p>Katie Crystal: Charts -- ~~~~</p>
<hr />
<div>== Charts -- [[User:Katie Crystal|Katie Crystal]] 15:35, 18 April 2011 (UTC) ==<br />
<br />
I'm not sure whether it would be easier for people to save this document to use or to see it laid out in tables on the website itself. It was faster to just upload the document but when we start running out of things to do on the site (ha, ha) we could maybe come back and enter the charts.</div>Katie Crystalhttps://surveywiki.info/index.php?title=File:Rtt_score_chart.pdf&diff=601File:Rtt score chart.pdf2011-04-18T15:32:03Z<p>Katie Crystal: Added RTT category</p>
<hr />
<div>[[Category:Recorded Text Testing]]</div>Katie Crystalhttps://surveywiki.info/index.php?title=File:Rtt_score_chart.pdf&diff=600File:Rtt score chart.pdf2011-04-18T15:31:29Z<p>Katie Crystal: </p>
<hr />
<div></div>Katie Crystalhttps://surveywiki.info/index.php?title=Create_Scoring_Chart&diff=599Create Scoring Chart2011-04-18T15:31:09Z<p>Katie Crystal: Document link</p>
<hr />
<div>{{Recorded Text Testing}}<br />
{{Assemble Pilot Test}}<br />
Scoring charts will help you remember to gather background information from each test subject and write it down. During test administration, it is helpful to see the questions written and the corresponding question track numbers listed for each of the reference tests. Each sheet also provides space for writing the test subjects' responses, scores, and answers to the post-RTT questions.<br />
<br />
[[Media:rtt_score_chart.pdf|This document]] shows an example scoring chart.<br />
[[Category:Recorded Text Testing]]<br />
[[Category:Outline]]</div>Katie Crystal