Calculating Discrimination Index - Revision history

Admin at 01:24, 13 July 2011

2011-07-13T01:24:35Z

Admin at 01:19, 13 July 2011

2011-07-13T01:19:20Z

Admin at 23:29, 21 June 2011

2011-06-21T23:29:14Z

Admin at 10:57, 16 May 2011

2011-05-16T10:57:43Z

Admin at 10:41, 16 May 2011

2011-05-16T10:41:47Z

Admin at 10:35, 16 May 2011

2011-05-16T10:35:38Z

Admin at 09:22, 16 May 2011

2011-05-16T09:22:01Z

Admin at 04:31, 14 April 2011

2011-04-14T04:31:51Z

Admin: Created page with '{| class="wikitable" ! S# ! colspan="2" ! Participants !! ST !! DL !! DI {{Alternating rows table section|es=background:#FDEEF4; | {{!}} {{!!}} {{!!}} | {{!}} 2 points {{!!}} …'

2011-04-13T06:13:59Z

Created page with '{| class="wikitable" ! S# ! colspan="2" ! Participants !! ST !! DL !! DI {{Alternating rows table section|es=background:#FDEEF4; | {{!}} {{!!}} {{!!}} | {{!}} 2 points {{!!}} …'

New page

{| class="wikitable"
! S#
! colspan="2" ! Participants !! ST !! DL !! DI
{{Alternating rows table section|es=background:#FDEEF4;
| {{!}} {{!!}} {{!!}}
| {{!}} 2 points {{!!}} one error in sentence
| {{!}} 1 point {{!!}} two errors in sentence
| {{!}} 0 point {{!!}} three or more errors in sentence
}}
|}

← Older revision		Revision as of 01:24, 13 July 2011
Line 27:		Line 27:
	<references/>		<references/>

−	~~[[Category:Methodology]]~~
	[[Category:Sentence_Repetition_Testing]]		[[Category:Sentence_Repetition_Testing]]

← Older revision		Revision as of 01:19, 13 July 2011
Line 28:		Line 28:

	[[Category:Methodology]]		[[Category:Methodology]]
		+	[[Category:Sentence_Repetition_Testing]]

← Older revision		Revision as of 23:29, 21 June 2011
Line 26:		Line 26:

	<references/>		<references/>
		+
		+	[[Category:Methodology]]

@@ Line 16: / Line 16: @@
 # '''Calculate the differences''' &rArr; Next, fill in the "Diff." row by completing the difference between the "Actual" and "Rearr." scores as shown in Figure 2.
 # '''Sum the differences''' &rArr; To calculate the DI for each sentence, simply add together the figures in the "Diff." row. Sentence 4 has a DI of 4 because there were four instances where the actual and rearranged scores were different. Because the rearranged values are ideal, if the actual scores are closer to the rearranged scores, they are more ideal. Thus a lower DI shows a sentence that is better at discriminating between proficiency levels. Thus, in Figure 2, sentences 6, 5 and 2 are best, and sentence 3 is good. Sentences 7, 4 and 1 are less than ideal. In this way, you can select the best 15 sentences from your 50 or so pilot test sentences. However, you should bear in mind that in real life with so many more participants and sentences, you are unlikely to get a 0 DI for any of your sentences. In addition, you'll also need to calcuate the diffulty level of each sentence before you can select your final 15 so that you get a set of 15 sentences that are truly representative of the range of levels you want to test.
-::::'''1- (number of correct points for that sentence) / (total possible points for that sentence)'''
+==Further steps==
+In order to continue developing your SRT, there are a few more steps you'll need to carry out using this table.
-::For example, sentence 3 has a total possible score of 45 but only 20 was scored. This means we calculate 1-20/45 to give 0.56. The closer to 1 this value is, the harder the sentence is.
+# '''Calculate the difficulty level''' &rArr; To be able to select 15 sentences that cover the range of levels you want to assess and to rank them in an increasing order of difficulty, you need to perform a simple bit of maths to calculate the difficulty level with the following formula: '''1- (number of correct points for that sentence) / (total possible points for that sentence)''' For example, sentence 3 has a total possible score of 45 but only 20 was scored. This means we calculate 1-20/45 to give 0.56. The closer to 1 this value is, the harder the sentence is.
+# '''Select the final 15''' &rArr; Ideally, in selecting your final 15 sentences, all the sentences should have low DIs and there should be a range of difficulty levels included among them. Your 15 should also be clear recordings and Radloff<ref>Radloff, Carla F. (1991). ''Sentence Repetition Testing for Studies of Community Bilingualism''. Dallas: Summer Institute of Linguistics.</ref> advises that you do not consider sentences with a difficulty level of more than 0.95 as these may be too hard for anyone.
-Ideally, in selecting your final 15 sentences, all the sentences should have low DIs and there should be a range of difficulty levels included among them. Your 15 should also be clear recordings and Radloff<ref>Radloff, Carla F. (1991). ''Sentence Repetition Testing for Studies of Community Bilingualism''. Dallas: Summer Institute of Linguistics.</ref> advises that you do not consider sentences with a difficulty level of more than 0.95 as these may be too hard for anyone.
+# '''Extract the final form scores''' &rArr; To calibrate your final 15 sentences, you'll need to recalculate your pilot test participants' scores with the 15 sentences you've selected as if they did the test only with these sentences. So, remove all the sentences except the 15 and then add their total scores together along the bottom axis again. Keep these scores as you'll need them in later development of the SRT.
 For the next step in Developing your SRT, go back to the [[Developing an SRT#Refining the Test|Refining the Test section of the Developing an SRT page]].
 <references/>

← Older revision		Revision as of 10:41, 16 May 2011
Line 22:		Line 22:
	::For example, sentence 3 has a total possible score of 45 but only 20 was scored. This means we calculate 1-20/45 to give 0.56. The closer to 1 this value is, the harder the sentence is.		::For example, sentence 3 has a total possible score of 45 but only 20 was scored. This means we calculate 1-20/45 to give 0.56. The closer to 1 this value is, the harder the sentence is.

−	Ideally, in selecting your final 15 sentences, all the sentences should have low DIs and there should be a range of difficulty levels included among them. Your 15 should also be clear recordings and Radloff advises that you do not consider sentences with a difficulty level of more than 0.95 as these may be too hard for anyone.	+	Ideally, in selecting your final 15 sentences, all the sentences should have low DIs and there should be a range of difficulty levels included among them. Your 15 should also be clear recordings and Radloff<ref>Radloff, Carla F. (1991). ''Sentence Repetition Testing for Studies of Community Bilingualism''. Dallas: Summer Institute of Linguistics.</ref> advises that you do not consider sentences with a difficulty level of more than 0.95 as these may be too hard for anyone.
		+
		+	For the next step in Developing your SRT, go back to the [[Developing an SRT#Refining the Test\|Refining the Test section of the Developing an SRT page]].
		+
		+	<references/>

@@ Line 1: / Line 1: @@
 Calculating the Discrimination Index (DI) may seem complicated but actually, it's fairly straightforward. It only seems complex because it involves numbers in tables and because there are a several steps invovled. But if you keep your cool and follow them one by one, you'll have a DI in no time and be able to judge which sentences are the best ones for determining second langauge proficiency.
-==Prepartion for Calculating the DI==
+==Preparation for Calculating the DI==
 [[File:Preparation for DI.png|thumb|300px|alt=Preparation for a discrimination index|Figure 1: In preparation for calculating the DI, subjects and sentences are arranged to the initial SRT scores.]]The first step involves using the test scores from your 50 or so participants that you pilot tested the 40-50 sentences with.
-# '''Order the participants by their scores''' &rArr; In Figure 1, the scores of 15 participants (labelled A to O along the top) are shown on the scale at the bottom. Note that no one scored the maximum 21 points for the 7 sentences shown in this example so the scale only needs to go up to the highest score of 20. You can see that for 50 participants doing 50 sentences, this table would be much, much larger. But in a spreadsheet program, even such a large table is very manageable.
+# '''Order the participants by their scores''' &rArr; In Figure 1, the scores of 15 participants (labelled A to O along the top) are shown on the scale at the bottom. Note that no one scored the maximum 21 points for the 7 sentences shown in this example so the scale only needs to go up to the highest score of 20. You can see that for 50 participants doing 50 sentences, this table would be much, much larger. But in a spreadsheet program, even such a large table is very manageable. The participants are arranged in the table according to their total score. In this example, A and E scored 1 and so are on the far left hand side. L, on the other hand, scored 20 and is on the right.
 # '''Order the sentences by points scored''' &rArr; The next step is to arrange the sentences from top to bottom according to the score that all the participants achieved for each sentence. In this example, sentence 2 was the hardest. If 3 represents a perfect repetition of the sentence, then theoretically with 15 participants all repeating it perfectly, each sentence could score 45 points. But the participants only achived 14 out of a possible 45 points for this sentence making it harder than all the others. Sentence 6, by contrast, was the easiest with participants scoring 32 out of the possible 45 points.
 # '''Fill in each participant's individual sentence scores''' &rArr; Finally, you fill in the centre of the table by writing what each participant scored for each sentence. In Figure 1, participant J scored 2 for sentence 6, 1 for sentences 3, 7, 4, 5 and 2 and a 0 for sentence 1.
@@ Line 14: / Line 10: @@
 ==Calculating the DI==
 [[File:Calculations for DI.png|thumb|300px|alt=Calculating a discrimination index|Figure 2: To calculate the DI, sentence scores are rearranged and subtracted from actual scores.]]

← Older revision		Revision as of 09:22, 16 May 2011
Line 1:		Line 1:
−	[[File:Preparation for DI.png]]	+	Calculating the Discrimination Index (DI) may seem complicated but actually, it's fairly straightforward. It only seems complex because it involves numbers in tables and because there are a several steps invovled. But if you keep your cool and follow them one by one, you'll have a DI in no time and be able to judge which sentences are the best ones for determining second langauge proficiency.
		+	==Prepartion for Calculating the DI==
		+	[[File:Preparation for DI.png\|thumb\|300px\|alt=Preparation for a discrimination index\|Figure 1: In preparation for calculating the DI, subjects and sentences are arranged to the initial SRT scores.]]The first step involves using the test scores from your 50 or so participants that you pilot tested the 40-50 sentences with.

−	[[File:Calculations for DI.png]]	+	# '''Order the participants by their scores''' ⇒ In Figure 1, the scores of 15 participants (labelled A to O along the top) are shown on the scale at the bottom. Note that no one scored the maximum 21 points for the 7 sentences shown in this example so the scale only needs to go up to the highest score of 20. You can see that for 50 participants doing 50 sentences, this table would be much, much larger. But in a spreadsheet program, even such a large table is very manageable.
		+
		+	The participants are arranged in the table according to their total score. In this example, A and E scored 1 and so are on the far left hand side. L, on the other hand, scored 20 and is on the right.
		+
		+	# '''Order the sentences by points scored''' ⇒ The next step is to arrange the sentences from top to bottom according to the score that all the participants achieved for each sentence. In this example, sentence 2 was the hardest. If 3 represents a perfect repetition of the sentence, then theoretically with 15 participants all repeating it perfectly, each sentence could score 45 points. But the participants only achived 14 out of a possible 45 points for this sentence making it harder than all the others. Sentence 6, by contrast, was the easiest with participants scoring 32 out of the possible 45 points.
		+
		+	# '''Fill in each participant's individual sentence scores''' ⇒ Finally, you fill in the centre of the table by writing what each participant scored for each sentence. In Figure 1, participant J scored 2 for sentence 6, 1 for sentences 3, 7, 4, 5 and 2 and a 0 for sentence 1.
		+
		+	Once these three steps have been carried out, preparation is complete and you can go on to actually calculate the DI.
		+
		+	==Calculating the DI==
		+	The rows containing what each participant actually scored are labelled "actual" in the image on the right (which you can click to enlarge).
		+	[[File:Calculations for DI.png\|thumb\|300px\|alt=Calculating a discrimination index\|Figure 2: To calculate the DI, sentence scores are rearranged and subtracted from actual scores.]]

@@ Line 1: / Line 1: @@
-{| class="wikitable"
+[[File:Preparation for DI.png]]
-! S#
-! colspan="2" ! Participants !! ST !! DL !! DI
+[[File:Calculations for DI.png]]