Statistical Refinement of an SRT

Calculating the Line of Estimation

This is the most helpful for us because once it has been calculated, you can use it to estimate a person's proficiency level once you have their SRT score and the SRT thus replaces the need for the kind of lengthy proficiency evaluation that you've already carried out as you developed the test. In order to do this we need to use a scary-looking but actually quite simple formula and put in some values both from our SRT scores and from the proficiency test we used to evaluate our pilot test participants.

The following calculations presume that you are using the Reported Proficiency Evaluation (RPE) method as described in detail in Radloff (1991) chapter 6. We can't be sure how you would apply these calculations if you were to use an alternative method of evaluating proficiency at the pilot testing stage.

You need to use the following formula which is based on a formula for least squares regression:

Estimated RPE score = a₀ + a₁*SRT score

In this formula

a₀ = (sum RPE)(sum SRT²) - (sum SRT)(sum SRT * sum RPE) / N(sum SRT²) - (sum SRT)²

and

a₁ = N(sum SRT * RPE) - (sum SRT)(sum RPE) / N(sum SRT²) - (sum SRT)²

and in this last formula

N = the number of participants for whom you have both sets of scores available.

To be able to draw the line of estimation, take one high SRT score and work the formula and then one lower one. This will give you two points on the graph and you can then draw the line between them.

Now if you have used the RPE to assess proficiency, you will know that raw scores of 43-52 indicate a proficiency level of +2 on the RPE test. By putting 43 and 52 separately into the following formula, you will know what SRT score range will indicate a +2 proficiency level.

SRT = (RPE - a₀) / a₁

You can then calculate these ranges for all the other proficiency levels you are assessing so that, as someone finishes an SRT during fieldwork, you can immediately determine what proficiency level they are.