Week 4: Reliability

With the student in mind from Week 3 who took the MAT, let’s consider if she would have retaken the MAT a month later.  What are the chances her scores would have gone up? What are the chances her scores would be lower? Is there some way to calculate the margin of error?  Would it be worth $200 to retake it for a higher score?
 

The focus this week, Chapter 5, presents information on reliability. This reflects the degree to which assessment scores can be counted on to be dependable, consistent, and stable at each instance of retesting. As you can imagine, if you cannot rely on the test scores to measure the same constructs the same way each time the test is given, you would not be able to trust the results! We will look at measurement errors involved, and how we can note them and be aware of them.

The most interesting part, I think, is how we estimate the reliability of test scores by calculating a reliability coefficient. The ways in which we use reliability coefficients is covered, as well as a short section on how to calculate the standard error of measurement! The standard error of measurement is helpful in interpreting reliability by using the statistic to create confidence intervals, which tell us the upper and lower limits within which a person’s true scores will fall. We will be able to let the student from Week 3 know whether or not it will be beneficial for her to take the MAT again by understanding the upper and lower limits within which her true scores will fall.

Session Objectives

By the end of this week, you will:

• Describe and explain the importance of reliability in assessment
• Identify the major methods of estimating reliability and describe their relationship to the various sources of measurement error
• Define Standard Error of Measurement and explain its relationship to reliability
• Explain and demonstrate how confidence intervals are calculated and what they mean in psychological and educational assessment
• Describe assessment issues for individuals with visual impairments, hearing impairments, intellectural disabilities, communications disorders, and other disorders