Foundations of Assessment Part 2

• Review technical adequacy of tests.
  • Reliability
  • Standard error of measurement and confidence intervals
  • Validity
  • Bias
• Case Study: Marco

Materials

• Institute of Education Sciences: Reliability and Validity video
• PowerPoint Slides for Session 3: Foundations of Assessment Part 2
  • Note that slide 12 and 13 has additional resources for people who would like additional information or if you are unsure if participants have understood all of the concepts presented in this session.
• Case Study: Marco

Define Session Goals

• Define reliability and validity.
• Explain the relationship between reliability and validity.
• Explain how standard error of measurement is used in score interpretation.
• Interpret scores with a provided confidence interval.
• Explain the importance of representative norms when selecting a norm-referenced test.

Technical Adequacy of Tests

• Review slide on technical adequacy (slide 4).
  • Watch the IES video on Reliability and Validity.
  • Ask participants to explain to the person next to them why reliability and validity are important considerations for assessment.
  • Review as a group.
• Review slides on error (slides 5-7).
  • Slide on Standard Error of Measurement (slide 5)
  • Slide on Confidence Intervals (slide 6)
  • Ask participants to work in pairs or groups of 3 to interpret the scores on the Your Turn to Interpret Slide (slide 7).
    • Have volunteers provide their interpretation of each score and provide corrective feedback or specific praise as needed.
• Review remaining slides on reliability and validity (slides 8-11).
  • Review slide on bias (slide 8).
  • Review slide on the relationship between reliability and validity (slide 9).
    • Ask participants to work in pairs to respond to the scenario: Lupe.
    • Ask them to discuss assessment considerations for Lupe while considering reliability and validity.
    • Return to the whole group and ask for example considerations.
    • Make sure to mention the appropriateness of the comparison to a norm group (e.g., background knowledge and experiences, proficiency in English necessary to understand the tasks required, etc.).
  • Review slide on important criteria for establishing validity (slide 10).

Case Study: Marco

Considerations for English Language Learners

• Handout the case study of Marco.
• Review the slide on Marco.
• Have participants work in groups of approximately 3 to respond to each of the questions posed.
• Return to the large group and have different groups report their responses to each question.
• Discuss...
  • The importance of assessing Marco’s reading in both English and Spanish.
  • The need to monitor Marco’s progress frequently as he is still learning English.
  • The importance of assessing Marco’s English reading skills for the purpose of planning targeted instruction.
  • Issues of validity that are problematic if using a norm-referenced assessment with Marco.
  • Skills that should be assessed (e.g., not reading fluency if Marco is not yet able to decode sentences).