Searching for Good Data about Instructional Efficacy
While vacationing, my husband and son and I were sitting in a local coffee shop, waiting for our pancakes. I didn’t mean to be listening to the conversation at the next table but the intensity of the young man drew my attention, as he half whispered, “You won’t hear them talk about that publicly. Look at the building. They have a lot of money. Only 26% of the elementary kids passed their reading tests though. You won’t hear that talked about.”
The elderly woman with him (grandmother perhaps?) murmured, “Oh, my. Poor kids. Such a shame.” I didn't get any clear signals that the young man was a teacher but he was tossing around test scores quite confidently. He mentioned the high school social studies scores which were an anomaly (81%) and attributed them to a particular teacher who, because of the small school size, taught social studies to all students from grades 7–12. His driving concern, though seemed to be the contrast between the “money” which the school apparently “had a lot of” and the test scores which were apparently “shamefully low.”
I always doubt the veracity of numbers that people toss around when they're angry. I wondered where the fellow got his statistics and ended up googling the school. I couldn’t easily find the elementary test results but I did find the secondary. In 11th grade, the students were one percentage above the state average in reading, and twenty points below in writing. Of course the state average isn’t anything to brag about.
So what’s in the data? What does it tell us about the quality of the school and its teachers, how well classroom instruction is addressing the needs and experiences of students, and the effective use of financial resources?
For political reasons, if not for pedagogical, educators pay close attention to the performance of their students on standardized assessments. We have locally elected boards made up of community members with diverse interests and expectations of schools, who typically hold models for evaluating the efficacy of instructional programs that have limited indicators for evaluating success.
Over the course of my career as an educator, I’ve significantly improved my knowledge and skills in developing informal assessment systems in classrooms that help me evaluate student thinking and learning. I’ve also learned how to unpack the goals behind standardized assessments to examine broad patterns in how classes of students respond. I recognize that there is limited utility in how one student responds on a given day to a prompt that may look very different from typical classroom work.
However, it is useful to see the patterns in how groups of students respond to items. Standardized tests are one way to step back and ask how effectively our curriculum is serving students in underrepresented populations vs. Caucasian students, girls vs. boys, or ELL students across the board. It also is informative to find, for example, that in the third grade, comprehension scores are weak, while fluency is high. Are we perhaps emphasizing decoding over comprehension instruction in early grades?
One of the most important areas of growth for me professionally was in developing expertise both in learning how to use standardized data effectively as a tool (which means understanding how and why it is limited in scope and what kinds of uses and interpretations are valid) and in developing multiple ways to assess student learning and thinking in the classroom so that I knew how to better plan instruction for tomorrow and next week.
Below you'll find some Exchange resources related to data usage. But first I’d like to invite you into a conversation where you help identify and develop the resources that we offer.
If you haven’t looked at the NCLE stakeholders, please do so. NCLE comprises 30 stakeholders representing most of the professional organizations in the disciplines, and they're committed to offering professional service through the Literacy in Learning Exchange site.
My primary NCLE role is to develop this site so that it offers the professional tools and learning that you need to do your work as an educator effectively. This week, I could use your help in learning more about the following:
- What tools and materials around using evidence do you need and would you like this website to provide? Over the next few weeks I’ll be working with NCLE staff to build our resources around using data effectively (to inform program evaluation, to assist teachers in daily planning, as a tool to inform better policy decision making). I invite you to login and use the comment button to share some ideas or nominate a resource.
- Do you or your schools have expertise that you would be willing to share (via video, web seminar, vignette, or discussion)? Share your story.
- Are you in a school team that is getting started with data teams this year? If so, would you be willing to form an Exchange group to make public both your questions and your insights about using data effectively to support student literacy learning?
My request to you is that you talk back and share what you would like to see us develop and provide on this website to inform conversations and actions around using data.
The following resources may be useful to you if you're currently working with this topic, or may give you ideas about additional resources you'd like us to add to this website:
- Read an article on Instructional Rounds or sign up for the webinar that looks at how school superintendants are using observations as a form of data to guide decision making and planning:
- Read "Coming to Know Students Well to Better Support Their Writing," a Perspective by Janet Angelis,written in response to the case "Conferring with Students: Examining Our Practices"
Read a chapter from Yetta Goodman’s book Valuing Language Study: Inquiry Into Language for Elementary and Middle Schools, focusing on taking a “kidwatching stance” and on the nature and function of informal, non-standardized “data” in generating pedagogically valuable insights about teaching and learning.