Prekindergarten Research in New Mexico: Setting a Research Agenda and Validating an Observation Tool
Early childhood education is a top priority for many states. In an effort to address states’ interests in this area, REL Southwest has two research alliances studying early childhood education questions: one alliance in Arkansas and one in New Mexico. Working with researchers from REL Southwest, each alliance identified key early childhood topics to study.
The New Mexico Prekindergarten Research Alliance includes stakeholders with a vested interest in early childhood education and includes state agencies, social services organizations, and foundations. Working with REL Southwest researchers, alliance members identified the following topics as especially important and in need of further research: preschool access, programs for special populations, kindergarten readiness, and other topics.
Setting the alliance’s research agenda was a collaboration between REL Southwest researchers and alliance stakeholders. Janice Keizer, liaison for the New Mexico Prekindergarten Research Alliance, explains, “The very first thing REL Southwest did was to host a meeting where alliance members identified key topics that they were interested in and felt they needed more information, more data.” The next step was making these topics actionable – that is, turning them into research questions that the alliance could study. Says Keizer, “Once the research questions were written, the REL Southwest team designed projects around the questions, and then the alliance reviewed the projects. All of our projects spring from the research questions developed by alliance members.”
Validating New Mexico's Kindergarten Observation Tool
One of the topics that the alliance identified was investigating kindergarten entry assessment tools. Several states are using assessment tools to determine children’s knowledge and skills at the beginning of kindergarten as this can help teachers know how to best support individual children at the start of school. New Mexico’s Kindergarten Observation Tool (KOT) is one such assessment tool, and the alliance was interested to study the validity of the KOT.
The KOT is an assessment that kindergarten teachers complete at the beginning of the school year. Teachers observe students as they engage in regular classroom activities during the first 30 days of school. Teachers then rate children on twenty-six indicators of children’s competency. Teachers can then use the data to individualize instruction, supporting each child in the areas where they still have the most room to grow.
REL Southwest examined the data from the 2015 version of the KOT. Katie Dahlke, principal investigator and principal researcher for REL Southwest, explains, “The study examined whether the tool is measuring the areas of knowledge and skill that it’s intended to measure, and if not, what areas of knowledge and skill it was measuring.”
The study includes psychometric analysis of the KOT items, cognitive interviews with teachers, and inventories of other states’ kindergarten entry assessments. In addition, REL Southwest analyzed the correlation between the KOT and the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) Next, another assessment of literacy skills.
Alliance members will be able to use the findings to determine appropriate uses of the KOT and to identify opportunities to improve the measure to better support New Mexico kindergarten teachers. REL Southwest’s work also reaches beyond the alliance. Based on the methods used in the KOT study, Dahlke presented at the National Principals Conference about the evidence needed to determine if an early grades assessment is valid. Dahlke conducted a similar presentation at the New Mexico Coalition of Education Leaders Conference.
Stakeholders have also validated the 2016 KOT which was administered statewide and also have completed standard setting for the 2016 KOT. Stakeholders are currently investigating the validity of the 2017 version of the KOT. As the New Mexico Public Education Department and the New Mexico Children, Youth, and Families Department continue to refine the KOT, they recognize the importance of testing the validity of new versions. Dahlke notes, “This shows New Mexico is committed to having a strong research base for their kindergarten entry assessment and its appropriate uses.”