research Evaluation Strategic Services Fall 2005

Inside

June
2007

Unlocking the Potential of Longitudinal Data Systems

Build Your Platform for Data-Based Practices

IQAT Charter Members Gather in Philadelphia

HA Digest





 

 


UNLOCKING THE POTENTIAL OF LONGITUDINAL DATA SYSTEMS:
Do researchers hold a key?
Barbara Storandt, MS.Ed., MS.
Senior Project Manager, Research and Evaluation

One theme of the No Child Left Behind Act (2001) is an emphasis on using data to make informed choices about education at federal, state and local levels.  To facilitate this process, many state education agencies and some local education agencies have developed longitudinal data systems with the intention of efficiently and accurately managing, analyzing, disaggregating, and using student data. These longitudinal data systems (LDS) are designed to respond to the multiple information needs of key stakeholders, support state and local decision-making and facilitate needed research to improve student academic achievement and close achievement gaps.

While the primary users of data are education practitioners such as administrators and teachers, researchers that have access to longitudinal data can more easily establish research priorities and determine the efficacy of educational initiatives.  In her testimony before the Commission on the No Child Left Behind Act in May of 2006, Aimee Guidera, Director of the Data Quality Campaign, urged policymakers to consider the importance of giving researchers easy access to educational data. "Researchers need access to the data to fuel their studies of effectiveness in school practice with the end goal of informing and improving teaching and learning."

As researchers who routinely evaluate educational programs and curricula, Hezel Associates seeks to quantify the impact of these initiatives using changes in student achievement to track performance among various cohorts of kids who are exposed to a particular initiative.  In the past, researchers typically had to administer their own learning assessments in order to determine whether the initiative being studied contributed to gains in student knowledge.  Now, longitudinal data systems show promise for replacing this process by offering publicly-available achievement data gathered annually from standardized tests and collated by various demographic and other variables. 

Despite their intention to facilitate educational research, many longitudinal data systems currently fall short of the mark.  The Data Quality Campaign has identified 10 essential elements and fundamentals[1] for inclusion in longitudinal data systems, which, taken together, may facilitate the collection, availability, and use of high-quality education data.  Only one state - Florida - has a longitudinal data system with all 10 elements, while greater than 65 percent of states have between four and seven elements[2]

Further, the presence of DQC characteristics doesn't guarantee that researchers have easier access to data.  As an example, New York's longitudinal data system has unique student identifiers as recommended by the DQC, yet researchers can only access individual student performance data at the local (i.e. school and district) level.  At the regional and state levels, all performance data are reported in aggregate so that no identifying student information could possibly be extracted.  No process currently exists that allows for easily sharing with researchers individual student performance data from multiple sites, while at the same time guaranteeing students' anonymity. 

To connect researchers with the data, data managers in each school and/or district must match a particular student's performance in a given year with his/her performance during subsequent years, create an additional identifier for each student to ensure anonymity, and then forward the newly coded data to researchers.  Subsequent collections for the same students are only possible if the data managers maintain an accurate record that matches students' state-issued and 'research-issued' identifiers.  Because the legwork and buy-in must take place locally, rather than regionally or at the state level, the process significantly increases the amount of time needed to conduct research.  Additionally, local sociopolitical factors may prevent researchers from ever gaining access to these data.  Even with the capacity to uniquely identify students, as recommended by the DQC, New York State's longitudinal data system does not facilitate researchers' analyses of individual student performance data. 

As NCLB enters its eighth year with a continuing emphasis on the important role of educational data at its core, designers of longitudinal data systems should strive to incorporate all 10 of the DQC's elements and consider the sociopolitical context that may confound researchers' use of LDS.  Without attention to both aspects, researchers may not be able to fully utilize longitudinal data systems.