Press Releases

Contact: Cheryl Pedersen, (609) 275-2258 cpedersen@mathematica-mpr.com

New Report Released on Congressionally Mandated Evaluation of 15 Educational Technology Products

Scientifically Based Study Examines Effectiveness of Computer-Based Reading and Math Programs

PRINCETON, N.J. (April 5, 2007)—A new report from Mathematica Policy Research, Inc., evaluates 15 computer-based reading and math products to determine the effectiveness of technology in bolstering student achievement. The study, the largest of its kind to use an experimental design to study reading and math software products implemented in a range of grade levels across the country, was mandated under Section 2421(a), Part D, of Title II of the No Child Left Behind Act.

Vendors volunteered their products for the study and provided evidence of effectiveness. Over 400 volunteering teachers were randomly assigned to use or not use the selected products. Fifteen products were implemented in 132 schools across the country during the 2004-2005 school year. The research team assessed student achievement for more than 9,000 students at the beginning and the end of the school year. Team members also observed classrooms three times during the school year to measure product use as well as the conditions and practices associated with product use.

The study assesses the impacts of five reading products on reading achievement in first grade, and four products in fourth grade. It also looks at impacts of three math products on math achievement in sixth grade, and three high school algebra products used mostly in ninth grade. Impacts are reported for groups of products at each grade level. A subsequent report will present impacts for each of the 11 products implemented and evaluated during the 2005-2006 school year.

Key findings include the following:

• On average, after one year, products did not increase or decrease test scores by amounts that were statistically different from zero.

• For reading products, effects on overall test scores were correlated with the student-teacher ratio in first-grade classrooms and with the amount of time that products were used in fourth-grade classrooms.

• For math products, effects were uncorrelated with classroom and school characteristics.

“This study offers the most extensive empirical evidence to date on this important issue and illustrates the power and utility of scientifically based methods that have become more prominent recently in educational research,” said Mark Dynarski, study director and senior fellow at Mathematica Policy Research, Inc.

The research is part of a $15 million congressionally mandated study funded by the U.S. Department of Education's Institute of Education Sciences. SRI International, based in Menlo Park, Calif., is a subcontractor to Mathematica for the study. Read more about the study's design.

Report: Effectiveness of Reading and Mathematics Software Products: Findings from the First Student Cohort, Mark Dynarski, Roberto Agodini, Sheila Heaviside, Timothy Novak, Nancy Carey, Larissa Campuzano, Barbara Means, Robert Murphy, William Penuel, Hal Javitz, Deborah Emery, and Willow Sussex

Mathematica, a nonpartisan research firm, conducts policy research and surveys for federal and state governments, as well as private clients. The employee-owned company, with offices in Princeton, N.J., Washington, D.C., and Cambridge, Mass., has conducted some of the most important studies of education, health care, welfare, employment, nutrition, and early childhood policies and programs in the United States. Mathematica strives to improve public well-being by bringing the highest standards of quality, objectivity, and excellence to bear on the provision of information collection and analysis to its clients.