A report on teacher evaluations recently released by the Bill and Melinda Gates Foundation has been refuted by one of the nation's leading economists, who found the widely published report to be seriously flawed. The Gates Foundation last month released the first report of its "Measures of Effective Teaching" (MET) project, which aims to develop a reliable method for evaluating teachers. The report was thoroughly reviewed for the Think Twice think tank review project by University of California at Berkeley economist Jesse Rothstein, former chief economist at the U.S. Department of Labor. Rothstein, who is also former senior economist for the Council of Economic Advisers, found the Gates Foundation's MET report to be based on flawed research and predetermined conclusions.
The review was produced by the National Education Policy Center (NEPC), housed at the University of Colorado at Boulder School of Education, with funding from the Great Lakes Center for Education Research and Practice.
Rothstein's analysis found the MET report draws conclusions that are not supported by its own facts, with some data in the report pointing "in the opposite direction" from what is indicated in its "poorly-supported conclusions."
Rothstein found several instances of conclusions not supported by data. One striking example: The MET report's data suggest that many teachers whose students have low math scores rank among the best at teaching "deeper" concepts. Yet the MET report draws the conclusion that teachers whose students score highly on standardized math tests "tend to promote deeper conceptual understanding as well."
Rothstein also found that the MET report relies heavily on standardized test scores and student surveys, which are insufficient measurements of teacher effectiveness, as teachers facing high-stakes testing will emphasize skills and topics geared toward raising test scores, while de-emphasizing those that aren't on the test. High-stakes student surveys, meanwhile, can be distorted by mischievous adolescents who may not answer honestly if they know their responses can affect teachers' compensation and careers, while teachers may be compelled to alter their practice to cater to student demands, Rothstein reported.
August 1970
Geographic Focus: North America / United States (Western) / California
This brief explores the financial resources of New York City charter schools. It also addresses differences in student population characteristics and student outcomes across New York City (NYC) charter schools, and evaluates how financial resources translate to other schooling inputs, such as more or less experienced teachers and smaller or larger class sizes.
These schools are examined within the broader context of school funding equity and factors that other research has shown to have the potential to advance or disrupt educational equity. In American public education, funding equity involves multiple levels, linked to the multiple levels of our school systems. State systems govern local public school districts, with schools nested within districts. Public charter schools are either nested within districts or operate as independent entities.
NYC charter schools are of particular interest to national audiences mainly because they have been used to argue that charter schools outperform public schools and that New York's experience with charter schools suggests a transferable, nationally scalable policy option. Three studies concerning NYC charter schools in particular are frequently cited: Dobbie & Fryer, 2009; Hoxby, Murarka and Kang, 2009; and CREDO, 2009.
It is important to note, however, that the NYC context may be unique in terms of the role played by philanthropy and so-called venture philanthropy. Significant philanthropic attention has been focused on charter management organizations like the Knowledge is Power Program (KIPP) and Achievement First, which manage charter schools in NYC and elsewhere. NYC charter schools are both touted and blasted in the popular media as being the new favored charities of, for example, wealthy hedge fund managers. The extent that NYC charters have become philanthropic favorites means that NYC charter schools may be quite different from those in places like Missouri or Arizona, distant from the NYC philanthropic culture. In fact, even charter schools in Albany and Buffalo or across the river in New Jersey may be insulated from this unique financial setting. Therefore, additional philanthropic resources may explain a great deal of the claimed success of NYC charter schools. If this is the case, attempts to replicate or scale up these supposed successes would be more difficult and costly than assumed.
This brief offers concrete information about NYC charters and their finances to help ground these important policy discussions.
This brief is published by the National Education Policy Center (NEPC), and is one of a series of briefs made possible in part by funding from The Great Lakes Center for Education Research and Practice.
August 1970
Geographic Focus: North America-United States (Northeastern)-New York-New York County-New York City
A review of the Brookings Institution report, Charter Schools: A Report on Rethinking the Federal Role in Education finds that it relies on a limited body of research, misstates key issues and makes some recommendations not supported by the evidence. The review, by Western Michigan University professor Gary Miron, was produced by the National Education Policy Center (NEPC), housed at the University of Colorado at Boulder School of Education, with funding from the Great Lakes Center for Education Research and Practice.
The Brookings report consists of a summary of evidence from five studies of student achievement in oversubscribed charter schools, plus two studies about charter school revenues. It also draws on information from exchanges between the six co-authors at a day-long Brookings conference. It ends with recommendations intended to help shape the federal role in charter school policy.
The evidence presented on student achievement suggests that charter schools are more effective at raising student achievement in popular urban charter schools. The evidence presented on revenues suggests that charter schools are short-changed in terms of the funding they receive.
Miron points out that the five studies of student achievement in oversubscribed charter schools cited in the report, "cannot validly be generalized to less-popular charter schools." Overall, the research on charter student achievement is much less positive. Even more troubling, he finds that the two studies on charter school funding cited in the report are used to justify recommendations that are "poorly developed and based on a narrow and misleading view of the evidence."
Miron criticizes the Brookings report for unquestioningly accepting the assertion by charter advocates that charter schools get some 20% less per pupil in public revenues than traditional public schools. In truth, he explains, "differences in revenues can largely be explained by higher spending by traditional public schools for special education, student support services, transportation, and food services." Moreover, there is great variation within the charter sector. Contrary to the Brookings recommendation, Miron concludes, "Recommendations regarding charter school finance should be targeted at the creation of better state funding formulas that are more sensitive to the diverse programs schools offer and the diverse needs of students that schools serve."
As a result of the shortcomings of its data and analyses, the report's recommendations related to charter school facilities and charter school finance inappropriately support policies intended to expand the number of charter schools in the short run at the expense of policies that will strengthen charter schools in the longer run.
The report is on stronger ground, Miron finds, in three areas: its call for the federal government to support and encourage the collection of more data and for charter school lotteries to be overseen by independent agencies; its proposal to set aside a portion of federal charter school funding for charter school authorizers and to make federal charter school funding contingent on rigorous oversight; and its call for a careful examination of unintended consequences in existing federal regulations on charter schools.
In the end, Miron says, federal policies that will strengthen charter schools in the long run "need to be based on a more representative body of evidence and a process of formulating recommendations that includes more voices and more than a day of conversations."
August 1970
Geographic Focus: North America / United States (Midwestern) / Michigan
Education and Literacy, Energy and Environment
The report calls on decision-makers to increase investments and on concerned stakeholders to plan and act in cooperation -- so that all children go to a school with child-friendly water, sanitation and hygiene facilities. The Call to Action campaign incorporates six key action points, one of which calls for improved monitoring of WASH in Schools programs. National monitoring systems for WASH in schools are often weak; many countries do not have even basic data on the WASH situation in schools. This lack of information on the status of WASH in schools hampers planning and resource allocation decisions, and makes it difficult to ensure accountability and evaluate progress. This package is designed to help address the WASH in Schools monitoring deficit at the national level. It is designed as a resource for WASH and Education professionals and practitioners to strengthen national monitoring systems and to improve the quality of monitoring at the project level.
August 1970
Geographic Focus:
Children and Youth;Education and Literacy
Student learning plans (SLPs) represent an emerging practice in how public schools across the country are supporting the development of students' college and career readiness skills. Learning plans are student-driven planning and monitoring tools that provide opportunities to identify postsecondary goals, explore college and career options and develop the skills necessary to be autonomous, self-regulated learners. Currently, 23 states plus the District of Columbia require that students develop learning plans, and Massachusetts state policymakers are considering whether all middle and high school students should be required to develop learning plans. Legislation is currently pending that calls for the Executive Office of Education to convene an advisory group to investigate and study a development and implementation process for six-year career planning to be coordinated by licensed school guidance counselors for all students in grades 6 to 12.
The purpose of the policy brief Student Learning Plans: Supporting Every Student's Transition to College and Career is to provide policymakers in Massachusetts with a better understanding of what student learning plans are as well as how and to what extent their use is mandated in other states. The brief is organized into five major sections: an overview of SLPs and the rationale for their use in public K-12 education; an overview of the research on the effectiveness of SLPs on improving a variety of student outcomes, including engagement, responsibility, motivation, long-term postsecondary college and career planning; current state trends in mandating SLPs for all students, including the structure and implementation of SLPs, their connection to other high school reform initiatives and their alignment with state and federal career awareness and workforce development initiatives; promising implementation strategies; and, considerations for state policymakers.
Considerations for Massachusetts policymakers include: learn from states that are pioneers in the implementation of SLPs for all students; develop a comprehensive implementation plan; and, strengthen career counseling and career awareness activities in Massachusetts schools.
The policy brief was the subject of discussion during a public webinar on June 30, 2011.
August 1970
Geographic Focus: North America / United States;North America / United States (Northeastern) / Massachusetts
Education and Literacy, Energy and Environment, Parenting and Families
As the world continues to add close to 80 million people each year, high population growth is running up against the limits of our finite planet, threatening global economic and political stability. To stay within the bounds of the earth's natural resources, the world's population will have to stabilize.
The United Nations' recently revised "medium" projection (http://esa.un.org/unpd/wpp/index.htm) shows world population exceeding 9 billion by 2045. In the "high" projection, which assumes high levels of fertility, world population would top 10 billion by the same year. But spreading hunger and poverty, along with the conflict and disease that come with them, could forcibly curtail growth before we reach 9 billion. Alternatively, the "low" projection suggests it is possible for world population to peak at just over 8 billion around 2045 if we voluntarily make rapid reductions in family size.
August 1970
Geographic Focus:
Children and Youth;Computers and Technology;Education and Literacy
The adoption of the Common Core State Standards offers an opportunity to make significant improvements to the large-scale statewide student assessments that exist today, and the two US DOE-funded assessment consortia -- the Partnership for the Assessment of Readiness for College and Careers (PARCC) and the SMARTER Balanced Assessment Consortium (SBAC) -- are making big strides forward. But to take full advantage of this opportunity the states must focus squarely on making assessments both fair and accurate.
A new report commissioned by the Rennie Center for Education Research & Policy and Policy Analysis for California Education (PACE), The Road Ahead for State Assessments, offers a blueprint for strengthening assessment policy, pointing out how new technologies are opening up new possibilities for fairer, more accurate evaluations of what students know and are able to do. Not all of the promises can yet be delivered, but the report provides a clear set of assessment-policy recommendations.
The Road Ahead for State Assessments includes three papers on assessment policy.
- The first, by Mark Reckase of Michigan State University, provides an overview of computer adaptive assessment. Computer adaptive assessment is an established technology that offers detailed information on where students are on a learning continuum rather than a summary judgment about whether or not they have reached an arbitrary standard of "proficiency" or "readiness." Computer adaptivity will support the fair and accurate assessment of English learners (ELs) and lead to a serious engagement with the multiple dimensions of "readiness" for college and careers.
The second and third papers give specific attention to two areas in which we know that current assessments are inadequate: assessments in science and assessments for English learners.
- In science, paper-and-pencil, multiple choice tests provide only weak and superficial information about students' knowledge and skills -- most specifically about their abilities to think scientifically and actually do science. In their paper, Chris Dede and Jody Clarke-Midura of Harvard University illustrate the potential for richer, more authentic assessments of students' scientific understanding with a case study of a virtual performance assessment now under development at Harvard.
- With regard to English learners, administering tests in English to students who are learning the language, or to speakers of non-standard dialects, inevitably confounds students' content knowledge with their fluency in Standard English, to the detriment of many students. In his paper, Robert Linquanti of WestEd reviews key problems in the assessment of ELs, and identifies the essential features of an assessment system equipped to provide fair and accurate measures of their academic performance.
The report's contributors offer deeply informed recommendations for assessment policy, but three are especially urgent.
- Build a system that ensures continued development and increased reliance on computer adaptive testing. Computer adaptive assessment provides the essential foundation for a system that can produce fair and accurate measurement of English learners' knowledge and of all students' knowledge and skills in science and other subjects. Developing computer adaptive assessments is a necessary intermediate step toward a system that makes assessment more authentic by tightly linking its tasks and instructional activities and ultimately embedding assessment in instruction. It is vital for both consortia to keep these goals in mind, even in light of current technological and resource constraints.
- Integrate the development of new assessments with assessments of English language proficiency (ELP). The next generation of ELP assessments should take into consideration an English learners' specific level of proficiency in English. They will need to be based on ELP standards that sufficiently specify the target academic language competencies that English learners need to progress in and gain mastery of the Common Core Standards. One of the report's authors, Robert Linquanti, states: "Acknowledging and overcoming the challenges involved in fairly and accurately assessing ELs is integral and not peripheral to the task of developing an assessment system that serves all students well. Treating the assessment of ELs as a separate problem -- or, worse yet, as one that can be left for later -- calls into question the basic legitimacy of assessment systems that drive high-stakes decisions about students, teachers, and schools."
- Include virtual performance assessments as part of comprehensive state assessment systems. Virtual performance assessments have considerable promise for measuring students' inquiry and problem-solving skills in science and in other subject areas, because authentic assessment can be closely tied to or even embedded in instruction. The simulation of authentic practices in settings similar to the real world opens the way to assessment of students' deeper learning and their mastery of 21st century skills across the curriculum.
We are just setting out on the road toward assessments that ensure fair and accurate measurement of performance for all students, and support for sustained improvements in teaching and learning. Developing assessments that realize these goals will take time, resources and long-term policy commitment. PARCC and SBAC are taking the essential first steps down a long road, and new technologies have begun to illuminate what's possible. This report seeks to keep policymakers' attention focused on the road ahead, to ensure that the choices they make now move us further toward the goal of college and career success for all students.
This publication was released at an event on May 16, 2011.
August 1970
Geographic Focus: North America / United States
Education and Literacy;Nonprofits and Philanthropy
Researchers have begun to investigate more deeply the specific effects of rising college costs, increasing debt, and the impact of financial aid on degree completion. Specifically, this paper describes the various sources and types of financial aid available to postsecondary students in Texas, how financial aid is packaged at different types of institutions, and the effects of financial aid types and packages on post-secondary persistence and completion. An appendix contains additional detail on federal, state, institutional and private aid sources as well as a list of the advisors, interviewees, and focus group members we spoke with during our research. While this paper focuses on financial aid in Texas given GTF's state-based purview, we believe many of the lessons are applicable across the country.
August 1970
Geographic Focus: North America / United States (Southwestern) / Texas
