History of IDEA
Jeannie Oakes, Founder and Former Director
Youth Voice in School Finance
NEW REPORT: Los Angeles' road to A-G for all
Without Dollars and Sense Powerpoint
Past Educational Opportunity Reports
A Report On The Readability Of The School Accountability Report Card (SARC)
A Summary
Neil Peretz, Andrea Luquetta, Gabriel Baca and Gary Blasi*
Introduction
The School Accountability Report Card is intended to be an important part of the educational accountability system in California. A School Accountability Report Card (SARC) is prepared for each school in California to inform parents and community members about how well a school is doing. Each school district is required by state and federal law to produce a SARC for all the schools in the district. The State provides a template for what the SARC should contain.
Over the past few months, faculty and students at the UCLA School of Law have investigated how well the School Accountability Report Card serves the goal of informing parents and community members about their schools. Our research question has been whether the SARC can be understood by parents and community members. The attached report documents our findings in detail. Put simply: The SARC is failing.
Methods
The investigation tested the understandability of the SARC format published by the California Department of Education. This format is used by the largest school district in the State and many others. We assessed the SARC through three different approaches. First, we analyzed the SARC with five proven computerized readability analysis programs. Second, we conducted extensive in depth focus groups with parents to evaluate their understanding of the SARC. Finally, we presented the SARC to well-educated, civic-minded citizens to assess how well the SARC enabled them to make factual judgments about schools. By all measures, the SARC fared poorly.
Readability Measures:
The 17th Grade?
Using commercial readability analysis software, we analyzed key sections of the SARC** on the five most established measures of “readability.” These computer program produce an estimate of the reading grade level required to understand a piece of text. We then ran the same tests on a sample of documents we thought most people would agree are not particularly easy to understand:
• A VIOXX Patient Information form from Merck
• Proposition 98 itself (which created the SARC)
• A form Lease Agreement for Month-to-Month Tenancy
• The Microsoft Windows XP Software Driver Installation Instructions
• The IRS Instructions for Form 6251 Alternative Minimum Tax
• The IRS Form 1040A Instructions
The median readability test score of the SARC indicate that the SARC requires 17.2 years of education to be properly understood, more than ANY of the comparison documents listed above.
Comprehensibility:
Can this Many Rotarians be Wrong?
It is not enough, or course, to be able to read a document. One must be able to make accurate judgments about the information in the document. For this part of our study, we obtained the help of 45 members of two Rotary Clubs in Ventura County and Los Angeles County. Rotary Club members are a cross-section of business and professional people who tend to be well-educated and actively involved in their communities. In addition to asking them about their impressions of the SARC, we asked them some objective questions that could be answered based on information contained in the SARC. The question areas and the performance of this group are set out below:
• Whether students were scoring higher or lower than the national average: About 1/3 were unable to determine the correct answer
• Whether the school was fully staffed and whether teacher credentials were improving: Nearly 2/3 were unable to determine the correct answer
• How many students were taking college prep courses: 80% of these subjects were unable to determine the correct answer
Given their level of education and strong civic interest, we think these Rotarians should be better than average at understanding public documents. It is obviously the documents themselves that are failing to communicate accurate information.
Do Parents find the SARC Useful and Understandable?
We conducted two focus groups at with parents at UCLA to get a more nuanced view of how people understand the SARC. These parents were diverse and included professors, administrators, students, and janitors and other campus workers. Participants provided detailed assessments of the SARC. One parent of a middle school child described the experience by saying, “I can understand the words and the numbers but it’s not making any sense to me.” Another parent, who is both a law student and former teacher, said, “I taught for years and I still don’t understand this.” All the focus group participants thought the information on the SARC was important, although few had been aware that such report cards existed.
For many parents and community members, the readability of the SARC is not relevant, because they are not literate in English. Census data indicates that fully 27% of households in California are “linguistically isolated,” meaning that there is no individual in the household able to speak English very well. For these families, translating the SARC into accessible English is only a first step.
Recommendations
The School Accountability Report Card has great potential for informing parents and communities about how well their schools are performing. That potential is not being realized because there are flaws in how it is written and presented. While these preliminary findings are drawn from the observations of a fairly small sample, it is clear that even well-educated, highly motivated people have great difficulty making sense of the SARC. Given the importance of the information on the SARC and the taxpayer resources already invested in collecting and disseminating this information, we recommend that:
1. The State should draw upon outside expertise to assess the comprehensibility of the SARC.
2. Based on that detailed study and on recommendations from experts in preparing such documents, the State should design, test, and publish a more comprehensible SARC template for use by school districts.
3. The State should require that the SARC be translated into languages used by significant groups within each school district’s population.
(notes)
* This summary was prepared by Professor Gary Blasi, UCLA School of Law, and summarizes a detailed report, Grading the Report Card: A Report on the Readability of the School Accountability Report Card (SARC), prepared by UCLA law d students Neil Peretz and Andrea Luquetta and Gabriel Baca, doctoral candidate in the UCLA Graduate School of Education and Information Sciences.
** We tested the portions of the SARC parents had identified in surveys as particularly important: (1) Teacher and Staff Credentials, (2) Post-Secondary Education Preparation, and (3) Standardized Test Scores.
Send this
Print this
