ATP HOME

Measurement is Everywhere Blog

 

Measurement is everywhere

By Quinn Sutton, Alpine Media

I am painfully reminded of this simple fact every time I reluctantly step on the scale. I am sure that I am not alone in my misguided cursing of the abusive numbers, the unfair machine, its unskilled manufacturers, the evil National Institute Health and their infernal Body Mass Index (BMI) and the pressures of society to conform to some unrealistic ideal based on the metabolic rate of a hyperactive teenager with the caloric intake of a gold fish.

When I am in this state of mind there is little that can be said, or done that will convince me that my feelings are not valid or real. I am upset and justifiably so. I have just had the carefully-crafted delusion that I am not that fat and that I really can still eat whatever I want, whenever I want, blown to smithereens by the cold hard facts glaring at me from the scale. It stings. A lot.

As I sip on a diet coke while I drive to the doctor to have my cholesterol checked, I am yet again reminded of measurement. Blast those bottlers for putting exactly 591 ml of soda into my bottle! I hate the metric system. How much is that in American? Dang these speed limits! I am going to be late. I wish my speedometer were broken so at least I could have some form of plausible deniability if I am pulled over. Great! Now, I have to wait in a room full of sick people and all their nasty germs just because my doctor was ranked in the top ten in the state.

Finally, I am called back and notice something is "off." Instead of Megan, Gabby or Derek who normally greet me with clean scrubs, clipboard and brushed teeth, there was this guy. His name was apparently Dashel based on the tattoo on his forearm "-,L" right underneath the cursive word 'Precous' which I later figured out was a misspelling of precious. It was not so much the purple mohawk, dirty jeans, or spiked collar that caused me to question his qualifications. It was when he reached into his duffle bag and pulled out a used juice box and a syringe from the Viet Nam war and said, "We're outta cups so pee in this while I get ready to take your blood." I left.

Measurement is everywhere. Lately, one particular form of measurement is coming under fire. Standardized assessments in k-12 education are a hot topic. A simple google search will serve up the latest provocative headlines like, "Poet: I can't answer questions on Texas standardized tests about my own poems," or "Is standardized testing putting too much pressure on teachers, students?"

So what's all the hullabaloo? Like two kids fighting in the back seat of the car, the positions put forward and refuted by the other:

He said…

She said…

Standardized testing has not improved student achievement. After No Child Left Behind (NCLB) passed in 2002, the US slipped from 18th in the world in math on the Programme for International Student Assessment (PISA) to 31st place in 2009, with a similar drop in science and no change in reading. A May 26, 2011, National Research Council report found no evidence test-based incentive programs are working: "Despite using them for several decades, policymakers and educators do not yet know how to use test-based incentives to consistently generate positive effects on achievement and to improve education."

93% of studies on student testing, including the use of large-scale and high-stakes standardized tests, found a "positive effect" on student achievement, according to a peer-reviewed, 100-year analysis of testing research completed in 2011 by testing scholar Richard P. Phelps.

Stricter standards and increased testing are better preparing school students for college. In Jan. 1998, Public Agenda found that 66% of college professors said "elementary and high schools expect students to learn too little.” By Mar. 2002, after a surge in testing and the passing of NCLB, that figure dropped to 47% "in direct support of higher expectations, strengthened standards and better tests.”

Standardized tests are an unreliable measure of student performance. A 2001 study published by the Brookings Institution found that 50-80% of year-over-year test score improvements were temporary and "caused by fluctuations that had nothing to do with long-term changes in learning..."

Standardized tests are reliable and objective measures of student achievement. Without them, policy makers would have to rely on tests scored by individual schools and teachers who have a vested interest in producing favorable results. Multiple-choice tests, in particular, are graded by machine and therefore are not subject to human subjectivity or bias.

Standardized tests are unfair and discriminatory against non English speakers and students with special needs.  English language learners take tests in English before they have mastered the language.  Special education students take the same tests as other children, receiving few of the accommodations usually provided to them as part of their Individualized Education Plans (IEP).

20 school systems that "have achieved significant, sustained, and widespread gains" on national and international assessments used "proficiency targets for each school" and "frequent, standardized testing to monitor system progress," according to a Nov. 2010 report by McKinsey & Company, a global management consulting firm.

Standardized tests measure only a small portion of what makes education meaningful. According to late education researcher Gerald W. Bracey, PhD, qualities that standardized tests cannot measure include "creativity, critical thinking, resilience, motivation, persistence, curiosity, endurance, reliability, enthusiasm, empathy, self-awareness, self-discipline, leadership, civic-mindedness, courage, compassion, resourcefulness, sense of beauty, sense of wonder, honesty, integrity."

Standardized tests are inclusive and non-discriminatory because they ensure content is equivalent for all students. Former Washington, DC, schools chancellor Michelle Rhee argues that using alternate tests for minorities or exempting children with disabilities would be unfair to those students: "You can't separate them, and to try to do so creates two, unequal systems, one with accountability and one without it. This is a civil rights issue."

"Teaching to the test" is replacing good teaching practices with "drill n' kill" rote learning. A five-year University of Maryland study completed in 2007 found "the pressure teachers were feeling to 'teach to the test'" since NCLB was leading to "declines in teaching higher-order thinking, in the amount of time spent on complex assignments, and in the actual amount of high cognitive content in the curriculum."

"Teaching to the test" can be a good thing because it focuses on essential content and skills, eliminates time-wasting activities that don't produce learning gains, and motivates students to excel. The US Department of Education stated in Nov. 2004 that "if teachers cover subject matter required by the standards and teach it well, then students will master the material on which they will be tested--and probably much more."

NCLB tests are drastically narrowing the curriculum. A national 2007 study by the Center on Education Policy reported that since 2001, 44% of school districts had reduced the time spent on science, social studies and the arts by an average of 145 minutes per week in order to focus on reading and math. [1] A 2007 survey of 1,250 civics, government, and social studies teachers showed that 75% of those teaching current events less often cited standardized tests as the reason.

Standardized tests are not narrowing the curriculum, rather they are focusing it on important basic skills all students need to master. According to a study in the Oct. 28, 2005, issue of the peer-reviewed Education Policy Analysis Archives, teachers in four Minnesota school districts said standardized testing had a positive impact, improving the quality of the curriculum while raising student achievement.

Instruction time is being consumed by monotonous test preparation. Some schools allocate more than a quarter of the year's instruction to test prep. [Kozol] After New York City's reading and math scores plunged in 2010, many schools imposed extra measures to avoid being shut down, including daily two and a half hour prep sessions and test practice on vacation days.  On Sep. 11, 2002, students at Monterey High School in Lubbock, TX, were prevented from discussing the first anniversary of the 2001 terrorist attacks because they were too busy with standardized test preparation.

Increased testing does not force teachers to encourage "drill n' kill" rote learning. According to a study in the Oct. 28, 2005, issue of the peer-reviewed Education Policy Analysis Archives, good teachers understand that "isolated drills on the types of items expected on the test" are unacceptable, and principals interviewed said "they would sanction any teacher caught teaching to the test." In any case, research has shown that drilling students does not produce test score gains: "teaching a curriculum aligned to state standards and using test data as feedback produces higher test scores than an instructional emphasis on memorization and test-taking skills."

Standardized tests are not objective. A paper published in the Fall 2002 edition of the peer-reviewed Journal of Human Resources stated that scores vary due to subjective decisions made during test design and administration: "Simply changing the relative weight of algebra and geometry in NAEP (the National Assessment of Educational Progress) altered the gap between black and white students."

State-mandated standardized tests help prevent "social promotion," the practice of allowing students to advance from grade to grade whether or not they have met the academic standards of their grade level.  A Dec. 2004 paper by the Manhattan Institute for Policy Research found Florida's 2002 initiative to end social promotion, holding back students who failed year-end standardized tests, improved those students' scores by 9% in math and 4% in reading after one year.

Standardized testing causes severe stress in younger students. According to education researcher Gregory J. Cizek, anecdotes abound "illustrating how testing... produces gripping anxiety in even the brightest students, and makes young children vomit or cry, or both."  On Mar. 14, 2002, the Sacramento Bee reported that "test-related jitters, especially among young students, are so common that the Stanford-9 exam comes with instructions on what to do with a test booklet in case a student vomits on it."

Testing is not too stressful for students. The US Department of Education stated: "Although testing may be stressful for some students, testing is a normal and expected way of assessing what students have learned."  A Nov. 2001 University of Arkansas study found that "the vast majority of students do not exhibit stress and have positive attitudes towards standardized testing programs." Young students vomit at their desks for a variety of reasons, but only in rare cases is this the result of testing anxiety.

Older students do not take NCLB-mandated standardized tests seriously because they do not affect their grades. An English teacher at New Mexico's Valley High School said in Aug. 2004 that many juniors just "had fun" with the tests, making patterns when filling in the answer bubbles: "Christmas tree designs were popular. So were battleships and hearts."

Most students believe standardized tests are fair. A June 2006 Public Agenda survey of 1,342 public school students in grades 6-12 found that 71% of students think the number of tests they have to take is "about right" and 79% believe test questions are fair. [22] The 2002 edition of the survey found that "virtually all students say they take the tests seriously and more than half (56 percent) say they take them very seriously."

Testing is expensive and costs have increased since NCLB, placing a burden on state education budgets. According to the Texas Education Agency, the state spent $9 million in 2003 to test students, while the cost to Texas taxpayers from 2009 through 2012 is projected to be around $88 million per year.

Standardized tests provide a lot of useful information at low cost, and consume little class time.  According to a 2002 paper by Caroline M. Hoxby, PhD, the Scott and Donya Bommer Professor in Economics at Stanford University, standardized tests cost less than 0.1% of K-12 education spending, totaling $5.81 per student per year: "Even if payments were 10 times as large, they would still not be equal to 1 percent of what American jurisdictions spend on education." Other cost estimates range from $15-$33 per student per year by the nonpartisan US Government Accountability Office (GAO), to as low as $2 per student per year by testing scholar and economist Richard P. Phelps.  A 50-item standardized test can be given in an hour and is graded instantaneously by computer.

The billion-dollar testing industry is notorious for making costly and time-consuming scoring errors. NCS Pearson, which has a $254 million contract to administer Florida's Comprehensive Assessment Test, delivered the 2010 results more than a month late and their accuracy was challenged by over half the state's superintendents.  After errors and distribution problems in 2004-2005, Hawaii replaced test publisher Harcourt with American Institutes for Research, but the latter had to re-grade 98,000 tests after students received scores for submitting blank test booklets.

Teacher-graded assessments are inadequate alternatives to standardized tests because they are subjectively scored and unreliable. Most teachers are not trained in testing and measurement, and research has shown many teachers "consider noncognitive outcomes, including student class participation, perceived effort, progress over the period of the course, and comportment," which are irrelevant to subject-matter mastery.

The multiple-choice format used on standardized tests is an inadequate assessment tool. It encourages a simplistic way of thinking in which there are only right and wrong answers, which doesn't apply in real-world situations. The format is also biased toward male students, who studies have shown adapt more easily to the game-like point scoring of multiple-choice questions.

The multiple-choice format used on standardized tests produces accurate information necessary to assess and improve American schools. According to the Center for Teaching Excellence at the University of Illinois at Urbana-Champaign, multiple-choice questions can provide "highly reliable test scores" and an "objective measurement of student achievement." Today's multiple-choice tests are more sophisticated than their predecessors. The Center for Public Education, a national public school advocacy group, says many "multiple-choice tests now require considerable thought, even notes and calculations, before choosing a bubble.”

America is facing a "creativity crisis," as standardized testing and rote learning "dumb down" curricula and jeopardize the country's economic future. A 2010 College of William & Mary study found Americans' scores on the Torrance Test of Creative Thinking have been dropping since 1990, and researcher Kyung-Hee Kim lays part of the blame on the increase in standardized testing: "If we neglect creative students in school because of the structure and the testing movement... then they become underachievers."

 

Finland topped the international education (PISA) rankings from 2001-2008, yet has "no external standardized tests used to rank students or schools," according to Stanford University researchers Linda Darling-Hammond and Laura McCloskey. Success has been achieved using "assessments that encourage students to be active learners who can find, analyze, and use information to solve problems in novel situations."

China has a long tradition of standardized testing and leads the world in educational achievement. China displaced Finland as number one in reading, math, and science when Shanghai debuted on the Programme for International Student Assessment (PISA) rankings in 2009.  Despite calls for a reduction in standardized testing, China's testing regimen remains firmly in place. Chester E. Finn, Jr., Chairman of the Hoover Institution's Koret Task Force on K–12 Education, predicts that Chinese cities will top the PISA charts for the next several decades.

Excessive testing may teach children to be good at taking tests, but does not prepare them for productive adult lives.  China displaced Finland at the top of the 2009 PISA rankings because, as explained by Jiang Xueqin, Deputy Principal of Peking University High School, "Chinese schools are very good at preparing their students for standardized tests. For that reason, they fail to prepare them for higher education and the knowledge economy." China is trying to depart from the "drill and kill" test prep that Chinese educators admit has produced only "competent mediocrity."

 

Using test scores to reward and punish teachers and schools encourages them to cheat the system for their own gain.  A 2011 USA Today investigation of six states and Washington DC found 1,610 suspicious anomalies in year-over-year test score gains.  A confidential Jan. 2009 memo, prepared for the DC school system by an outside analyst and uncovered in Apr. 2013, revealed that 191 teachers in 70 DC public schools were "implicated in possible testing infractions," and nearly all the teachers at one DC elementary school "had students whose test papers showed high numbers of wrong-to-right erasures," according to USA Today.  178 Atlanta public school teachers and administrators from 44 schools were found to be cheating on standardized tests according to a July 2011 state report. At one school, teachers attended "weekend pizza parties" to correct students' answers, according to ABC News.  Ultimately in Apr. 2015, 11 of those district employees were convicted of racketeering, which carries a sentence of up to 20 years in prison.

Cheating by teachers and administrators on standardized tests is rare, and not a reason to stop testing America's children. The Mar. 2011 USA Today investigation of scoring anomalies in six states and Washington DC was inconclusive, and found compelling suggestions of impropriety in only one school.  The US Department of Education’s Office of Inspector General said on Jan. 7, 2013 that an investigation had found no evidence of widespread cheating on the DC Comprehensive Assessment System tests.  It is likely that some cheating occurs, but some people cheat on their tax returns also, and the solution is not to abolish taxation.

Standardized tests are an imprecise measure of teacher performance, yet they are used to reward and punish teachers. According to a Sep. 2010 report by the Annenberg Institute for School Reform, over 17% of Houston teachers ranked in the top category on the Texas Essential Knowledge and Skills reading test were ranked among the two lowest categories on the equivalent Stanford Achievement Test. The results "were based on the same students, tested in the same subject, at approximately the same time of year, using two different tests."

Yep.  The test was not intended for that purpose.

Each state develops its own NCLB standards and assessments, providing no basis for meaningful comparison. A student sitting for the Connecticut Mastery Test (CMT) is asked a completely different set of questions from a child in California taking the Standardized Testing and Reporting (STAR) test, and while the former includes essay questions, the latter is entirely multiple-choice.

Each state's progress on NCLB tests can be meaningfully compared. Even though tests are developed by states independently, state scores are compared with results on the National Assessment of Educational Progress (NAEP), ensuring each state's assessments are equally challenging and that gains in a state's test scores are valid.

Open-ended questions on standardized tests are often graded by under-paid temporary workers with no educational training. Scorers make $11-$13 per hour and need only a bachelor’s degree, not necessarily related to education. As one former test scorer stated, "all it takes to become a test scorer is a bachelor’s degree, a lack of a steady job, and a willingness to throw independent thinking out the window…”

 

Schools feeling the pressure of NCLB's 100% proficiency requirement are "gaming the system" to raise test scores, according to an Arizona State University report in the June 22, 2009, edition of the peer-reviewed International Journal of Education Policy & Leadership. Low-performing students are "encouraged to stay home" on test days or "counseled to quit or be suspended" before tests are administered. State education boards are "lowering the bar": manipulating exam content or scoring so that tests are easier for students to pass

 

An obsession with testing robs children of their childhoods. NCLB's mandate begins in third grade, but schools test younger students so they will get used to taking tests.  Mar. 2009 research from the Alliance for Childhood showed "time for play in most public kindergartens has dwindled to the vanishing point, replaced by lengthy lessons and standardized testing."  A three-year study completed in Oct. 2010 by the Gesell Institute of Human Development showed that increased emphasis on testing is making "children feel like failures now as early as PreK..."

 

Parents hate standardized assessments.

Most parents approve of standardized tests. A June-July 2013 Associated Press-NORC Center for Public Affairs Research poll found that 75% of parents say standardized tests "are a solid measure of their children's abilities" and 69% say the tests "are a good measure of the schools' quality." 93% of parents say standardized tests "should be used to identify areas where students need extra help" and 61% say their children "take an appropriate number of standardized tests."

Teachers and administrators hate standardized assessments.

Most teachers acknowledge the importance of standardized tests and do not feel their teaching has been compromised. In a 2009 Scholastic/Gates Foundation survey, 81% of US public school teachers said state-required standardized tests were at least "somewhat important” as a measure of students’ academic achievement, and 27% said they were "very important " or "absolutely essential."  73% of teachers surveyed in a Mar. 2002 Public Agenda study said they "have not neglected regular teaching duties for test preparation."

Most teachers and administrators approve of standardized tests. Minnesota teachers and administrators interviewed for a study in the Oct. 28, 2005, issue of the peer-reviewed Education Policy Analysis Archives (EPAA) approved of standardized tests "by an overwhelming two-to-one margin," saying they "improved student attitudes, engagement, and effort."  An oft-cited Arizona State University study in EPAA's Mar. 28, 2002 edition, concluding that testing has little educational merit, has been discredited by educational researchers for poor methodology, and was criticized for wrongly blaming the tests themselves for stagnant test scores, rather than the shortcomings of teachers and schools.

 

Many objections voiced by the anti-testing movement are really objections to NCLB's use of test results, not to standardized tests themselves. Prominent testing critic Diane Ravitch, Research Professor of Education at New York University, concedes standardized testing has value: "Testing... is not the problem... information derived from tests can be extremely valuable, if the tests are valid and reliable." She cites the National Assessment of Educational Progress (NAEP) as a positive example, and says tests can "inform educational leaders and policy-makers about the progress of the education system as a whole."

 

Physicians, lawyers, real-estate brokers and pilots all take high-stakes standardized tests to ensure they have the necessary knowledge for their professions.  If standardized tests were an unreliable source of data, their use would not be so widespread.