What Everyone Missed on the Pineapple Question

When the New York Daily News posted an article about an Aesop-inspired fable that appeared on the standardized test eighth graders in New York state had to take last month — about a pineapple challenging a hare to a foot race through the forest — all hell broke loose because the passage was so poorly written and the questions about it so incomprehensible. The fable described several animals assuming that the pineapple must have a trick up its sleeve that would enable the immobile fruit to win the race, and when they discovered that it didn’t, they ate it. Test-takers were asked: Why did they eat the pineapple? The correct answer: because the animals were annoyed. And who was the wisest of the animals? An owl that was never mentioned in the passage. Anti-testing activists responded with fury that this set of questions showed why standardized testing is worthless. New York officials quickly turned tail and tossed out the pineapple passage, declaring that they would not count it on this year’s test and would not use it in the future.

There was just one problem: much of the uproar was based on bad information.

(MORE: The Case of the ‘Talking Pineapple,’ the Reading-Comprehension Test Stumper)

Who screwed up in Pineapplegate? There is plenty of blame to go around various parties, and their roles in this debacle illuminate many of the bigger problems facing education reform today. Where to begin? Let’s start with:

The media. Standardized tests are closely guarded to prevent cheating, so when the Daily News ran its story, the reading passage and accompanying questions had never before been made public. The newspaper apparently plucked the information off of anti-testing online message boards – always a reliable source, right? The passage the paper ran was so poorly written that it would indeed have been inexcusable. Tests are shoddy, case closed! Except that the passage the Daily News published on April 19 was not the actual one on the test; it was an incomplete paraphrase, leaving out such things as — you guessed it — the owl.

As the article went viral, the Daily News posted a new version of its story with the correct passage but didn’t run a correction. The paper only mentioned, toward the end of a follow-up story it ran the next day, that there were “slight variations” between the version it had published and what was on the state test. (Click here for the actual passage.) NPR called the Daily News on this sleight of hand, but elsewhere people continued to react to the initial paraphrase, and the story took on a life of its own.

The government. New York education officials responded by quickly declaring that they would toss out the six questions about the fable, a move the public equated with an admission of guilt. What these officials should have done was shed some light on the situation, not only by correcting the record more aggressively, but by explaining that nothing makes its way onto standardized tests by accident. Every item on a standardized test is reviewed by testing experts, teachers, and content specialists, and every item has documentation of its background and use. That doesn’t mean mistakes aren’t made. Just this week, New York threw out another confusing item, this time from its math test — which, like the reading one, was created by education conglomerate Pearson – but the process is not a casual one.

(MORE: A Brief History of Standardized Testing)

An employee of New York state government, frustrated with the lack of transparency, slipped me a memo from Pearson about the section on the pineapple and the hare. The item went through a regular review process and has been used since 2004 in Alabama, Arkansas, Delaware, Illinois, New Mexico, and Florida as well as Chicago, Fort Worth and Houston. (Alabama used it for seven years, Delaware for five.) And in the parlance of the industry, the questions “perform” as they are supposed to — both in New York and nationally. That means it reliably measures the ability of students to read a passage and answer questions or make inferences from it and that the series of questions can differentiate between higher-performing and lower-performing students. (No. 10, for example, requires a particularly close read.) It also means that the set of questions would be hard to guess your way through correctly. All of which would have been helpful to know last week when pundits were seizing on Pineapplegate to ridicule education testing in general.

The tests. Standardized tests are neither as bad as their critics make them out to be nor as good as they should be. Testing companies, in this instance Pearson (a company for which I have done a small amount of consulting), are too secretive about their tests and don’t share a lot of information, which only fuels the conspiracy-minded. Making some test materials public would drive up costs because they couldn’t reuse that material, but it could strike a better balance between helping people understand how tests work and maintaining the integrity of assessments. Most of the people commenting in the media about testing — and even many policymakers deciding how these tests will be used or determining what counts as a passing score — have rarely if ever seen the actual assessments. Bottom line: more transparency would defuse some of the hysteria. When I contacted Pearson, Shilpi Niyogi, Pearson’s Executive Vice President for Public Affairs, acknowledged the problem and told me, “It’s imperative that we provide greater clarity and transparency to students, teachers, parents and taxpayers—and Pearson is committed to working with state leaders and educators to do this.”

The testing marketplace is also messy because there are just a handful of companies that can reliably design and implement large-scale assessments. The result is that state officials are often choosing not the best but rather the least problematic option to do their state assessments. And that includes state officials shying away from reading passages by Mark Twain or about the Vietnam war or anything else that contains even a hint of controversy. “The Hare and the Pineapple” is an absurd and almost trippy story, and it is emblematic of the sort of sanitized material that makes it onto tests and into too many classrooms because, as described in Diane Ravitch’s 2003 book The Language Police, interest groups have collectively created a culture in education that makes rich and provocative content out of bounds and leaves fun but nonsensical passages like “The Hare and the Pineapple” to fill the void. It’s an enormous problem and this episode highlighted it, but that lesson was lost in the din.

(MORE: Our Love-Hate Relationship with the SATs)

The solution is not to dump standardized testing, as many advocates want, but instead to either increase competition in the industry or adopt a common national assessment. Right now, two national consortia, each consisting of more than 20 states, are developing new tests with federal dollars. It remains to be seen how those efforts will play out, but both consortia want the tests to include better quality passages for students to read.

Moral: The moral of the story on New York’s test is that pineapples don’t have sleeves — that’s true literally and, in the case of this weird little fable about perceived trickery, metaphorically. Something else that is generally true is that when an educational issue seems all tied up neatly with a bow, be careful. Today’s education questions, such as whether standardized tests are doing what they’re supposed to do, are not cut and dry. The inaccurate version of the reading-comprehension passage that made the rounds on the Internet and in the media was ridiculous. So much so that it should have aroused some skepticism and perhaps set off a more substantive conversation rather than a firestorm. For all the talk about teaching critical thinking to kids, Pineapplegate shows the adults could use some too.

ncG1vNJzZmihlJqutHrToqSeZpOkunB%2Bj2ppaGhlZH11e9ahmK1llauys8XOp5xmpZmowKawjKilZqyYmnqxtc2emKmonJp6ssHErKuip55k