Post-73: Test Prep and Its Ethical Discontents (Or, a Perfect SAT Score Explained)

6/5/2013

A friend sent me an analysis into the background of Senator Schumer [Harvard '71] vis-a-vis the early "SAT prep industry". It'd been reported that Schumer got a perfect 800 on SAT-Verbal and a perfect 800 on SAT-Math, but there's a big backstory to how. ("Did Senator Schumer really get a perfect 1600 SAT score?"). There is a lot to digest there, so I'll try to summarize. (I think it's important because it relates to my idea of test-prep as fundamentally unethical, which is, I think, why the friend recommended this pertinent article.)
___________________________________________
Very few people achieved perfect scores on the SAT before the 1995 "recentering" (e.g., in 1987, only nine students got perfect scores). Therefore, if Schumer got a perfect score, he may be among the smartest men alive in the USA.

The thing is, Schumer (b. 1950) worked for Stanley Kaplan from ages 14-17 in the 1960s. Kaplan was pioneering the "SAT test prep" business around that time. Kaplan hired kids to take the test and memorize questions. They'd rush back to his office afterwards to report the questions. The compiled questions formed the early Kaplan "test prep" material. Future-Senator Schumer was deep in this venture, manning the "Gestetner machine" [a mimoegraph]. Schumer thus saw all material that came into Kaplan's hands, and (Kaplan himself says, in his autobiography) Schumer pored over it all. Kaplan also reports than Schumer got a "near 1600", not 1600, so the mystery deepens.

This was all before anything like an "SAT test prep" business existed. The makers of the SAT would not have yet been prepared for a concerted attempt by an aggressive group to "game" the test in this way. Thus, when Schumer took the test (probably multiple times), he may well have been the most "prepared for the test" of any student alive in the 1960s. He'd had years of defacto "SAT-prep" (seeing the actual questions used by the SAT), before the field of SAT-prep even existed. That this boosted the scores of a smart boy into the near-1600-range is plausible.
____________________________________________
The way Kaplan, Schumer, and co. aggressively "gamed" the SAT in the 1960s is distressing, yet even "honest" test prep strikes me as at least a bit ethically dubious. If standardized tests are supposed to be about how smart (or how informed on a given subject) a person is, then highly-aggressively "prepping" muddies the waters, and can be seen as a form of non-punishable cheating. It tests what you studied, not what you know, if that makes sense.

See the Ethics subsection of the "Teaching to the Test" Wiki article:

Because of its shortcomings, the practice of teaching to the test is often considered unethical. A 1989 study on teaching to the test evaluated the ethical "continuum" of the practice, and identified seven practice points, ranging from most to least ethical:

    1. General instruction on local objectives
    2. Instruction on general test-taking skills
    3. Instruction on objectives generally measured by standardized tests
    4. Instruction on objectives specific to the test used
    5. Instruction on objectives specific to the test used and using the same format
    6. Instruction using a released test or a "clone" test that replicates the format and content of the test used
    7. Instruction using the test to be used, either before or during test administration

The study concluded that the ethical boundary fell between points three and five, with points one and two being ethical and points six and seven being unethical.

Kaplan and Schumer, it seems, engaged in #6, and arguably/potentially dabbled in #7 as well. I really agree that only #1 and #2 above are solidly-ethical forms of studying/instruction.
_____________________________________________________
In extreme cases (Korea) too much time devoted to test-prep seriously dulls the intellectual passions of teenagers.

A lot of what I've taught over the years in Korea has been test-prep, I must confess. It's been principally for TOEFL. Parents of Korean teenagers want this, for some reason. Korean parents really seem unaware/unconcerned when it comes to this "dulling the intellectual energy" effect of endless, mindless test-prep.

The lion's share of regular-schooling for Korean students is also directed towards test-prep, especially for their version of the SAT called the Su-Neung. Are the Koreans, after literally countless thousands of hours of test-prep for the Su-Neung test, really better-off -- "better-educated", better-informed-about-the-world, able to make more informed choices, possessing of more intellectual passion and curiosity -- than they would have been if they'd done no prep at all, and used that time for other purposes? Not from my (limited) observation. / The test-prep obsession is a manifestation of something else, which is harder to pin down, but test-prep is a "tentacle" of it. This "tentacle" seriously stunts the personal development of many Koreans, and (as I understand) many East-Asians, generally.

Fighting the System
A year ago, my coworker-friend C.H. (an American in his 30s from California) quit his job here in Korea, partly due to frustration with the above-described system. He spoke a lot about how bad he felt it was. In his final weeks, C.H. eagerly recommended this TED talk to all students. / You know how, in a certain kind of movie, when the good-guy is surrounded, outnumbered, and begins to run away, he tends to shoot wildly into the distance while making his escape, right? He's shooting at the bad-guys. The good-guy is hoping to score some hits, sure, but the shooting may be mostly for his own satisfaction, I think:"I fought the good fight and I did the best I could". C.H. was the good-guy, and each of his recommendations of that particular TED Talk was a bullet fired at the enemy.

4 Comments

Pam

6/7/2013 12:26:54 pm

I loved the "Ted Talk" referred to in your post. That was one confident 6-year old who told her teacher we would all know what God looked like in a few minutes! Teaching for the test is made to sound bad nowadays, but if the material that will be on the test is well thought out and relevant, then the students do need to learn that material. It can be taught in creative ways though, so those "teenagers" don't hate school. For example, learning to work in peer groups to develop a project or test a hypothesis could be valuable (and there are many ways to test a given hypothesis). They can learn from one another with the teacher serving as a guide or consultant after initial instruction. They love hands-on learning activities and a variety of modalities/mechanisms for learning--not the same old style over and over. Often the passion of the teacher flows into the class and can be contagious. Combining creativity (any of the arts, for example, or creative ideas) with the curriculum seems like a recipe for success.

Peter

6/7/2013 03:19:41 pm

That TED Talk would really deserve the adjective "subversive" applied to it, when within the context of Korean education. C.H. knew it, too.

Maybe another, more precise analogy could be an outgoing McDonald's manager handing out copies of "Supersize Me" to all customers in his final weeks. If everyone actually took the message of that documentary to heart, the McDonald's model would fall apart.

Lynn

6/8/2013 04:16:08 pm

Well I taught math in the Peace Corps (albeit a long time ago) in Malaysia. They are indeed very test conscious and go so far to publish booklets on just what concept appeared on each exam over the last few years. So this is important to learn and this is not so much. In some ways this is what mastering material -- this is what my "superiors" say I need to do, so I do it because they got something or can confer something I want -- certificate, promotion, degree, bonus, whatever. So give it to them and move on.
I have also worked hard at test prep on other levels -- state level tests, SAT prep, and AP Exams. The AP of course being by far the most rigorous. Going through the curriculum and preparing for the test is much the same thing -- if the test is sufficiently challenging, rigorous, comprehensive, and open ended enough so clever multiple guess strategies get nullified to an extent -- like in the scoring procedures on AP and the free response part. The caveat is that the test must be relevant and challenging.
Around 2 out of 10,000 get perfect scores on SAT. Statistically it is known approximately how many will score "perfect" (incidentally this doesn't necessarily mean they got every question correct) on nation wide normed tests -- AP and SAT questions are pretested for how hard they are for students -- hard means how many get it correct.
Well, we learn how to write well by reading good writing and then practicing, we learn how to play baseball by playing baseball and being around others who can challenge us, we learn how to take tests by being tested. We learn all these by practicing and paying attention. Life is full of little tests of of all sorts, always has been. But never lose the faith either, the zen of it all, that's the key. To join the scholar-gentry, back in the days of Confucius, one had to suffer all sorts of ritual and exams -- as they say character is built through ritual, and exams.

6/10/2013 06:57:53 am

Imagine having a textbook that consists of nothing but sample tests for a specific standardized test, and imagine "classwork" that (thus) consists of nothing but running through these practice tests. It becomes a kind of "training for the test". Very little actual learning or real improvement goes on, even if test scores may rise. This has been my experience.

There are a few subjects, I guess, in which a 100%-test-prep "curriculum" (if it can be called by that name) may work to some extent or other. When it comes most subjects that I can think of, though, especially on the humanities side, test prep does not actually add anything to "learning", it seems to me.

Test-prep may be particularly problematic when it comes to language learning, since language is "dynamic" and active but test-prep trains the mind to function in a "static" and passive way (if that makes sense). This test prep hysteria in Korean education produces students without any intellectual vigor at all, who will only write essays in one boring, cookie-cutter way, and never in any other, "because that's how we should do it on the test" [according to some mediocre-quality textbook]. It also produces students who can't communicate simple ideas in spoken English, after years of study, but who CAN pass tests in English.

Post-73: Test Prep and Its Ethical Discontents (Or, a Perfect SAT Score Explained)

Leave a Reply.

About

Archives