Why is the ETS lobbying against immigration?

This report came across my desk; thanks Miss MK of Washington, Columbia. It's from the Educational Testing Service, the people you had to take the SAT from, and is entitled "A Human Capital Concern: The Literacy Proficiency of U.S. Immigrants".

It reports on some literacy tests run on foreign-born and native-born folks in the U.S.A.; the question that immediately springs to mind is, what language is the test in? It turns out that ETS has two tests, the National Adult Literacy Survey (NALS) and the International Adult Literacy Survey (IALS). At no point, over the entire sixty pages, does it tell you what language the tests are in. OK, it indirectly points out that the NALS scores indicate English proficiency, but I couldn't even find indirect evidence of the language the IALS was in. I was further frustrated by the fact that the report uses the term "literacy" throughout, and at no point that I could find qualifies it to "English literacy".

This is important because immigrants did significantly poorer on these tests than natives. If both tests are in English, then this is entirely unremarkable. If the IALS is in the immigrant's native tongue, then this points to a low level of what some migration scholars like to call the `quality of immigrant'.

Between my linguist pal at ETS and Google, I was able to find out that the NALS is indeed in English, and the IALS is in the national language of the country, and English for the U.S.A. That is, if somebody in the U.S. is selected to take the exam, then the IALS will be in English regardless of where the person lived before. Which means that the tests support the dull result: non-native English speakers don't fare as well in tests of English.

But check out the title again: "A Human Capital Concern". Here's a line from the report's cover letter when it was sent to (name of think tank): "...the nation needs a spirited debate about how to boost the human capital of this growing population [of immigrants]." How did we go from the obvious "native English speakers speak better English" to "immigrants have low human capital"?

Even the "growing population" subclause is a bit suspect. From page 1 of the report: "New immigration reached historically high levels during the decade of the 1990s..." Yeah, in absolute terms. In percentages, the 1990s were sort of middling.

Other research on the `quality of immigrant' generally finds that immigrants are not of exceptionally low quality. The poorest of the poor don't move; it's the moderately educated lower-middle class and above that have the wherewithal to change countries. So if the ETS were to claim that immigrants were in the bottom quartile of literacy for their native language, then this would be an innovative finding. But that's not what they find.

By page 11 I was in fits. "...the differences in mean test score performance between the native and foreign born were equal to 1.0 to 1.1 standard deviations. From both a statistical perspective and an educational policymaking standpoint, these are extraordinarily large differences." Most statisticians prefer to call a 1 SD difference a `null result' or `insignificant', but maybe they're just not as perky and excited as the ETS people.

They have a few measures of civic involvement, and find a negative correlation between these and literacy. One of these measures is library attendance.

They show (p 46) that people who score better on the tests were more likely to have been taking some sort of education/training course in the past 12 months, indicating that better English implies more facility to learn useful skills. But they fail to mention whether these training courses could have been ESL classes, which would send the causation in the exact opposite direction.

I could go on. But my big question is this: why did ETS publish this? These guys have a large academic staff who often publish in peer reviewed journals---but before they do, they have to pass an internal peer review process. You may hate the idea of standardized testing, but these guys live and die by compiling the most statistically sound and defensible standardized tests humans can compile. So why are they publishing a polemic with tables?

Nor did it have to be this way. This could have been a paper about ESL programs and their efficacy, or about the demographics of English adoption. All this would have been useful, interesting, and not a stretch of the data. But instead, the report stretched the data well beyond what it could possibly support, and did so through misleading language (like refusing to admit that by "literacy" they mean "English literacy") and not-so-hot statistics (like the above standard deviation faux pas, and many paired t-tests when they should have been running at least a linear regression controlling for age and years of schooling).

So I'm not sure what's up with the ETS. My pal points out that (apart from the internal peer review) individual scholars can independently publish anything they darn well please to, but this isn't just another journal article by ETS employees; it has the ETS logo all over it, is (c)2004 ETS, and has a lengthy preface signed by the Senior VP of research. The report itself, with fifty tables, glossy cover, and evidently a mass mailing to the think tanks of the world, looked like a big deal that a large number of people took seriously. Yet the main conclusion from the title on down, that immigrants have low human capital, is completely unsupported by the data.

Dear ETS: You do good standardized tests. But please get out of the lobbying business.