Supreme Court Decides on Ricci Case

The U.S. Supreme Court justices handed down their decision June 29 on the Ricci v. DeStefano case. In a 5-4 decision, the justices stated the city’s action in discarding the tests violated Title VII of the Civil Rights Act of 1964.

Detailed stories can be read on the New York Times.com, CNN.com, and various other news Web sites.

To voice your own opinions on the decision and its implications for the field of industrial-organizational psychology, add your comments below!

Comments

[Michael A.]

Thank you all for your very expert comments. It is now obvious to me that the City of NEW HAVEN GOT JUST WHAT IT WANTED, A VIRTUALLY ALL WHITE COMMAND FORCE! The errors in judgement and presentation goes beyond mere incompetence and failure to seek out competent consultants. The anti affirmative action or equal opportunity for anyone other than whites forces were more than aided and abetted by the actions of the City of New Haven. The municipal office holders accomplished their goal of aidingthe interest of whites by failing to do even a modicum ofthe work necessary to defend their actions or lack thereof. Justice Alito's fixation on the actions of Fire Commissioner Kimber points to his complete ignorance of the issues at play and his real goal of awarding a group of white males what he considered their meritorious award for their whiteness..Please continue to write, study and otherwise support the development of fair and effective testing procedures. It would also be my hope that any less than ethical practioners of testing design are exposed, opposed and denied the opportunity to aid in the effort to deny protected classes of individuals, equitable opportunity.

[Kick]

These comments are a load of crap. Job knowledge is important and assessment centers do not measure job knowledge - everyone knows they are a scam to race-engineer test results. Plus, the I/O amicus guys got so many facts wrong in their amicus brief it was pathetic. The "command presence" thing was a complete farce - there is no such thing as "command presence" - it's a made-up term used by someone trying to concoct a basis to attack a test result. Command presence in the Ricci case, assuming the term means the ability to lead others effectively and run an emergency scene, WAS IN FACT tested in the oral assessment phase. We all know you compete for business but cannibalizing your fellow I/Os is extremely unseemly and reflects poorly on the character of SIOP members.

[Eric Dunleavy]

Also note that the presentation by Dr. Outtz at the August luncheon meeting of PTC/MW can be heard remotely via webcast. Check out more information at:

http://www.ptcmw.org/lunch.htm

[Lance Seberhagen]

RICCI v. DeSTEFANO: WHAT DOES IT MEAN?

Dr. James Outtz, Outtz & Associates, Washington, DC, will be the featured speaker at the August luncheon meeting of the Personnel Testing Council of Metropolitan Washington (PTC/MW). Jim is one of the five distinguished SIOP Fellows who submitted an amicus brief to the Supreme Court in Ricci.

DATE: August 12, 2009. 11:30 am.

PLACE: GMU, Arlington, VA.

TOPIC: "Ricci v. DeStefano: What Does It Mean?"

REGISTRATION: http://www.ptcmw.org.

[Eric Dunleavy]

It is mid-July and we have had a few weeks to digest all of the reactions to the Ricci ruling, and to see how the ruling plays out in the confirmation process for Supreme Court Nominee Sotomayor. There has certainly been quite a bit of reaction in the popular press, and some reactions have been somewhat extreme (e.g., an end to affirmative action, changes to the job-relatedness burden under Title VII, the end to adverse impact as we know it, etc.). While the case obviously has narrower implications for a number of large-scale legal issues in the employment context, I-O psychologists were perhaps most interested in the space the Supreme Court dedicated to understanding the employment testing context at the heart of the case.

Art has already pointed out some critical context, namely, that this was a disparate treatment case and not an adverse impact case. However, ‘impact-related’ issues were considered in phase two of the treatment case, where the city had to show that there were legitimate non-discriminatory reasons for the employment decisions that were made. The results of a proactive adverse impact analysis (not actual Title VII litigation) were essentially the reason articulated by the city, and the majority of the Court applied a ‘strong basis in evidence’ standard to this evidence. In other words, the city had to provide a strong rationale that the tests had impact, were not job-related (an ‘upside-down’ burden), and/or that reasonable alternatives were available as part of the city’s proactive analysis.

The majority and dissenting Justices disagreed on whether this was the appropriate standard to use. There were 4 opinions that were written. A majority of 5 Justices (Roberts, Alito, Scalia, Thomas, and Kennedy) concluded that the city discriminated against the White and Hispanic plaintiffs because of their race, in part because the city failed to show ‘a priori’ that (1) the tests were flawed and (2) there were reasonable alternatives. A dissenting group of 4 Justices (Ginsburg, Souter, Breyer, and Stevens) concluded that the city did not discriminate in part because (1) the tests were flawed and (2) there were reasonable alternatives available (e.g., an assessment center). Additional opinions were written by Justice Scalia (arguing that adverse impact-equal protection friction still exists) and Justice Alito (arguing that even if the tests were flawed or if reasonable alternatives were available, a reasonable person may still conclude via pretext that the city discriminated intentionally based on race).

One of the most interesting distinctions between the majority and dissenting opinions concerns the information that was considered. Specifically, it appears that the majority considered only the adverse impact, lack of job-relatedness, and reasonable alternative information that was available to the city before they made the decision to throw out the promotion list. This included a 4/5th rule analysis, some opinions on job-relatedness, and reasonable alternative considerations as part of the proactive analysis. The dissenting justices, on the other hand, considered a set of additional information gathered after the decision was made to throw out the promotion list, and up until the oral argument in front of the Supreme Court. This included the SIOP-fellow brief that Art mentioned, and other resources that point out (1) some ‘fatal flaws’ of the tests used and (2) assessment centers as a reasonable alternative to written job knowledge tests for the jobs of interest. The dissenting justices also considered various textbooks and scholarly articles from the personnel selection literature. This distinction makes the majority and dissenting opinions read almost like two different cases.

As Art points out, these different sets of information make all the difference. Essentially, the majority concluded that the city didn’t make a ‘due diligence’ effort to understand whether the tests were adequate or whether alternatives were available, and thus did not provide ‘strong rationale’ that the tests were flawed or that there were reasonable alternatives. This conclusion was based in part on the fact that, after conducting proactive adverse impact analyses, the city chose not to have content-oriented validity evidence documented in a technical report. Additionally, the city declined criterion-related validity research. The dissent, on the other hand, took other compelling expert opinion into account, and concluded that (1) ‘in truth’ the tests were flawed because of characteristics like arbitrary weighting schemes, arbitrary cut scores, and not measuring important constructs, and (2) alternatives were known and available. In theory, had the city done their ‘I-O’ due diligence, the outcome would have been different. The ruling will be reviewed in much more detail in TIP.

[Art Gutman]

On the same day as the Ricci ruling, the Supreme Court remanded Oakley v. City of Memphis for reconsideration in light Ricci. The Court stated:

"The petition for a writ of certiorari is granted. The judgment is vacated and the case is remanded to the United States Court of Appeals for the Sixth Circuit for further consideration in light of Ricci v. DeStefano, 557 U.S. ___, 2009 U.S. LEXIS 4945 (2009)."

Oakley looks like a weaker case than Ricci. In Oakly, Gerry Barrett put together a test for promotion of police captains to major. His validation report was extensive, and had a section in which alternatives were considered. Evidence was presented that the test was content valid. However, the Director of Human Resources refused to certify the test and the 6th Circuit ruled for the city based primarily on the 2nd Circuit ruling in Ricci.

I have a feeling that the "fun" has just begun.

Stay tuned.

[Tom Janz]

IMO, both irony (as aptly highlighted in Art Gutman's comment) and lamentation abound in this whole affair. The Supreme Court decision might well have been different had the city proactively sought out a strong-basis-in-evidence for the failings of the assessment procedure, instead of relying on the findings of adverse impact as the basis for failing to certify the assessment outcome.

Lamentations arise for the persons and organizations that come off looking bad and suffer avoidable losses. The city of New Haven and its officials look bad for having commissioned the development of a rigid and flawed assessment process (see the Amicus brief) and then rejected the assessment results relying primarily on the obvious adverse impact. A small additional investment in expert guidance up front, guidance that would have advised against the heavy reliance on written exams that tap crystallized job knowledge and arbitrary weighting procedures, could have avoided all the rancor and litigation expense so far and yet to come. The IO Psychology firm retained by the city looks bad for carrying out the out-dated procedures commissioned by the city and failing to include the critical skill of ‘command presence’.

Last and least, an IO Psychologist retained by the city to comment on the assessment process developed to their specifications (Dr. Hornik) looks bad. I leave it to the reader to see why, by quoting a paragraph directly from the majority opinion of Justice Kennedy—“And there is no doubt respondents fall short of the mark by relying entirely on isolated statements by Hornick. Hornick had not “studied the test at length or in detail.” Id., at A1030. And as he told the CSB, he is a “direct competitor” of IOS’s. Id., at A1029. The remainder of his remarks showed that Hornick’s primary concern—somewhat to the frustration of CSB members—was marketing his services for the future, not commenting on the results of the tests the City had already administered. See, e.g., id., at A1026, A1027, A1032, A1036, A1040, A1041. Hornick’s hinting had its intended effect: The City has since hired him as a consultant.” While few people read the full text of Supreme Court decisions, this characterization of Dr. Hornick does not bode well for the public/professional perception of IO psychologists.

One final comment. I find it curious that the distinguished authors of the Amicus brief joined with Dr. Hornik in vigorously advancing the assessment center method as the primary remedy for the ills of the written, multiple guess, job knowledge test-- brushing aside oral interviews as not much better than the written test. For another purpose, they cite the Gaugler et al. meta analysis of assessment center validity, but they seem to have overlooked the substantially lower population corrected meta-analytic validity for assessment center vs. structured behavioral interviews, as documented by Gaugler et al. and reported in the widely cited Schmidt and Hunter (1998) Psychological Bulletin review. While I am clearly biased, having written and worked extensively in the area of patterned behavioral interviewing, IMO we look better as a profession when we consider all the science, and not just selective science, in advocating a given view or action.

[Art Gutman]

As a starter, everyone interested Ricci must read the amicus brief submitted by 5 SIOP Fellows (Aguinas, Cascio, Goldstein, Ouutz & Zedeck). They highlight four major reasons why the test stunk (colloquial for invalid). IMO, they are 100% correct. Therefore, if the test was certified, and blacks sued on grounds of adverse impact ... they would have won hands down. Irony #1.

However, it was a disparate treatment claim. If you examine the district court ruling carefully, a reasonable person could conclude that the City would have certified had the results been different. Kennedy's ruling requires more than "good faith" that a test is bad – it requires a “strong basis in evidence” that the city would have lost an adverse impact challenge.

Hence – Irony #2 --- The SIOP fellows examined the test in detail, and provided more than enough detail to satisfy Kennedy’s standard …. AFTER THE FACT. The City needed this information BEFORE THE FACT. What they had was testimony by phone by a legit expert (though with an obvious conflict) that did not review the test, but spoke to the general issue of the virtues of assessment centers.

Irony #3 --- The evidence cited by the SIOP Fellows backs up his advice on assessment centers --- once again, AFTER THE FACT.

Conclusion --- a mixed message from the Court that says, they did it wrong here’s how to do it right [follow instructions from the 5 fellows]

[BryanB]

On balance a definite win for employers who take the time required to develop and administer appropriate exams. Also a wake-up that employers need to be 100% prepared to live with the results of their exam.