Do I-O Psychology and SIOP Matter?

Let’s Have a Serious Conversation about the Science-Practice Divide and Let’s Do Something About It

By Herman Aguinis, U. of Colorado Denver

The purpose of this brief blog editorial is to serve as a catalyst for serious action-oriented exchanges about the impact of I-O psychology and SIOP.  Does I-O psychology matter? Does SIOP matter? What is the impact of I-O psychology research on important human-capital trends? To what extent does I-O psychology research inform the work of practitioners? To what extent is I-O psychology research fulfilling SIOP’s mission to “enhance human well-being and performance in organizational and work settings by promoting the science, practice, and teaching of industrial-organizational psychology”?

My colleague Wayne Cascio and I have been thinking about and researching these questions in depth (For more information on this research, see Aguinis, H., & Cascio, W. F. (2008) here and Cascio, W. F., & Aguinis, H. (2008) here.) We reviewed 5,780 articles published in Journal of Applied Psychology and Personnel Psychology from 1963 to 2007 to consider this problem in greater detail. Our conclusion: “(I)f we extrapolate past emphases in published research to the next 10 years, we are confronted with one compelling conclusion, namely, that I-O psychology will not be out front in influencing the debate on issues that are (or will be) of broad organizational and societal appeal. It will not produce a substantial body of research that will inform HR practitioners, senior managers, or outside stakeholders, such as funding agencies, public-policy makers (including elected officials), or university administrators who control budgets.”

Wayne and I strongly believe that a change in course is clearly needed. Researchers can make conscious choices now to understand current and emerging human-capital issues more deeply, as well as the contextual constraints that managers face and the needs of organizational members, and to use their well-honed research skills to conduct research that addresses those trends and informs the debate over the relative merits of alternative positions.  Yet the changes needed are more than simply motivational.  Certainly the incentive structure of academic research is unlikely to be altered substantially in the near future, which could be a big impediment for change, given that performance management systems can shape the culture and orientation of organizations and entire professions. Similarly, some other colleagues and I addressed the issue (see Aguinis, H., Werner, S., Abbott, J. L., Angert, C., Park, J. H., & Kohlhausen, D. [in press]) by proposing a new way to report research results, labeled customer-centric science, consisting of three interdependent steps:

1. Set an alpha level (i.e., a priori Type I error rate) that considers the relative seriousness of falsely rejecting a null hypothesis of no effect or relationship (i.e., Type I error) relative to not detecting an existing effect or relationship (i.e., Type II error) and reporting the actual observed p-value (i.e., probability that the data would be obtained if the null hypothesis is true);

2. Report estimates of the size of the effect or relationship, which indicate the extent to which an outcome is explained or predicted; and

3. Report results of a qualitative study to gather evidence regarding the practical significance of the effect or relationship.

Our proposal to report research results with rigor, relevance, and practical impact involves important changes in how we report research results with the goal to bridge the science-practice gap.

What can researchers do to enhance the impact of their work? What can practitioners do to enhance SIOP’s impact? What can researchers and practitioners do together to improve the impact and influence of I-O psychology? Let’s get the conversation going!

Comments

[Ronald Kennedy]

As a practitioner of I-O Psychology who received my Masters in 1977 I would like to add my thoughts to this important discussion.

1) Industrial-Organizational Psychology is IMPORTANT. It is the basis for most management, HR, OD, etc. theory and practice. It is the hard science that underlies all aspects of human organizations. It needn't take a back seat to any business or management curriculum. And frankly, as it relates to teaching salaries, it should at least be on a parity with those of Business/Management schools.

2) I-O Psychology is NOT HR. I-O Psyc. permeates all organizations from the very top, including corporate governance, to the janitor. It stretches from the perception of stockholders to the reaction of consumers to a company's products and services. It stretches from disaster avoidance to disaster abatement (in the case of BP). In short, I-O Psychology is there wherever and whenever there is human-organization interaction. I-O Psychology is definitely NOT limited to things HR. This is a mind-set that we must eliminate.

3) Since I have been in the field, it seems that we (I-O Types) have an ongoing need to validate the importance of I-O Psyc. There is no need to do so,so let's stop waisting our time. I-O Psychology IS IMPORTANT.

4) Rather than continually trying to justify our fields' importance, let's change our tack. Let's spend our time informing others about I-O Psyc. and its impact on all aspects of human endeavour.

5) Regarding the science-practice gap-As a practioner of I-O Psyc. in the world, I would find it useful for all research published in appropriate journals to be coded with icons indicating whether or not each manipulation has practical implications for practitioners like myself. Perhaps the researchers could attach the icon, providing such a coding system was agreed upon and publicized through SIOP. It would have to be standardized.
Perhaps the icons could be displayed next to the research in the table of contents so the research could easily be found and assimilated. This system would help practioners identify, read and apply current thinking on I-O Psyc. topics.

Last) There are many of us in SIOP who have Masters Degrees as a terminal practice degree. Even though we possess terminal practice degrees in I-O Psyc, we cannot be licensed to practice Psychology, nor can we call ourselves I-O Psychologists, not having Doctorates. So, what do we call ourselves--HR Consultants, Personnel Consultants, Management Consultants, etc.??? These titles are "watered down" at best as anyone can call themselves any of those titles. It seems to me that there should be some agreement as to what we Masters level practioners can properly title ourselves--and it should include "I-O Pschologist"--for example, "Masters Level I-O Psychologist" or some other appropriate title.

I would be interested in any and all comments on the above points.

[Gary B. Brumback]

Dear Herman and Wayne,

I came to the same conclusion many years ago.

Please take off your research blinders and think conceptually about what organizations, institutions, people who work for and with them, and all of the foregoing's broadest context.
If you do, I believe you will see that the only way organizational psychology and, for that matter, the whole field of psychology, to really be of value to organizations, institutions, people, society, and the environment is to broaden the educational requirements for entrants into our field and also to build interdisciplionary bridges to other fields that are wearing no blinders or smaller ones.

SIOP needs to revamp its education and training curriculum guidelines and to create committees to build those bridges!

[Jayne Speicher]

I am a practitioner as well; I think this topic is of critical importance to the field and have suggestions to the three questions posed. The first as to what can researchers do – begin by doing systematic reviews or syntheses of the research already executed. These reviews, used in medicine and other evidence-based disciplines, can provide practitioners and fellow researchers with the current state of the evidence on a practitioner-oriented topic. Recently on the APA Division 13 (consulting psych) list serve the question was posed "what works in performance management?" This is the type of question an I/O systematic review can address by comprehensively pulling together what is known from quantitative and qualitative studies, unpublished dissertations, etc. and then providing responses at different levels of detail – from the 5th grade reading level quick-read through academic journal-ese.
As to the second question what practitioners can do – become champions of evidence-based management practice. Scientist-trained practitioners understand the research data, are aware of the needs of organizations, and have the ability to blend our experience with the first two. These are the grounding principles of evidence-based management practice.
The final question – what can we do together? – take advantage of what we know about the diffusion of innovation, stages of change, and social networking concepts to generate customer demand for evidence. Pushing out more research, no matter how customer-centric, will not yield greater uptake if there is no demand for it from our end-users (HR specialists, managers, business leaders). The key to changing demand is employing strategies that build motivation in end-users to take action based upon I/O research. We must make end-users aware of the danger of being poorly informed of relevant research when making management decisions.

[Eva]

As a practitioner, I agree with many of BryanB's points. Research needs to be aggregated and simplified - drastically (i.e., write for a 5th grader audience, your grandmother, etc.) Eric and Lorin make excellent points and suggestions in their comments.

Practitioners can do their part by educating others and addressing incorrect assumptions gracefully when they arise, as George has done.

Lastly, (and this is a personal bias) I think both researchers and practitioners can branch out from HR rather than trying to fit into HR's little box. Yes, it's a great career path for some, but I think there is a role for us to make impact in all aspects of the business. You see this more in OB than in I-O, but it's still relatively rare.

[Tom Baker]

Herman,
It's good to hear of I-O's continued interest in staying relevant to HR. It's sad to see an analysis of 5,000 (past) articles indicate a (future) trend of irrelevancy. Man, that sounds like a lost decade to me. I never understood how I-O got so sidetracked.

Tom Baker
HR Kraft
Richmond, VA

[George Guajardo]

I have recently engaged in a similar discussion with a number of HR professionals. It is becoming clear that scientific rigor has little to do with the value academics can provide practitioners. The sad fact is that many people who are selecting organizational interventions do not have any confidence in the utility of psychology in the workplace.

If you are interested in what practitioners are thinking, check out this blogpost along with the entries it refers to.
http://jungleblog.foresightint.com/2009/04/14/is-psychology-a-science-yes/
It will be pretty educational.

[Eric Williams]

I see an important distinction between behavioral sciences and physical sciences that no one has apparently touched on (in this instance of blogging). In the physical world theory and practice are virtually synonymous. I am no expert in physics, but it appears that the forces of the world are, as they are, unchanged and we can only discover cause and effect. Once we discover such things, they can be applied to vehicles, buildings, cell phones…whatever. Consider pharmaceutical development and cellular biology. There is not really a “make-shift” way to address cancer. As such, the science precedes any sort of solution to the problem.

In I/O, or HR practices, it is most often the exact opposite. We hire someone, reward them, promote them, train and then fire them…all without the aid of science. The science provides “suggestions” on how to do things best. The practice certainly precedes the science. People were organized in hierarchies long before we called them tall or flat…humans were resources long before Human Resources. Our structure and knowledge of the field is always several steps behind what is actually happening. But, there are always exceptions to the rule…none that I can think of at the moment, but I am sure they exist. *edit-the EEOC, ADA and other similar bodies create a set of rules that all people must abide by, where changes must first be instigated by legislature or research prior to the implementation in practice.

From this perspective, if I/O academics want any particular weight or recognition from the relevant world, their research must be quick, proactive (if possible) and aimed at tangible outcomes. Either that or the academics must figure out a way to create circumstances on which the real world waits on their results before action is taken. *I see few opportunities to recreate the impact of the EEOC.

I certainly do not wish to discourage the pursuit of any idea, applicable or not, I am simply suggesting that in order to pull up a seat to the world table of politics, business or law, I/O research must keep abreast of what is happening in the world as opposed to debating antecedents of job satisfaction.

[Lorin Mueller]

Sorry. link:

http://www.washingtonpost.com/wp-dyn/content/article/2009/04/12/AR2009041202260.html

[Lorin Mueller]

Here is an interesting Washington Post article on a similar topic - the fact that academics aren't influencing the public policy debate and aren't being appointed to important public policy positions.

If academics are going to get serious about doing relevant work, they need to get serious about finding out what kind of work would be relevant. I don't think fixing the alpha level or other methodological concerns is the problem. The problem is that practitioners just don't care about what is being published. Open a JAP or a PSpych and ask yourself, "if I tell one of my clients the findings in this journal, would they do anything with that information?" In too many cases, the answer is either, "I don't know," or "probably not."

Here are some suggestions to address the problem:

1) Expand "Industrial and Organizational Psychology: Perspectives on Science and Practice." This is easily our best journal, in my view, and should have 1-2 more articles per issue. Perhaps we could add a couple of more focused "best practices" pieces with shorter commentary.

2) Include more practitioners on editorial boards - consider people who contribute to the feild in ways other than publishing empirical studies. Maybe they would just rate articles for impact rather than methodology... but my guess is there are a bunch of people out there who can look at both ends of the spectrum.

3) Using the new editors, reduce the journal publication backlog. By the time something is published, it's already irrelevant. I suggest literally reviewing things that have already been accepted and figure out which ones meet the new criteria. For the ones that don't, either send them back to the authors or publish them online in a couple of special issues. Then make sure anything you accept gets published no later than the following year.

4) Most importantly, quit hiring people right out of grad school for new faculty positions. I know there are a lot of folks who can step in and do great research, but I think they would be better informed by serving in the government or doing some consulting for a few years, then coming back to academics with a hatful of impactful ideas and connections to people with real issues to be addressed.

[BryanB]

1) Get more enmeshed with practitioner groups and publications. The outreach to SHRM is a great example.

2) Be open to allowing business challenges to drive research focus.

3) Simplify our communications to the general public, then simplify some more. Most people don't know what a p-value is.

4) Publish easy-to-digest, targeted, short summaries of I/O issues for practitioners.

5) Provide training for practitioners. This includes webcasts as well as in person. One of our biggest strengths is our classroom interaction; why isn't this environment more frequently shared?

6) Develop a plan for training and educating the HR workforce, now and in the future.