The Lost Potential of Institutional Research: Insights from 'Outsourcing Student Success'

Alex Usher: Hi everyone. I'm Alex Usher, and this is the World of Higher Education podcast.
Inside every higher education institution lies a secret cabal of people — gurus really — who know everything about the institution and how it works. They're called institutional researchers. And yet, despite all this specialized knowledge, the field (it's not really a profession) is not usually at the heart of university decision making. Why is that exactly? Today, my guest is Joseph Wycoff. He's the author of 'Outsourcing Student Success, the History of Institutional Research, and the Future of Higher Education'. It's a fascinating century long view of the development of a field of endeavor, which is specific to higher education, and which you might think has been central to its political development as well. But the story Wycoff tells about this group is both an odd and surprising one. Though we live in a society where virtually any occupation can be, and usually is, professionalized, the field of institutional research, or its leadership in the Association of Institutional Research at any rate, took a voluntary vow of deprofessionalization about 60 years ago, renouncing the idea that what they did was science, and instead claim to just be an art, one that needed to be bespoke to each institution.
This seemed to me to be a very unique story, not just in higher education, but anywhere in the modern economy. Who avoids professionalization? The strange story of how this deprofessionalization came about, and what its consequences were, is at the heart of the interview you're about to watch. And from here, I'll let Joe take it away.
So, Joe, let's start at the beginning. How did offices of institutional research begin? What's the origin story of this rather peculiar you know, area of institutional life?
Joseph Wycoff: Well, I'm glad you asked about offices rather than institutional research in general, because there was a narrative, I call it the consensus paradigm, that tries to trace institutional research to the entire history of universities and particularly back to Yale and some self studies that they had done. I think the Offices of Institutional Research, as we know them and understand them today, actually originates at the University of Illinois in 1918. So it's about a hundred years old. There may have been others. It was actually called the Bureau of Institutional Research. It was led by a person named Coleman Griffith, who's regarded as the father of sports psychology, because he once, well, he studied psychology, was a scholar of psychology and actually worked with the Chicago Cubs for a short period of time. But largely his history had been lost in the telling of what institutional research was for about 50 years after the organization of the Association for Institutional Research, and this other narrative came about how institutional research has always been around or originated as soon as Yale was formed. So I think that's, you know, where I would draw the line.
Alex Usher: And then the system, or sorry, these offices grew and grew, I think, like through to the 1950s and 60s, that period of big massification in American higher education. And you recount how they played a very big role in the creation of these state systems of higher education. And it seemed at that point, like institutional research offices might become quite central to the management of universities. Is that about right? When was the high point of institutional research as a discipline?
Joseph Wycoff: Uh, well, I think it never really became a discipline, but the height actually like through the thirties, I think there were some references to maybe there were about a dozen, maybe two dozen offices of institutional research. Coleman Griffith, I mean, he started the Bureau at the University of Illinois in 1918 or, associated with this formation. He first published about it in the mid 1930s, and then due to the depression, due to the war there wasn't a lot of momentum for more institutional research offices until after World War II and then the period of massification as you refer to it. There was another publication out of the University of Minnesota by a scholar I think named Ruth Eckert and she should be regarded as the mother of institutional research and that really spurred interest once again in what these offices could be doing at the institutional level. And I would say from there the work of institutional research offices really started to steamroll, and I think it was really integral to what happened in California and the California system. Before there was a 1960 master plan, there was a series of research studies and research studies and research projects that were going on that were led by individuals from the University of Minnesota and others who had been really integral to building institutional research at a state level, who participated in, contributed to, and really kind of set the parameters for what would become the master plan. And so in my estimation you know, that really is a significant, step forward for institutional research.
Alex Usher: So after that, I mean, in you're telling, one of the things that, that seems pretty clear is that as the sixties wore on institutional research, came to be viewed with a certain amount of suspicion by the faculty. You know, that, IR is sort of the high modernist version of institutional management, right? You can manage everything through, through figures. And I think, the impression I get is that faculty became very suspicious of this because this was an alternative source of authority about what higher education was supposed to be or what it could be was supposed to look like in a managerial— and so they start to become suspicious of IR. How do IR offices react to that?
Joseph Wycoff: So, so that's difficult for me to actually speak to because I didn't go and do research on individual IR offices during the period. What I focus on is the literature about institutional research offices. So it's hard for me to ever draw a line or conclusion about institutional researchers. There is some acquiescence, obviously, but there's also a level of ignorance. There's you know, these offices are just coming into existence at the time and the people are going into these offices don't even really know the history of it. And then one of the major themes that occurs in the backlash to it all, to institutional research, is that the predominant well, the consensus paradigm, this is what I keep calling it, it's kind of redundant, but the consensus paradigm you know, says that institutional research is not a profession, that institutional researchers don't need a discipline. Institutional researchers learn what they do on the job. And so there's this real effort by a small group of people who position institutional research as a threat to the institution.
And I, I see this too today. There are individuals who are very knowledgeable about what makes the science, but they deploy what their understanding of science in order to undermine or discourage people from thinking of institutional research is potentially science. So, so institutional research exists from a later chapter in this pre paradigm state and never actually, you know, further develops in that regard. Now, just so it's clear, I don't say it's only faculty. There are administrators as well, who are not interested in having institutional research become any kind of authoritative voice about what goes on in higher learning.
Alex Usher: Well, that's my next question, because I think you know, what strikes me is, labor and management never agree on anything in higher education, and yet they both seem to have agreed that they didn't want IR to become — the IR offices — to become a source of authority about how higher education works. So, so why wasn't management more keen on having this piece of, you know, high modernist intellectual machinery to help them run places.
Joseph Wycoff: You know what, that's probably the one question that I struggle with the most because it was really these offices of institutional research were organized under presidents. Presidents wanted these offices and had initiated their existence essentially. So then at the height of the activity and the capabilities of institutional research, there's this recoil from, you know, like actually pursuing it any longer.
I think part of it is that institutional research was once regarded as self study, and as soon as institutional researchers took it upon themselves to publish results, actually turn it into something that was you know, up for discourse then it, you know, it escapes. It's outside what, they call the walled city of the university. And so, at that point, you know, it's not just, something that's under the control of the individual institution, it's potentially something that's out of their control because, you could learn to say like, what's the optimal path to take essentially, and how you run a university.
And so what the faculty, typically faculty who are writing, and I mean scholars of higher education, those individuals, scholars of higher education who actually are trained in other fields, what they're actually arguing is that institutional research represents a threat to the entire institution. And so what they actually do is kind of marshal this, you know, sort of a reconciliation that you're talking about between administration and faculty, and to say that institutional research is actually an instrument of the state and of state control. And so there are many, there's several different authors who kind of, like, lay this out in in a way that kind of suggests that institutional research is a foreign agent operating inside of higher education.
Alex Usher: So a threat to autonomy more than anything else.
Joseph Wycoff: Well, autonomy, I know, is, like, something that you've addressed in your podcast, and autonomy is something that I address more fully, and I have two works, actually, on this subject. Autonomy, I mean, it doesn't really even exist until after, you know, the 1950s so it's like, it's a it's actually a post hoc explanation for why universities need to be protected from the state and all these, you know, like, external agencies essentially outside the walled city of the university.
So, I mean, institutional autonomy is part of what I regard as sort of like a un, acknowledged, you know, litmus test for being regarded as a scholar of higher education. And that litmus test is, do you regard every institution as unique? No one defines unique, no one can measure unique. It's like, it's entirely just sort of something that's thrown out there that's intended to stop the idea that we can even think about institutions and their commonalities, that we can do a scientific study, that we can regard them as having any kind of like, you know, nomothetic, I think one of the writers said, or Wall Lite, kind of like aspects to them. There's no point to even trying to create a system. There's no even point to trying to manage one institution, essentially. Because every institution is unique. And then beneath the uniqueness of the institutions, another thing that really is kind of at work and what faculty are really concerned, is that academic freedom, is also in every discipline. Every department is unique. So once you continue on with this concept or this like sort of proliferation of uniqueness at every level of the institution, then there's actually no reason to go into studying it. And most of the scholarship, I mean, if people went back and actually read the scholarship from 50 years ago, you'd be surprised at how often these authors are actually like being anti intellectuals and trying to discourage thinking about higher education. They just want it to stop. They want it to, you know, like just sort of come to a standstill. And again, institutional research, institutional researchers exist in a pre paradigm state, because this is the consensus paradigm.
Alex Usher: We're going to take a short break. We'll be right back.
Advertisement: This podcast is brought to you by Higher Education Strategy Associates, and we’re pleased to be hosting a major national event on AI and higher education: the AI-CADEMY: Canada Summit for Post-Secondary Education, which will be held March 6th-7th, 2025 in Calgary, Alberta.
The summit will engage with key themes such as the impacts of AI on the future of learning and of work; the interaction between AI and teaching and learning, research, and operations; and the development of institutional AI frameworks, and will include immersive opportunities for attendees to interact with the latest products, services and technologies from leading industry partners.
Tickets are selling fast. Register today and join us! Visit higheredstrategy.com or see the episode description for links.
Alex Usher: And we're back. Okay, so in your telling, basically I mean, the impression I get is that IR profession, I know professionals is a weird word cause they eschew professionalism but practitioners really they do self abasement. They try to bargain with fact with faculty and I guess to some extent with management — We'll pretend we're not a science and you help us get our data to do our reporting and our, you know, the kind of stuff that we need for rankings and accreditors and those kinds of things. Is that a fair description of the bargain that IR people kind of made with themselves and if so, did it work? Like, was that, did they at least get the bargain they thought they were getting?
Joseph Wycoff: Uh, yeah, so again, I mean, it's hard to speak for institutional researchers as, you know, a group of, like you say, professionals or even practitioners, because, they're individuals who are, you know, basically having to adapt to situations at a local institution because of the way this, understanding of institutional research came about the line, the argument is that an institutional researcher needs to spend up to, like, 20 years at an institution before they actually become effective at what they do. So they only know one institution. And what does that mean? They have to ingratiate themselves with that institution. They have to do everything they possibly can to be there for 20 years. And like at this moment in time, I'm not a member of the Association for Institutional Research. I don't believe it represents my professional interests. I don't actually work full-time as an institutional researcher, I take interim roles, and I go around the country for maybe, you know, like each year, try to get like one post, stay there for about six months, and I see the same things over and over again. And these individuals who give up like 20 years of their life to one institution under this argument you mentioned they're not ever highly respected. It's like at the very end of their lives, they can retire and there's no party thrown for them.
Again, when institutional researchers were reaching out to each other and publishing for the first time, they were behaving as scholars. They were doing scholarship and they were leading the way and making themselves into a scientific community. But it was thwarted. It was just cut off and it all begins in 1965 with the formation of the Association for Institutional Research.
Alex Usher: Well, okay. So let's talk about the Association for Institutional Research. I mean, one of the things that struck me as I was reading it was, and I, I guess I just never thought very much about it before I said, well, it is interesting how different the conversations are at AIR versus say the Association for the Study of Higher Education, right? And we have similar, in Canada, we've got SERPA and the CSSHE, and in Europe they've got European Association of Institutional Research, and the Conference for Higher Education Researchers Chair. So there is a divide everywhere between practitioners, the people who actually get the data on higher education and who know their institutions and the so called scholars of higher education. Do you think that splits necessary? And who loses more from this, from this divide?
Joseph Wycoff: So I can't speak to what goes on outside of the United States. In my understanding and telling, I believe these offices originate in the United States and United States provides an example to others around the world. The whole entire narrative about why American higher education is what it is driven by this, this contest in the United States between scholars and institutional researchers. My sense is that this actually came about because scholars of higher education wanted to define their field of study as a profession. They wanted themselves to be regarded as scholars. They had to demonstrate that they were scholars. They had to do something about it. And one of the ways in which they did it was to cut off institutional research as an area of study. Who does that hurt? It hurts college students. I mean, obviously, it's beneficial to the scholars of higher education, but, and it's detrimental to the institutional researchers in this country, but really, the most important thing is that there's policymakers institutional leaders, et cetera, just don't have good information about what, why colleges work, how colleges work.
Alex Usher: And one of the areas where I think, you know, there, there could in theory be some pretty easy wins, and maybe we saw it through the work of George Koo and Nessie, is the ability to do, you know, inter institutional comparisons. And I think, the impression I get from your book is that, you know, in that rush to say, oh you need 20 years at one institution to do anything, as you say, people become very focused on their own institution and not that interested in doing good comparative work, right? Like I think this is not just the United States, this is in many places, there's not as, IR offices do not do as much good comparative work as they could. Was that necessary? I mean was that de emphasis of comparative work a necessary corollary of the of the path that the that the field took?
Joseph Wycoff: Well, I think first of all comparative work is obviously one of the threats for, you know, having the kind of external agency able to look at what institutions do. But I think that it would be wrong to say that it doesn't occur, because there's all sorts of voluntary you know, inter institutional sort of comparison still there's these, sort of compliant ones where you have to submit your data to the U.S. News and World Report so they can make a few bucks off of all this. I mean, IPEDS demands that, you know, we turn over institutional data in order be eligible for Department of Education in the United States, demands that we turn over data to the integrated post secondary educational data system in order to get money for financial aid. I mean, there are, so many different ways in which there are institutional embarrassments, but even then, I mean, you mentioned
Alex Usher: But those come through compliance measures, right? Like they're not
Joseph Wycoff: They're not driven by institutional researchers. In fact, they're actually considered to be burdens that are placed upon institutions, you know, in order to be eligible or regarded as institutions of higher education, right?
So it's like none of this is actually being driven by a, well let me pull that back a little bit. Obviously in the United States, in the Department of Education there's a group of scientists who are publishing educational data, right? They're, they are statisticians. They're doing work. Institutional researchers are supporting them. And when I work in their office, I mean, I have this sort of like whole theme about how our profession can be a science and I must bemuse like the people I work with because I can't deliver it for them. No one person can become a scientific community. You need actual scientists to actually help you. And then the, people that I work with locally who are dedicating their lives to doing this, I tell them, you know what, maybe we're never going to be more than scientific technicians, maybe we can't actually form ourselves into a scientific community, but we deserve a scientific community that respects us and treats us as members of a community of scholars. We don't have that and the reason we don't have that is because of an Association for the Study of Higher Education and others that don't recognize institutional research as a productive line of inquiry.
Alex Usher: Okay. Last question. As I was reading your book, there was one other group within universities that, that popped into my head and that's librarians. Now, librarians are not academics. They get treated like academics, though, like they're treated as part of the scholarly enterprise, their tenure, they're often in the same bargaining units as the rest of faculty. Institutional research could have done that, right? Like, I mean, you could imagine a world where people sort of said, actually, the institutional research is such an important part, understanding what we're doing is so important and this is, you know, this is a science and there are, you know, it has a scholarly community or a para scholarly community. I mean, imagine let's go back to the 1960s and say some other path was taken and IR would you know, was treated like libraries. How do you think universities would be different? How do you think the profession would be different?
Joseph Wycoff: Well, I think obviously that'd be transformative and I don't want to say that there wasn't ever like another option or opportunity, you know, history isn't determined. It doesn't just like go in a straight line. There were people who actually really laid out as I lay out in the first few chapters of my book, what institutional research as a science could actually be or become. And there were advocates for maybe not for tenure, but for the recognition of institutional research as a fruitful line of inquiry of fruitful scientific, you know, endeavor, essentially. So I think that exists essentially. But again, it's like, I keep coming back to over and over again how any kind of like effort to even posit that gets, you know, suppressed.
And I just want to be clear about something. I'm self published, so anyone who's reading my work, if you see grammatical errors, I apologize for that. But I three times tried to present at the Association for Institutional Research because I was opposed to what they were doing with their aspirational statements. And in all three times I was rejected and primarily without comment. I mean they wouldn't even provide feedback to me about how I could actually get myself published. Then I did try to publish it through John Hopkins University Press and I got back a, a review that was, you know, just blistering and sophomoric, I thought, so, whoever wrote it. So, you know, it's like it was like, I mean, I have had extraordinary difficulty getting my ideas out to the public space. I have met, you know, the leaders of the Association for Institutional Research in Europe at the European Association for Institutional Research. Not once have I been asked to talk about my work. Not once have I been asked to contribute to their professional journals. Not once, I mean, have I been asked to even participate in the direction of the Association for Institutional Research. Since the publication of their aspirational goals for the profession, since my publication, the individual membership in Institutional Research in AIR has dropped by 50 percent. And there's not much concern about that. They did change things up, so now they can have institutional memberships, but they're just taking their idea that everybody does institutional research to its next logical step. And that's the further end of the profession.
Alex Usher: I've been speaking to Joe Wyckoff. He's the author of 'Outsourcing Student Success, The History of Institutional Research and the Future of Higher Education'. Joe, thanks very much for being with us.
Joseph Wycoff: Thank you. I should have said up front. I really appreciate the opportunity to talk about my work.
Alex Usher: And it just remains for me to thank our excellent producers, Tiffany McLennan and Sam Pufek, and you, our viewers and listeners for tuning in. If you have any questions about today's episodes or suggestions for future ones, please don't hesitate to contact us at podcast@higheredstrategy.com.
Join us next week when our guest will be Sharowat Shamin. She's an assistant professor of law at the University of Dhaka, and she'll be joining us to talk about Bangladesh, its student movement, its graduate labor market, and the summer riots that ousted a prime minister. Bye for now.