Responding to Online Entrance Exam Cheating at UNAM

Alex Usher: Hi, everyone. I'm Alex Usher, and this is the World of Higher Education podcast.
Higher education dominated the front pages of Mexico's media for most of this past summer. The reason? Universidad Nacional Autónoma de México, the country's largest, most prestigious, and selective institution, better known as UNAM, it put its entrance exam online for the first time. 158,000 students took the exam, and the results, well, they weren't good.
As the exams were marked, it became evident that there were an unusual number of extremely high, near-perfect scores. Suspicions of cheating were naturally aroused. The possible vectors for cheating were numerous. The test was taken over a period of a number of weeks, which provided one outlet for chicanery.
The test was also not especially well invigilated, which raised questions about whether students had used AI or received other forms of assistance from persons off-camera. But proof of cheating by specific individuals was hard to come by. It was the test administration in toto which was called into question.
Amid widespread criticism, UNAM did something extraordinary. It called together an independent commission to try and work out what to do, and that panel's recommendations was a partial rerun of the exams. You can only imagine how the media and politicians reacted.
With me today to discuss all this is The Pod's regular Mexico correspondent, Dr. Alma Maldonado-Maldonado. She is, as many of you will remember from her previous appearances, an education researcher at the Centro de Investigación y de Estudios Avanzados del Instituto Politécnico Nacional, otherwise known as CINESTAV. She was also one of the externally appointed members of UNAM's commission investigating the test schmazel, so she's uniquely well-placed to explain just what went wrong and how UNAM has been trying to put things right.
There's a lot of blame to go around in this fascinating story, but I was really struck by a point that Alma made towards the end of the interview. We live in a world where taking responsibility for failure is out of fashion. In some countries, taking responsibility is never in fashion. Alma pointed out that at UNAM, at least, there was no attempt to try and bluff this one out.
There was a problem, the institution admitted errors, its secretary general made an honorable decision to resign, and they got on with trying to fix the situation. And in Mexico, none of that was necessarily a given when the story broke in July.
But enough from me. Let's hear from Alma
Alma, welcome to the show.
Alma Maldonado-Maldonado: Hi, Alex. It's very nice to be back.
Alex Usher: Great. Listen, explain to us a little bit about the background here. Why does UNAM have its own entrance exam? Do all universities at Mexico have one or is it just prestigious universities like UNAM? Like, why isn't there a national system like the, like the SATs in the United States?
Alma Maldonado-Maldonado: Yeah, we don't have any national system that's historical. And also because our universities are autonomous, so that's very important. So each institution has to decide what is the best method for the entrance. Now the thing is, UNAM together with other very important public state universities they are very demanded, right?
They they have a high number of um,
Alex Usher: Applicants, yeah.
Alma Maldonado-Maldonado: So, in this case they have to apply exams. There are different systems, like UNAM, it has its own high school system. So there is a way to pass from the high schools, UNAM high schools, to the undergraduate programs.
That depends on the time you spend in high school, your average, also your, program, the program you want to enter. So there are different requirements. So that's one way, and that's the most important way to get into the undergraduate UNAM programs. The other one is the exams.
So, basically, that's the situation, and every institution has different ways.
Alex Usher: Right. Now I understand this year's entrance examination was the first one that was conducted fully online for all students. I think there was a pilot for evening students or, or part-time students the year before. What was the motivation for going online? And how was the move to online managed? That's a big shift.
Alma Maldonado-Maldonado: Yeah, this was the first one for undergraduate and for the whole population. Previously there was for online programs. That was kind of a pilot. And before that, it was for high schools. That was the big change because there was a policy change in Mexico City, so, they had to apply together with the IPN uh, Polytechnic National Institute.
So they have to put together the exam, and the IPN was doing it online, so they kind of invited UNAM to join them in this online examination, so that was the, the whole point to change. And this year they decided to do it for the undergraduate program.
Alex Usher: And I suppose that would have been a big boon to students who are from, I don't know, Chiapas or, you know, a long way away. They wouldn't have to come to Mexico City to do the exam.
Alma Maldonado-Maldonado: Yeah, it was, although there were regional, you know, UNAM uh, units, so, they were conducting also exams, but they were in a very minor scale. So the whole idea of going online, yeah, opened the, you know, the possibility for students even abroad. There were students in the US or other parts in Europe that wanted to apply online.
So yeah, that created that possibility.
Alex Usher: And so the exam took place over the course of three or four weeks. I guess if you're doing it online, you, you can do it whenever you want really. You can sit, you can start as long as you've only got so many hours to finish the exam. 160,000 students took the exam between late May and mid-June.
What happened next? How did suspicions of cheating materialize?
Alma Maldonado-Maldonado: Yeah, I think there were so many problems at different stages and different responsibilities. So at the beginning, they decided to, to have this exam in different weeks. I think that was a problem because, of course, if you are doing this online, well, it created the chance to cheat and to, you know, pass the, the questions, the items.
So that was one problem. And then I think realizing that there was a problem with the results didn't create the adequate response at that moment, right? It took a lot of time to respond to something that was pretty obvious. There was a problem, and they had to fix it, and, you know, it took time.
Alex Usher: What was it that made it obvious?
Alma Maldonado-Maldonado: Yeah, I think the results.
I mean, if you see the statistical results, the first results already presented many more students answering correctly the exam, right? So the curve started moving to the right,
Alex Usher: Mm-hmm.
Alma Maldonado-Maldonado: To more perfect exams. That was another indication that clearly there was a problem. So I think they didn't read the signs or it took time. I mean,
Alex Usher: It took about a month, right, before there was actually action on this is my understanding.
Alma Maldonado-Maldonado: Yeah. Also, UNAM was on vacation, so I think that that was the other factor that they didn't respond on time.
Alex Usher: But that sounds just like suspicions of cheating. I mean, there's, it seems to me like there's good reason to suspect that something was going on, but how do you prove cheating in a case like this? How do you distinguish between a legitimate high score and cheating?
Alma Maldonado-Maldonado: Well, yeah, we couldn't as a commission when we were doing this job. What we can tell is there was a problem with the result, and there are irregularities, right? Something is wrong as a whole. Now, you cannot detect individually who was you know, a legit person answering the exam and who is not.
So we as, as the commission, we couldn't blame anyone in particular. But what is clear is if you see these tendencies, statistical tendencies, it's not normal, you know, unless something very unusual happened, like suddenly students are smarter, or suddenly the exam is easier, or suddenly something like that happened, and that's not possible, you know?
There is no logic explanation about that. So yeah it's impossible to detect individuals, but as a whole, you can tell there is something wrong.
Alex Usher: There was a lot of speculation that students were using AI during the exam, and that maybe was what explained the different results. Did you ever get any sense of a problem with the security of the exam that way? Like, normally if I take an exam, I have to be in a particular program, and I can't leave that program while I'm taking the test, so that would make it harder to use AI.
Did, did you ever get any, any evidence about that kind of thing?
Alma Maldonado-Maldonado: Yes. I think the whole analysis that the commission did included so many ways to cheat in the whole process. Like historically, there has been this knowledge about the questions, and there is this traffic, illegal traffic, of course, of questions. That's historically.
You have that. Then there were technical difficulties to really capture the whole space where the exam took place. Let's say I am looking at you at this screen, right? But you don't see what am I, what am I doing with my hands. It was the, exactly the same situation with the exam.
You didn't have an extra camera looking at the whole surroundings. So that's another problem. Then there were issues with let's say not cheating the whole exam, but you can cheat in some questions. And in this exam, some questions are crucial, right? If you answer correctly 10 questions, then that probably will give you the pass for UNAM.
So, there were a network I would say, of, of ways to cheat in this case that it is not only AI or it is not only the traffic of the questions or answering with the phone or, you know, sending the questions to other person. But there was another way. There was another person answering the exam for me, and that's a surprise for the platform.
So they created a, a second camera, and you have a person here pretending answering the exam, but in reality, the, another person behind you or in front of you was answering the exam. It was really crazy.
Alex Usher: Okay. We're gonna take a short break. We'll be right back.
And we're back. Okay, so obviously in the aftermath of this discovery of anomalous results, the university has to decide what to do. What was the decision that was taken, you know, as a result, I understand you had to have a retest of some kind?
Who was asked to take that retest?
Alma Maldonado-Maldonado: Yeah, the problem is that since you cannot blame anyone in particular this commission decided to look for a second exam, inviting all the students who passed the first exam and all the students who were close to pass, looking at the, points, at the historical points from five years ago,
Alex Usher: Uh-huh.
Alma Maldonado-Maldonado: up to now, and then they were invited to also do the exam.
That means people who were studying, were preparing, but because this inflation of grades, they couldn't pass. So, they were invited, these two groups, to present the exam.
Alex Usher: And I mean, obviously there were then some protests, I understand, from students who'd got in, you know, and presumably some of them had gotten in without cheating. They said that set retake was unfair to them. Was that expected? How was that opposition handled?
Alma Maldonado-Maldonado: Yeah, it was expected, of course because it was unfair in many ways of, you know, if I prepare, I study, and then I have to retake the exam three months later. I mean, that was very stressful and difficult for families. Some people were asked to come to a place because that was the condition of the exam.
Alex Usher: Mm-hmm.
Alma Maldonado-Maldonado: It was presented electronically, but at a place where you can see what people are doing. So yeah, there were protests, and as we expected, a lot of people didn't present. And you know that you can figure it out, or you can speculate why they didn't present. But, you know, some people didn't show up because they supposedly they were cheating.
And that would be, you know, very hard that you say to your family, "Oh, I, I passed. I was okay, and now I present the exam and my points collapsed from the first exam." That would be very strange to explain.
Alex Usher: I, I wouldn't wanna do it with my parents. So how did the reexamination go? How many did take the exam? And I mean, how many people in the end lost their place because they didn't do well on the retest?
Alma Maldonado-Maldonado: Yeah, as you said, there were 20,000 spaces available, and the first exam was taking uh, about 58,000. And um, it was 38,000 people who presented the second exam. It was called a control exam.
Alex Usher: Mm-hmm.
Alma Maldonado-Maldonado: So, yeah, that there was a lot of... A- and they were invited 58,000 to present.
present. So a lot of people didn't show up. I have
Alex Usher: okay. That's a lot.
Alma Maldonado-Maldonado: I, I have to say there was a previous control exam. Everything that I am telling you, I mean, we studied the whole case.
And the previous control exam was taken by students who scored the highest points, and they were called to present again the exam. That was an exercise that UNAM thought at, at the beginning that was going to solve the problem. Of course, it didn't solve anything. But in this exercise, a lot of people show up.
Not too many wanted to present the exam, and the collapse of the points from the first exam to this exercise was a lot. You know, people take people who supposedly, you know, got
Alex Usher: Yeah
Alma Maldonado-Maldonado: 120, and now they got 40 points. No explanation about that, right? And, and es- and especially because that exercise took place before, right before this second attempt that UNAM presented.
So we had enough elements to suggest that there was a huge problem in terms of the application, in terms of the cheating, in terms of other strategies because the market is very high, you know? The demand to go to UNAM because it's free, because it's the most important public university in the country.
So the demand really has exposed this huge market for looking ways to enter. I mean, I think that explains a lot why the whole thing was out of control.
Alex Usher: I think you just mentioned one of the stipulations of the reexamination was not just that this year's test takers had to retake the exam, but some previous years with high scores were also asked to retake. What was the thinking there?
Alma Maldonado-Maldonado: Yeah. No, it was more to consider those points, you know? We try to be inclusive. So we said if we just take this year, this year is not a normal year, right? So we need to take other points previous years and that was in a way fairer for the students in order to be inclusive and to try to take the exam again. That was the whole point of doing that.
Alex Usher: And so these were test takers who had taken the exam but not passed? Like they were on the line?
Alma Maldonado-Maldonado: No, they were, they were in this year. I mean, they took the exam this year, right? But the points to consider to retake the test, was we expanded five years the points. Because every year you can see the points basically don't change that much. I mean, that was a clear indication something wrong happened this year, because the point go- the, the inflation of the point was extreme.
So we, we tried to take who is going to have the right to retake the exam. So we went five years ago to see what were the points, which pretty much they keep steady. So we took those points to make sure people that could have entered three years ago, they were able to retake the exam now.
Alex Usher: Right. Okay. Where does the blame for all of this lie? I mean, I know the university secretary general Patricia Davila, has resigned already over the affair, resigned about six weeks ago. But is it all on her? I mean, are, are there other people? Like, what's the bigger issue here, and, and what's the role of the company that was managing the exams for you?
Alma Maldonado-Maldonado: Yeah, it is not up to me to say, you know, who is the responsible because in the commission we made sure we tried to explain what happened. We tried to make with a solution at the moment to see what UNAM can do. And now there are other processes trying to understand who is responsible and, you know, probably name these people.
But Patricia Dávila was in charge of the whole process. I mean, to me, that was basically natural, right? I mean, she was the person in charge. Now, is she the only responsible? It's not to me to say, but there are still more investigations about who else is responsible for this uh, mess.
Alex Usher: Right.
Alma Maldonado-Maldonado: And um, that, that regarding UNAM. Now, I would like to say that in a country where people basically never take responsibility and where we pretend everything is okay and we don't solve anything, that was the easy way, right? UNAM could have said, "Well, I'm sorry, you know, this happened, but let's continue, and next year we will change."
And I think it was you know, courage. It took some courage to create this commission and to say, "Well, we have to redo the exam." And we are aware, of course about other countries that are handling with these situations about online testing problems, cheating. I mean, Mexico was not the only one.
Alex Usher: Not by any means.
Alma Maldonado-Maldonado: Yeah, I know. So at least, I mean, that was a positive point to say let's do something. Now, I don't know what would be the latest consequences of this, but this was important to do it. And I mean, it took time, it took money. Of course they, you know, people are, will be looking for who's responsible for all the money was spent again in this exercise.
But I think that was something. And regarding the, the the platform, yeah, I mean, there are also discussions about what were the mistakes, what were the problems. There was these legal issues that have to be reviewed as well, like, in the contract, if everything was okay, if the decisions were correct.
This is you know, this business it's everywhere. they conduct and they apply exams in the US, in other countries. They present us, you know, their whole portfolio of evaluations. They have experience. I don't think they were ready for something like the UNAM examination tests.
Alex Usher: Well, because UNAM is a very politically sensitive, politically powerful, politically sensitive institution. You know, and that must have conditioned how the government reacted to all this, right? So I know President Sheinbaum rarely misses a chance to beat up on universities that her party didn't create.
And so, you know, was, was-- has there been a reaction from the political side on all this?
Alma Maldonado-Maldonado: Yeah, of course. I mean, it was the most important topic in Mexican news for a couple of weeks. Everybody was talking about this. Everybody has a solution or a comment or, or to point out a problem.
Yeah, the president, of course, took the chance to talk about against the examination, the entrance exams.
They took the chance to talk about the institutions they have created where you don't have entrance examinations. So yeah politically it was a big debate. And again, I think that in a country where politicians normally never ask for forgiveness or recognize mistakes that was important at least at that moment
Alex Usher: Yeah. And what about public perception? 'Cause, you know, I mean, UNAM has a reputation not just with government but with society. Like, what is the reputation suffering with the general public? What will the consequences of all this be?
Alma Maldonado-Maldonado: Yeah, I think there was a part of you know, a group of people who thought and took the chance to, to talk bad about UNAM to, you know, criticize the institution and look how much money they receive. They are a public institution, what they do. And also, I think there was another group of people who thought, well, at least they correct the mistake, or they try to.
And you know, the director apologized to the families, to the students who had to retake the exam. Again, this is not common in Mexican politics politicians
apologizing for anything or recognizing that there was a mistake. And I, I think it really puts us in the debate about the new technologies, the online tests, and everything that is coming.
I, I think that was a good way, probably very painful way, to start the debate in Mexico about all these issues, because UNAM is not the only one that uses Territorium, this platform, right? I mean, other institutions, there were other problems. Even the government is using for their micro-credentials program, which is a big thing for the government here in Mexico, and they are using Territorium.
So the scandal in that sense was yeah, big.
Alex Usher: Oh, that's for the SubX platform, the, the, the micro-credentials. Yeah. Oh, wow. I didn't realize that. Interesting.
Alma Maldonado-Maldonado: Yeah, no, the links where, you know, it's crazy. Everything is connected.
Alex Usher: Yeah. Will next year's exams be held online? And if so, what do you think will change?
Alma Maldonado-Maldonado: No, they won't. The recommendation of the commission was to make sure that before UNAM goes online again they cover all the security problems and issues that we observed, right? We didn't finish. There is another investigation about the platform, and they will say what else they find, right?
Because there were so many ways of cheating that we don't understand the whole thing, right? And because we didn't have the time to go into the platform and to check the videos, they have supposedly the videos of every exam, so you can look at what was happening. The- there were the two people who were detained because they were selling what I told you this way that I can pretend, or I can help you to pretend you are answering the exam but another people is answering.
Just something funny but sad, in the videos there were people sleeping while supposedly answering the exam because, you know, you have to be two hours there while the other person is answering, so they were sleeping. So that's how crazy this can be.
Anyway,
Alex Usher: So you don't think you'll solve all these problems in time for next year?
Alma Maldonado-Maldonado: Yeah, they won't. So what they are doing is they have the exam again in a place where you probably will be using some electronic device to answer both on site.
Alex Usher: Got it. Alma, thank you so much for joining us today.
Alma Maldonado-Maldonado: Well, thank you, Alex. I hope this is interesting to, your audience
Alex Usher: It certainly is. And it just remains for me to thank our excellent producers, Noah Arney and Sam Pufek, and you, our listeners and readers for joining us. If you have any questions or comments about today's podcast or suggestions for future episodes, please don't hesitate to get in touch with us at podcast@higheredstrategy.com.
Join us next week when our guest will be Randy Boyagoda. He is a professor of English at the University of Toronto and a novelist, and also the author of the scorching new campus novel, The Lords of Serendipity. Don't miss it. Bye for now.