Ask an examiner from 20 years ago the risk of allowing people to take exams in their own home. They would have said 'cheating', even with no concept of AI.
Here is what happened. ACCA, one of several accountancy bodies in the UK, charge their students extraordinary sums of money to take their exams. When I took accountancy exams there were 9of 3 hour written exams, in a real building, with real invigilators. All of the bodies at the same time realised that they could charge the same amount, pay Pearson to administer an electronic test and make more money out of their students. It was a disgrace then and it is a disgrace now
> Ask an examiner from 20 years ago the risk of allowing people to take exams in their own home. They would have said 'cheating', even with no concept of AI.
AI has taken it to the next level. Previously, with many exams you would still have to know how to identify the concepts and related keywords in a word problem to even know what words to look for in the index of the books on hand before you could get to the right page to start cheating.
Some of the certification exams I had to take back in the day even came with their own little reference manual that everyone got and was free to use to look up concepts and equations like you would in the real world. The book wasn’t helpful if you didn’t know how to recognize the way to solve the problem and look it up, though.
AI changes that. Now you don’t need to know anything at all. You don’t even need to parse the question or even speak the same language. Copy the problem into ChatGPT with a prompt attached. Copy the answer into the solution box.
Anecdotally, the rise of ChatGPT has also normalized the concept of cheating among students. The common thinking is that everyone is using ChatGPT, therefore you’ll be left behind if you don’t cheat.
> Ask an examiner from 20 years ago the risk of allowing people to take exams in their own home.
Isn't this like an "open-book" exam? We had them 50 years ago when I was doing my A-levels in the UK, and I always thought it was a good system. The trouble now is of course that you can ask the book to look up the answer, unless the question is very well thought out, which is hard. The open-book thing worked best IMHO for things like practical chemistry, where you needed the technique as well as the theory.
Not really. An open book exam still requires you to know which book to bring in, understand the concepts, and be able to reference them on the fly to answer questions. Basically, you need a reasonable grounding in the material to know where to start figuring out your answer.
What’s different with at-home exams is there’s nothing stopping your ringing your friend to ask for the answer, or looking it up on Google (now ChatGPT), or asking your parents who happen to be in the industry, if you want to go really old school!
Seriously. Kids are going to cheat. It's already easy enough to just throw the test material into the LLM and get a bunch of flash cards on relevant content and memorize that. I Wish I had AI in college.
From watching slightly younger than college age kids adapt to the current world, I think you should be glad you did’t have access to LLMs during your learning years.
It’s too easy to slip from the idea that you’re just going to use the LLM to generate study materials into thinking that you’re just going to let the LLM do this homework assignment because your tired and then into a routine where ChatGPT is doing everything because you’ve come to rely on it. Then the students get slapped in the face with a sudden bad grade because the exams are in-person and they got all the way to the end of the semester with A-graded homework despite very little understanding of the material.
> It's already easy enough to just throw the test material into the LLM and get a bunch of flash cards on relevant content and memorize that
LLM summarisation is broken, so I wouldn't expect them to get very far with this (see this comment on lobste.rs: https://lobste.rs/c/je7ve5 )
Also, memorizing flashcards is actually, to some point, learning the material. There's a reason why Anki is popular for students.
Ultimately, however, this comes down to the 20th+21st century problem of "students learning only for the test", which we can see has critical problems that are well-known:
I remember reading something when I was studying for AWS certs (might’ve been from AWS itself): the goal of the certifying bodies is to make as much money as possible. For this to happen, the exam can’t be so hard that nobody takes it, but it can’t be so easy that everyone takes it and it loses its value.
Organizations have been coasting on their pre-Covid reputations for a while. Now it’s time for them to adjust the slider the other way.
I don't know about this part. Years ago, my friend in college was taking all kinds of Microsoft certification exams and passing them with near perfect score. Thing is, he had no clue about most of the topics he passed, he had never worked with those tech. He just spent a bunch of time collecting questions (which wasn't that hard to find) and memorizing the answers. They could've made it difficult enough so just rote memorization wouldn't work, but they didn't (don't know if it has changed now).
Companies had long figured out these certifications are just easy money. It is hard to resist the temptation to just charge hundreds of dollars for a test and add it as a "profit center"
I don't want to sound heartless, especially as I'm in the high-risk category myself, but I think it's important to recognise that while COVID hasn't gone away, it is no longer a pandemic.
It is now endemic instead, and needs to be managed as such.
My wife is a teacher of physics and math for an online highschool. Its very common for kids to go into the in person exam with a mark in the 80s and 90s and get a failing grade on the exam.
The web wasn't alwasy that useful for cheating on timed exams as it was essentially like being able to bring in a formula sheet.
LLM's changed this such that you can type in the question and get a fully correct answer in a lot of cases.
The only solution that I see in education is that in person exams start to represent a larger and larger portion of a students grade such that the mid term and final will be more than 50% of a students grade for most classes going forward due to the gratuitous use of llms by students.
When I took quantum mechanics in grad school, I struggled through the weekly (and intense) homework sets. My TA was a hardass, I’d spend hours on some problem, several few pages of math work just for one problem, and make some dumb mistake in an integral somewhere, being off by a factor of 2 at the end and only getting 2 of 4 points.
It was painful, and I felt like a dumbass seeing the other kids regularly getting perfect scores.
Then the midterm came and I blew them all out of the water. I hadn’t realised they somehow had the solutions manual so just got perfect scores all along but clearly didn’t learn the material like I did.
I failed one exam in my final year of uni (marginally), but passed the module because of excellent coursework. I put in an order of magnitude more work into the coursework for that class than I did any other class because I knew I was going to struggle in the exam.
In all honesty I shouldn’t have passed that course but it is what it is - and as far as I was (and still am) concerned, it was a bolt on course that I am ok being limited in my knowledge of.
I don't know, I've known many people that struggle with exams even if they know the material and even more people that excel with exams that learn nothing. Falling back on any kind of exam is just a recipe for more rote learning and that doesn't create better people (although possibly better readers, which we need).
(Preface: I am not a teacher, and I understand this is a hot take). At the end of the day there's an unwillingness from every level of education (parents, teachers, administrators, school boards, etc) to fight against the assault on intelligence by tech.
I don't think kids should have access to the public internet until they're adults, and certainly should never have it in schools except in controlled environments. Schools could create a private networks of curated sites and software. Parents don't have to give their kids unfettered access to computers. It's entirely in the realm of possibility to use computers and information networks in schools, accessed by children, designed to make it impossible to cheat while maximizing their ability to learn in a safe environment.
We don't build it because we don't want to. Parents don't care enough, teachers are overworked, administrators are inept, and big tech wants to turn them into little consumers who don't have critical thinking and addicted to their software.
Until quite recently, it was trivial to cheat on remotely proctored exams. All you had to do is spin up a VM, take the exam inside the VM, and use the host system to look up answers. I believe the main proctoring services now have crude VM checks. You can probably still use a KVM switch or a DP splitter and a buddy...
My point is that since it is so incredibly easy to cheat (despite countermeasures that are essentially theater), returning to in person exams is probably a good thing.
"We are doing what we can to hang on to relevancy as gatekeepers who already held way too much authority over a field". They are going to use AI on the job anyway.
This also applies to universities. The world has changed but they have not and they will make sure to try and stay relevant as much as they can to continue to take money.
Edit: looks like it will take a while for some people to accept that we are not going back from this. The cat is out of the bag and your certificates are increasingly irrelevant. Sorry if you spent a lot of money and time to get it.
Certifications have always been irrelevant for me, but that's only because my goal has always been what I'm capable of doing on my own AND (this one is a biggie) I was unbelievably fortunate to have several people in my career who trusted that I could get the job done.
Certifications are about low trust. With the advent of modern LLM tech, trust levels are probably not going up.
Nobody needs to hire someone who can use an LLM because if that is the skill they're looking for they can just use the LLM themselves.
So if you need to hire someone because the LLM isn't cutting it, then you'll by definition need to be hiring someone who isn't using an LLM. Someone who isn't just using an LLM to make you think that they aren't using an LLM.
How is that going to be done? Sounds like a job for certifications to me. Not today's certifications, but a much more in depth, in person, and gatekeepery certification.
My guess would be that certifications, unfortunately, will be significantly more relevant in the days of LLMs. Not less.
I don’t think it will be too long before the pendulum swings back towards “real people who actually know the subject”. At that point, I might feel bad for everyone who coasted on AI.
I've had no end of problems with accountants regardless of their certifications, they operate in a domain with an incoherent body of contradictory and highly subjective rules yet make it out to be a science.
My conclusion as a whole is that accountancy as a profession rarely delivers any actual value to their customers, where much of the job is compliance theater at best.
One of the main issues I had when I took accounting was that you often couldn't figure out things from first principles because the "right" way was whatever the relevant financial accounting standards board said it was. But following that standard is what companies need to do--and therefore has value--even if it's arguably arbitrary (within some general framework).
Yeah ... that's kind of the point. The money doesn't exist, but the violence people will use if their money is misappropriated is very real. Accounting is loophole patch stacked on loophole patch for thousands and thousands of years.
It's not intellectually enriching, but like it has the weight of society going back forever with dire consequences when it fails. That's not nothing even if it's boring from a technological point of view.
I think of it sort of like git. Technically, any sort of distributed version control would have served our industry just fine. Git didn't need to win, but things are vastly simplified having basically one version control framework to rule them all.
Get back to me when AI is actually reliably correct about any technical field.
Accounting exams are gatekeeping, yes. The good kind of gatekeeping where you make sure the people doing the job are actually capable. And you have avenues to punish those who fail their clients.
> This also applies to universities
Eh. I’d say the actual academics are about 1/3 of the university experience. The rest is networking and teaching you how to think and solve problems on a more abstract level. I’d say the people who farm that (and particularly the abstract thinking part) out to AI are going to be the ones left at disadvantage in the future. You’re completely replaceable.
At the end of the day the job market will correct itself accordingly which is what most people who bother going to university or collecting any certificate care about. And right now it is already looking bleak. https://accountancyage.com/2025/09/29/pwcs-graduate-glow-up-...
Might be time we start adapting the pipeline into employment and start revising the importance of some of these gatekeepers before more people fall into unnecessary debt.
Ask an examiner from 20 years ago the risk of allowing people to take exams in their own home. They would have said 'cheating', even with no concept of AI.
Here is what happened. ACCA, one of several accountancy bodies in the UK, charge their students extraordinary sums of money to take their exams. When I took accountancy exams there were 9of 3 hour written exams, in a real building, with real invigilators. All of the bodies at the same time realised that they could charge the same amount, pay Pearson to administer an electronic test and make more money out of their students. It was a disgrace then and it is a disgrace now
> Ask an examiner from 20 years ago the risk of allowing people to take exams in their own home. They would have said 'cheating', even with no concept of AI.
AI has taken it to the next level. Previously, with many exams you would still have to know how to identify the concepts and related keywords in a word problem to even know what words to look for in the index of the books on hand before you could get to the right page to start cheating.
Some of the certification exams I had to take back in the day even came with their own little reference manual that everyone got and was free to use to look up concepts and equations like you would in the real world. The book wasn’t helpful if you didn’t know how to recognize the way to solve the problem and look it up, though.
AI changes that. Now you don’t need to know anything at all. You don’t even need to parse the question or even speak the same language. Copy the problem into ChatGPT with a prompt attached. Copy the answer into the solution box.
Anecdotally, the rise of ChatGPT has also normalized the concept of cheating among students. The common thinking is that everyone is using ChatGPT, therefore you’ll be left behind if you don’t cheat.
> Ask an examiner from 20 years ago the risk of allowing people to take exams in their own home.
Isn't this like an "open-book" exam? We had them 50 years ago when I was doing my A-levels in the UK, and I always thought it was a good system. The trouble now is of course that you can ask the book to look up the answer, unless the question is very well thought out, which is hard. The open-book thing worked best IMHO for things like practical chemistry, where you needed the technique as well as the theory.
Not really. An open book exam still requires you to know which book to bring in, understand the concepts, and be able to reference them on the fly to answer questions. Basically, you need a reasonable grounding in the material to know where to start figuring out your answer.
What’s different with at-home exams is there’s nothing stopping your ringing your friend to ask for the answer, or looking it up on Google (now ChatGPT), or asking your parents who happen to be in the industry, if you want to go really old school!
Also, in a well constructed open book exam having the book won’t help you worth a damn if you haven’t already read it at least once.
Seriously. Kids are going to cheat. It's already easy enough to just throw the test material into the LLM and get a bunch of flash cards on relevant content and memorize that. I Wish I had AI in college.
> I Wish I had AI in college.
From watching slightly younger than college age kids adapt to the current world, I think you should be glad you did’t have access to LLMs during your learning years.
It’s too easy to slip from the idea that you’re just going to use the LLM to generate study materials into thinking that you’re just going to let the LLM do this homework assignment because your tired and then into a routine where ChatGPT is doing everything because you’ve come to rely on it. Then the students get slapped in the face with a sudden bad grade because the exams are in-person and they got all the way to the end of the semester with A-graded homework despite very little understanding of the material.
You can also just pay attention and practice
Paying attention is very hard
Yes, thanks to a trillion-dollar digital advertising and propaganda industry strip-mining our attention spans for profit.
Found the nerd!
The act of making the flash cards is more important than having them when you've finished.
> It's already easy enough to just throw the test material into the LLM and get a bunch of flash cards on relevant content and memorize that
LLM summarisation is broken, so I wouldn't expect them to get very far with this (see this comment on lobste.rs: https://lobste.rs/c/je7ve5 )
Also, memorizing flashcards is actually, to some point, learning the material. There's a reason why Anki is popular for students.
Ultimately, however, this comes down to the 20th+21st century problem of "students learning only for the test", which we can see has critical problems that are well-known:
https://matheducators.stackexchange.com/a/8203
https://www.youtube.com/watch?v=J6lyURyVz7k
This isn't just about AI, the exams were only moved to remote for COVID.
There will be a lot of COVID-era qualifications that are treated with a hint of suspicion in the future.
Take a look at A-level scores: https://schoolsweek.co.uk/a-level-results-2024-future-exams-...
( direct link to graph: https://schoolsweek.co.uk/wp-content/uploads/2023/04/Overall... )
It's unfortunate for those affected either way. It was a difficult time when drastic measures needed to be taken at short notice.
It's right to go back to in-person testing if there is a problem keeping remote exams fair.
I wonder why it wasn't done earlier as the pandemic has been over for a while.
I remember reading something when I was studying for AWS certs (might’ve been from AWS itself): the goal of the certifying bodies is to make as much money as possible. For this to happen, the exam can’t be so hard that nobody takes it, but it can’t be so easy that everyone takes it and it loses its value.
Organizations have been coasting on their pre-Covid reputations for a while. Now it’s time for them to adjust the slider the other way.
everyone takes it and it loses its value
I don't know about this part. Years ago, my friend in college was taking all kinds of Microsoft certification exams and passing them with near perfect score. Thing is, he had no clue about most of the topics he passed, he had never worked with those tech. He just spent a bunch of time collecting questions (which wasn't that hard to find) and memorizing the answers. They could've made it difficult enough so just rote memorization wouldn't work, but they didn't (don't know if it has changed now).
Companies had long figured out these certifications are just easy money. It is hard to resist the temptation to just charge hundreds of dollars for a test and add it as a "profit center"
> the pandemic has been over for a while
The pandemic isn't actually over, at least, not for disabled people.
I don't want to sound heartless, especially as I'm in the high-risk category myself, but I think it's important to recognise that while COVID hasn't gone away, it is no longer a pandemic.
It is now endemic instead, and needs to be managed as such.
https://en.wikipedia.org/wiki/Endemic_COVID-19
My wife is a teacher of physics and math for an online highschool. Its very common for kids to go into the in person exam with a mark in the 80s and 90s and get a failing grade on the exam.
The web wasn't alwasy that useful for cheating on timed exams as it was essentially like being able to bring in a formula sheet.
LLM's changed this such that you can type in the question and get a fully correct answer in a lot of cases.
The only solution that I see in education is that in person exams start to represent a larger and larger portion of a students grade such that the mid term and final will be more than 50% of a students grade for most classes going forward due to the gratuitous use of llms by students.
This was going on long before LLMs.
When I took quantum mechanics in grad school, I struggled through the weekly (and intense) homework sets. My TA was a hardass, I’d spend hours on some problem, several few pages of math work just for one problem, and make some dumb mistake in an integral somewhere, being off by a factor of 2 at the end and only getting 2 of 4 points.
It was painful, and I felt like a dumbass seeing the other kids regularly getting perfect scores.
Then the midterm came and I blew them all out of the water. I hadn’t realised they somehow had the solutions manual so just got perfect scores all along but clearly didn’t learn the material like I did.
Yeah, I'm not a frat guy, but don't groups like Fraternities build "Study Guides" that are often just brain dumps of tests with correct answers?
I failed one exam in my final year of uni (marginally), but passed the module because of excellent coursework. I put in an order of magnitude more work into the coursework for that class than I did any other class because I knew I was going to struggle in the exam.
In all honesty I shouldn’t have passed that course but it is what it is - and as far as I was (and still am) concerned, it was a bolt on course that I am ok being limited in my knowledge of.
I don't know, I've known many people that struggle with exams even if they know the material and even more people that excel with exams that learn nothing. Falling back on any kind of exam is just a recipe for more rote learning and that doesn't create better people (although possibly better readers, which we need).
(Preface: I am not a teacher, and I understand this is a hot take). At the end of the day there's an unwillingness from every level of education (parents, teachers, administrators, school boards, etc) to fight against the assault on intelligence by tech.
I don't think kids should have access to the public internet until they're adults, and certainly should never have it in schools except in controlled environments. Schools could create a private networks of curated sites and software. Parents don't have to give their kids unfettered access to computers. It's entirely in the realm of possibility to use computers and information networks in schools, accessed by children, designed to make it impossible to cheat while maximizing their ability to learn in a safe environment.
We don't build it because we don't want to. Parents don't care enough, teachers are overworked, administrators are inept, and big tech wants to turn them into little consumers who don't have critical thinking and addicted to their software.
https://archive.is/tiqef
> outpacing... safeguards
Calculations must be getting accurate now. Not only questions about vocabulary or domain concepts.
Until quite recently, it was trivial to cheat on remotely proctored exams. All you had to do is spin up a VM, take the exam inside the VM, and use the host system to look up answers. I believe the main proctoring services now have crude VM checks. You can probably still use a KVM switch or a DP splitter and a buddy...
It is incredibly trivial to stick a knife into human flesh.
Triviality is not a dimension of ethics as far as I have come to understand it.
My point is that since it is so incredibly easy to cheat (despite countermeasures that are essentially theater), returning to in person exams is probably a good thing.
"We are doing what we can to hang on to relevancy as gatekeepers who already held way too much authority over a field". They are going to use AI on the job anyway.
This also applies to universities. The world has changed but they have not and they will make sure to try and stay relevant as much as they can to continue to take money.
Edit: looks like it will take a while for some people to accept that we are not going back from this. The cat is out of the bag and your certificates are increasingly irrelevant. Sorry if you spent a lot of money and time to get it.
Certifications have always been irrelevant for me, but that's only because my goal has always been what I'm capable of doing on my own AND (this one is a biggie) I was unbelievably fortunate to have several people in my career who trusted that I could get the job done.
Certifications are about low trust. With the advent of modern LLM tech, trust levels are probably not going up.
Nobody needs to hire someone who can use an LLM because if that is the skill they're looking for they can just use the LLM themselves.
So if you need to hire someone because the LLM isn't cutting it, then you'll by definition need to be hiring someone who isn't using an LLM. Someone who isn't just using an LLM to make you think that they aren't using an LLM.
How is that going to be done? Sounds like a job for certifications to me. Not today's certifications, but a much more in depth, in person, and gatekeepery certification.
My guess would be that certifications, unfortunately, will be significantly more relevant in the days of LLMs. Not less.
I don’t think it will be too long before the pendulum swings back towards “real people who actually know the subject”. At that point, I might feel bad for everyone who coasted on AI.
The damage has already been done.
Much like how if you stop going gym you lose muscle mass, the same happens with knowledge and understanding with the brain.
I've had no end of problems with accountants regardless of their certifications, they operate in a domain with an incoherent body of contradictory and highly subjective rules yet make it out to be a science.
My conclusion as a whole is that accountancy as a profession rarely delivers any actual value to their customers, where much of the job is compliance theater at best.
Accounting is a PvP profession. It's you against the taxman and others who want to issue fines etc.
One of the main issues I had when I took accounting was that you often couldn't figure out things from first principles because the "right" way was whatever the relevant financial accounting standards board said it was. But following that standard is what companies need to do--and therefore has value--even if it's arguably arbitrary (within some general framework).
Yeah ... that's kind of the point. The money doesn't exist, but the violence people will use if their money is misappropriated is very real. Accounting is loophole patch stacked on loophole patch for thousands and thousands of years.
It's not intellectually enriching, but like it has the weight of society going back forever with dire consequences when it fails. That's not nothing even if it's boring from a technological point of view.
I think of it sort of like git. Technically, any sort of distributed version control would have served our industry just fine. Git didn't need to win, but things are vastly simplified having basically one version control framework to rule them all.
I don’t really agree with this. Sure there are standards but there are underlying first principles with some quirks to make things balance.
I'm not sure we really disagree. Sure, there are foundational principles but how to handle non-routine transactions aren't necessarily at all obvious.
Get back to me when AI is actually reliably correct about any technical field.
Accounting exams are gatekeeping, yes. The good kind of gatekeeping where you make sure the people doing the job are actually capable. And you have avenues to punish those who fail their clients.
> This also applies to universities
Eh. I’d say the actual academics are about 1/3 of the university experience. The rest is networking and teaching you how to think and solve problems on a more abstract level. I’d say the people who farm that (and particularly the abstract thinking part) out to AI are going to be the ones left at disadvantage in the future. You’re completely replaceable.
At the end of the day the job market will correct itself accordingly which is what most people who bother going to university or collecting any certificate care about. And right now it is already looking bleak. https://accountancyage.com/2025/09/29/pwcs-graduate-glow-up-...
Might be time we start adapting the pipeline into employment and start revising the importance of some of these gatekeepers before more people fall into unnecessary debt.