> patterns suggest that overqualified individuals are less motivated, but still outperform others in their same job. Underqualification results in a polar opposite set of findings, suggesting these individuals are motivated to put forth more effort, but still struggle to compete when judged relative to others.
So... the system works?
At least within the very constrained universe of what the Air Force is doing/testing for?
It's not about the system working or not, it's more about whether it's well calibrated. You'd expect performance to scale linearly with skill (f(skill) = job_quality * x), but the data look more like a sigmoid: overqualified people disengage and never hit their full potential. Underqualified folks aren't that far behind their peers, yet in a better-matched role, they'd perform better with less effort.
I find it kinda sad, honestly. But these are aggregated statistics, I still think people can overcome their perceived limitations, I'm certain some of the underqualified folks are the ones that drag that curve up a lot.
I think it's stating that overqualified individuals are likely to get worse performance evaluations than if that same individual was in a job they were qualified for (e.g. they try harder when challenged and earn better ratings for themselves on a comparative basis), but are still more likely to get promoted than qualified/underqualified peers.
They did say they outperform persons better aligned to the job even though they are less happy. I'm guessing their review is worse relative to the same person in a job they aren't over qualified for, as opposed to comparing to another person in the same job who is not overqualified.
for sure. sometimes getting stuff done involves stepping on the wrong person's toes, even if it was completely unintended. we've all been there before! a strong boss is required to smooth things out in those situations
I think they mean worse performance evaluations relative to others with similar test scores, but more likely to be promoted compared to other people working the same job
Could be the credentials themselves get you marked for promotion.
Monsieur Bouvier was a high school French teacher who was one time the only teacher with a PhD in my school district. He knew a lot about pedagogy and evaluation and certainly French but being in his classroom I think he lacked "soft skills" and was not good at dealing with bullshit which is a lot of what the teacher job entails.
He got promoted to assistant principal on the basis of his credentials and put in charge of gifted and talented programs, I think the honors program succeeded precisely because people above him bypassed his authority and overruled him quite often. He was not really a good leader or manager -- as assistant principal he got to do some of what he was good at but he had to do more of dealing with the bullshit that the first and second tier couldn't deal with.
My school had Roy Downton as principal for the longest time and he was really great at the job and hard to replace. There was a lot of jockeying for the position and Bouvier lost out and Mr. Adamankos finally won. I think Boivier's credentials got him a certain advantage in promotion but fortunely he didn't get promoted too far beyond his competence.
im considering the possibility that the performance reviewer is harder on the overqualified candidate for any subjective assessment because expectations are greater or the goal of the lukewarm assessment is to motivate or challenge more capable candidates. an underqualified candidate only has to keep up to pass subjective assessment and the motivation is built-in
many people are immune to basic motivation tactics but im surprised how many of my peers i see influenced by reviews which seem mostly motivational, and occasionally political, to me
My interpretation based on experience is that the underqualified individuals will often be more prone to volunteering, participation in clubs and committees, and politicking. The overqualified individuals also do this, but will likely be required to do "the real job" more often, leaving a less qualified individual more free to do those things.
Community involvement is a significant factor on both enlisted and officer performance reports. Gotta fill that section in no matter what, and if your section is poor it drags your overall score down.
However, promotion testing is purely knowledge and skill based. A good test taker can overcome the weight of lower performance report scores.
Reminds me of the Gervais framework that someone made (as a kind of parallel to The Office) where you had three groups, confused, losers, psychopaths (don't read too much into the names). Confused work really hard at things they don't get recognized for (Dwight), psychopaths underperform and get promoted (Ryan).
I had job mis-matches for years, and it helped me. I could do most of my work for almost all of my roles in half or less time. I Used the extra time to start successful businesses and now a successful consulting career.
I think they mean the job assignments were real, and what they're "simulating" is a randomised controlled experiment where they tell the Air Force who to assign to which position. The wording is confusing, but the methodology sounds reasonable.
It would be really interesting to know to what extent their definitions of being over or under qualified is an approximation of IQ test scores. IQ tests have lovers and loathers but a correlation or otherwise would say they are measuring something or not…
Total anecdata, but whenever I've had to take an IQ test for an interview process, it's never the IQ test that's used to disqualify me. It's usually something like culture fit.
But these days if a job requires me to do an IQ test to join, I'll use that as a signal to get the fuck out of there and find a different role. So again, anecdata, but I suspect I'm not the only one who would eschew those results.
Amazon did this, at least for a time, as part of a "No See, No Hear" hiring pilot program.
The purpose was to see if they could hire university graduates with a minimum of human interviewing effort. They selected from a handful of universities, gave a couple online tests, verified the candidate's identity as the test-taker, then would give out offers sight-unseen.
I was hired this way in 2015. From my perspective, I had taken a couple online tests, then months later had a thirty-minute identity verification call, then a couple months later, was sent a job offer. I thought it was by mistake, so I didn't ask too many questions. I had a thirty-minute call with a hiring manager I otherwise never interacted with, then accepted, flew internationally back to the states to Seattle to start, met him and all my teammates for the first time on my first day of work.
I found the internal documents about this program later on spelunking in the internal wiki.
that is wild! I could certainly see this as an attempt to eliminate hiring bias maybe? that was super popular in that time frame, but never heard anybody taking it that far.
It's funny how people pushed hard for removing hiring bias until they realized that this meant that you'd get even more people from unfavorable social groups.
Anyway, looking from today's perspective, this seems like an obvious attempt at automating the hiring process itself in order to fold the HR department. If only back then they had today's AI technology.
I applied for a management-consulting-ish job a decade ago (I was desperate!) at a big firm and had to take what was basically an IQ test. I have no idea if the test literally calculated my IQ, but the questions were exactly the questions you'd see in an IQ test (e.g. next item in some geometric sequence) so it may as well have.
This was in a group interview for recent university graduates at a very big company. I assume their hiring process was pretty standardized, so there were probably thousands of people taking this test every year in North America.
General IQ tests probably aren't legal in the US; Griggs v Duke Power Co [1] says (more or less) that employment tests have to be job related if the results of the test have disparate impact on protected classes of people.
It's hard to argue that a general IQ test is job related, but they're likely to have a disparate impact on protected classes of people.
This is a weird pernicious Internet myth. It obviously can't be true, because there's a big, well-known company that delivers these tests for employment/recruiting settings and they have a logo crawl on their page that include several giant companies. If those tests were illegal, employment lawyers would be making bank off it.
I'm not an expert/lawyer, but this does seem to indicate that the situation is a bit more complicated than either "pernicious myth" or "probably illegal" in general (but much closer to toast0's understanding); my interpretation is that you can either avoid an 80% threshold of "disparate impact" or you can in theory formally validate that a particular test measures/predicts performance at a particular job; that all sounds compatible with "companies do it in the open, but very few, and you can easily get in trouble for doing it wrong"
https://www.law.cornell.edu/cfr/text/29/1607.15
The comment to which I responded claimed that IQ tests "probably aren't legal in the US", which is false. They aren't more widely used because they don't work well for candidate selection, but they are used by very large companies that are attractive targets for employment discrimination suits, and wouldn't be if they were legally risky. There are well-known tech companies that up until a few years ago gave IQ tests to candidates!
Empirical observation trumps axiomatic derivation in this case.
The military was pretty big on IQ testing, from what I understand. I’m not sure if this is still the case, but it seems like an efficient first pass to figure out where people should go.
From what I recall, the ASVAB wasn't really a general IQ test. It was more like an SAT with a broader focus than academics, basically "Are you basically adequate with Words/Numbers/machines/ etc"
Huh. The IQ tests that I was administered as a kid I remember as being primarily focused on abstract pattern recognition, which is what I think of when I think "IQ test". But I suppose due to IQ's definition, if you just give someone a wide enough battery of cognitive tasks, you can derive the IQ score that way.
I guess I always assumed that the ASVAB was used a bit more literally. "Ah, this person is barely literate, but knows all the parts of an engine. TO THE MOTOR POOL"
I had my IQ tested recently and the pattern recognition was part of it, but so were the word associations.
Depending on how young you were, maybe they weight it more toward the abstract patterns, because no kid isn’t going to have a fully developed vocabulary.
> The IQ tests that I was administered as a kid I remember as being primarily focused on abstract pattern recognition, which is what I think of when I think "IQ test".
That's just one specific IQ test, Raven's Progressive Matrices.
> I guess I always assumed that the ASVAB was used a bit more literally. "Ah, this person is barely literate, but knows all the parts of an engine. TO THE MOTOR POOL"
> there is only one exam, the Armed Services Vocational Aptitude Battery, or ASVAB for short. The ASVAB has 10 tests. Your scores from four of the tests—Word Knowledge (WK), Paragraph Comprehension (PC), Arithmetic Reasoning (AR), and Mathematics Knowledge (MK)—are combined to compute your score on what is referred to as the Armed Forces Qualification Test (AFQT). Scores on the AFQT are used to determine your eligibility for enlistment in the Army, Navy, Air Force, or Marine Corps. Scores on all of the ASVAB tests are used to determine the best job for you in the military.
You have to clear an IQ threshold to be eligible to enlist at all. Only after you're smart enough to be in the army will they consider which jobs you might do relatively better in.
If by "IQ test" you mean "any test that correlates well with other accepted IQ tests", then, sure, I guess. But there's a reason clinicians administer the Wechsler and not the SAT, and it's not just that the SAT takes longer.
Sure, the reason is that they want to do something that is "the same" as what they've done in the past, so that previous research will remain as valid as they hope it is.
My mother is an obstetrician, and something that has always bothered her is that American hospitals have women lie on their backs to give birth. This is not a natural position, it's not comfortable for the women, and it can make it more difficult to get the baby out.
So why do we do it?
The answer is that, a long time ago, doctors who assumed that that was the correct way to give birth developed a set of standard measurements that determine what doctors today think of how far into the labor process a woman is. These measurements are only valid for a woman lying on her back - they will change if she shifts positions. They would have to be redone and revalidated for a woman in a natural delivery position. And nobody wants to do that.
The SAT correlates as well with any given IQ test as other, "official" IQ tests do. It is an IQ test. It serves all of the purposes that IQ tests serve, and it cannot serve any purpose that they can't.
Not a hill I'm going to die on, but the SAT has many attributes IQ-ists insist IQ tests are insulated from: it's straightforwardly trainable, culturally loaded, samples only math, processing speed, and verbal reasoning, and tracks prior educational experience as much as it does aptitude.
Draw-a-Person basically isn't an IQ test at all, so I don't see how that comparison clears anything up.
> Not a hill I'm going to die on, but the SAT has many attributes IQ-ists insist IQ tests are insulated from
This is false. In particular:
> it's straightforwardly trainable
No, it isn't. There is an extensive literature on SAT prep, finding that it's worth a couple of points on the test. It is widely described as being trainable, but the opposite was always a design goal, and historically that goal was achieved very well.
You might note that the Raven's matrices are infamous for huge training effects; that test relies on the testee having never seen it before. The SAT doesn't.
> culturally loaded
This claim is true, but nobody claims that IQ tests are insulated from being culturally loaded. The purpose of Raven's is to be a culture-free test. Wechsler makes no such pretense.
> samples only math, processing speed, and verbal reasoning
I'm not sure what you're saying here.
> and tracks prior educational experience as much as it does aptitude.
And this one is false. The point of the SAT is to test only low-level material so that you can be confident the entire test-taking population has been exposed to the material. Aptitude has a very large influence on SAT score; prior education has a negligible influence.
(Prior education will have a larger influence if the population you're investigating includes a lot of people with no education, the kind of people who left school after or before kindergarten. But that scenario isn't relevant to... pretty much any question about the SAT.)
> Draw-a-Person basically isn't an IQ test at all, so I don't see how that comparison clears anything up.
It is an IQ test by the standard you defined: it holds itself out as being "an IQ test", and it is used by researchers to study the intelligence of testees. Did you want to use a different definition?
I've had some IQ tests, one company is now bankrupt, was quite a famous Danish bankruptcy (they offered me the job but I turned it down which looked prescient a couple years later when they went bankrupt), another was Klarna, Boozt also uses a cognitive assessment.
Klarna's seemed like the most proper IQ test although it had at least one question that was wrong.
Military takes the ASVAB which is basically the SAT / IQ Test which qualifies you were certain jobs. For example Nukes in the Navy (which is considered one of the hardest programs for obvious reasons) is 252 score for some combination of scores.
I've worked with at least two such "Nukes". One a researcher, the other a tech startup founder. Both very smart, team-oriented, amiable and down to earth, and inspiring respect (through their ability and how they conduct themselves).
I suppose, on a sub, besides whatever admissions filtering they do for aptitude, you have to take work seriously, and also cooperate and get along with people.
If the ASVAB is basically an IQ test, I’m horrified by how quickly my intelligence has decayed.
When I took it in HS, I scored quite well. I’m almost certain I’m bomb it completely now. Particularly any mathematics I haven’t touched since high school would look foreign to me.
It's exactly what it is. When someone enlists in the US military, they take what is called the ASVAB (Armed Services Vocational Aptitude Battery). It's more specific to the military than Stanford-Binet, but effectively it is still an IQ test. Specific military occupational specialties have minimum score requirements to qualify for that job. Which job you actually end up getting is a combination of scoring high enough to qualify, scoring higher than everyone else who wants the same job, doing well on other things they assess you on, and needs of the branch of service at the time you enlist. Score high enough for cyber but they only need mechanics, then you're becoming a mechanic. Conversely, if they badly need cyber professionals but nobody scores high enough to meet the official bar, they're still putting people into that job. Hence, why the mismatch exists in the first place and why there's a study like this.
My understanding is that IQ tests can measure a type of intelligence, but fixation on its narrow metrics can lead one to overlook other attributes that are just as likely to be relevant, but aren’t something that necessarily shows up in standardized tests. Add to that the fact that IQ tests can be heavily biased, and leave a lot of ambiguity for the proctors to interpret, it’s not a surprise that they’re so controversial.
I personally would be very suspicious if asked to sit for an IQ test as part of a job evaluation. I have worked for places that blindly worship context-free performance metrics, and it was insufferable.
IQ is a correlation variable that pops out when you measure any group of people's aptitude at a battery of tasks that involve thinking. Basically, if someone's good at one task that involves thinking (say, chess), they're more likely to be good at another task that involves thinking (say, reading comprehension). Apply some Bayesian statistics, and boom, you've concluded that there's some confounding variable (that we call IQ). Then you can start measuring how strongly certain tasks correlate with variable. Turns out abstract pattern recognition very strongly correlated with this unknown variable, so we can use that to predict what someone's IQ likely is.
The point is, due to the very definition of IQ, it's not a narrow metric, and selecting for it does tend to find you individuals who are going to be better than average at most anything. That said, it would seem alarming to me for a job to give me an IQ test instead of cutting out the correlation coefficient and just judging me on the task they're hiring me for.
"Combined, these patterns suggest that overqualified individuals are less motivated, but still outperform others in their same job. Underqualification results in a polar opposite set of findings, suggesting these individuals are motivated to put forth more effort, but still struggle to compete when judged relative to others."
Did Captain Obvious wrote this working paper or all NBER research is like this?
Well I worked as a civilian in that branch a decade ago. Purely technical work, and I got the assigned tasks done in about 7 months.
You also have to look at the culture of the place- and although that varies from base to base, whether or not someone is more or less qualified may be completely irrelevant. Cultural fit is very important, for example. Favoritism may be rampant, just like any other workplace. High performers who concentrate solely on the relevant tasks have less time to make the rounds around the office, playing petty politics, just like anywhere.
> patterns suggest that overqualified individuals are less motivated, but still outperform others in their same job. Underqualification results in a polar opposite set of findings, suggesting these individuals are motivated to put forth more effort, but still struggle to compete when judged relative to others.
So... the system works?
At least within the very constrained universe of what the Air Force is doing/testing for?
It's not about the system working or not, it's more about whether it's well calibrated. You'd expect performance to scale linearly with skill (f(skill) = job_quality * x), but the data look more like a sigmoid: overqualified people disengage and never hit their full potential. Underqualified folks aren't that far behind their peers, yet in a better-matched role, they'd perform better with less effort.
I find it kinda sad, honestly. But these are aggregated statistics, I still think people can overcome their perceived limitations, I'm certain some of the underqualified folks are the ones that drag that curve up a lot.
[flagged]
>We find that being overqualified for a job causes worse performance evaluations...and these individuals are more likely to be promoted.
Can someone explain this apparent contradiction, specifically in the context of the Air Force/military?
I think it's stating that overqualified individuals are likely to get worse performance evaluations than if that same individual was in a job they were qualified for (e.g. they try harder when challenged and earn better ratings for themselves on a comparative basis), but are still more likely to get promoted than qualified/underqualified peers.
They did say they outperform persons better aligned to the job even though they are less happy. I'm guessing their review is worse relative to the same person in a job they aren't over qualified for, as opposed to comparing to another person in the same job who is not overqualified.
for sure. sometimes getting stuff done involves stepping on the wrong person's toes, even if it was completely unintended. we've all been there before! a strong boss is required to smooth things out in those situations
I think they mean worse performance evaluations relative to others with similar test scores, but more likely to be promoted compared to other people working the same job
So cranky, maybe a pain in the ass, but still deliver the bacon?
Could be the credentials themselves get you marked for promotion.
Monsieur Bouvier was a high school French teacher who was one time the only teacher with a PhD in my school district. He knew a lot about pedagogy and evaluation and certainly French but being in his classroom I think he lacked "soft skills" and was not good at dealing with bullshit which is a lot of what the teacher job entails.
He got promoted to assistant principal on the basis of his credentials and put in charge of gifted and talented programs, I think the honors program succeeded precisely because people above him bypassed his authority and overruled him quite often. He was not really a good leader or manager -- as assistant principal he got to do some of what he was good at but he had to do more of dealing with the bullshit that the first and second tier couldn't deal with.
My school had Roy Downton as principal for the longest time and he was really great at the job and hard to replace. There was a lot of jockeying for the position and Bouvier lost out and Mr. Adamankos finally won. I think Boivier's credentials got him a certain advantage in promotion but fortunely he didn't get promoted too far beyond his competence.
im considering the possibility that the performance reviewer is harder on the overqualified candidate for any subjective assessment because expectations are greater or the goal of the lukewarm assessment is to motivate or challenge more capable candidates. an underqualified candidate only has to keep up to pass subjective assessment and the motivation is built-in
many people are immune to basic motivation tactics but im surprised how many of my peers i see influenced by reviews which seem mostly motivational, and occasionally political, to me
Maybe not in the context of military as I haven't served, but in private sector reviews can often determine your bonuses.
if performance reviews are based on peer feedback likely this - https://en.wikipedia.org/wiki/Crab_mentality
Not about your specific context, but this contradiction has a name: Dilbert Principle
Ever seen Office Space?
Well, people that are better evaluated at a job should have a higher chance of staying in that job, no? (Only half joking)
My interpretation based on experience is that the underqualified individuals will often be more prone to volunteering, participation in clubs and committees, and politicking. The overqualified individuals also do this, but will likely be required to do "the real job" more often, leaving a less qualified individual more free to do those things.
Community involvement is a significant factor on both enlisted and officer performance reports. Gotta fill that section in no matter what, and if your section is poor it drags your overall score down.
However, promotion testing is purely knowledge and skill based. A good test taker can overcome the weight of lower performance report scores.
Just my opinion, though.
Reminds me of the Gervais framework that someone made (as a kind of parallel to The Office) where you had three groups, confused, losers, psychopaths (don't read too much into the names). Confused work really hard at things they don't get recognized for (Dwight), psychopaths underperform and get promoted (Ryan).
I believe this is the article. Discussed on HN earlier:
https://news.ycombinator.com/item?id=881296
A Straight Shooter with Upper Management Written All Over Him
Cool, clever, noted plagiarist, a true breath of fresh air.
I had job mis-matches for years, and it helped me. I could do most of my work for almost all of my roles in half or less time. I Used the extra time to start successful businesses and now a successful consulting career.
how did you know what to do with the extra time? I’m in this situation with similar goals but no idea how to approach them
I simply play video games and ride my bike.
dang I could franchise this out to my kids and be 4x productive
https://www.nber.org/system/files/working_papers/w34215/w342... this is the direct link to the paper.
You should always - always - get jobs that you are underqualified for. Because someone even more underqualified than you will get that job.
I'm confused by the wording "we create simulated job assignments"
Was this entire study based on real data or a simulation?
I think they mean the job assignments were real, and what they're "simulating" is a randomised controlled experiment where they tell the Air Force who to assign to which position. The wording is confusing, but the methodology sounds reasonable.
It would be really interesting to know to what extent their definitions of being over or under qualified is an approximation of IQ test scores. IQ tests have lovers and loathers but a correlation or otherwise would say they are measuring something or not…
Total anecdata, but whenever I've had to take an IQ test for an interview process, it's never the IQ test that's used to disqualify me. It's usually something like culture fit.
But these days if a job requires me to do an IQ test to join, I'll use that as a signal to get the fuck out of there and find a different role. So again, anecdata, but I suspect I'm not the only one who would eschew those results.
I have never heard of IQ tests for hiring, is this for real? I've seen Myers-Briggs and similar personality sorting hats, but never IQ.
Amazon did this, at least for a time, as part of a "No See, No Hear" hiring pilot program.
The purpose was to see if they could hire university graduates with a minimum of human interviewing effort. They selected from a handful of universities, gave a couple online tests, verified the candidate's identity as the test-taker, then would give out offers sight-unseen.
I was hired this way in 2015. From my perspective, I had taken a couple online tests, then months later had a thirty-minute identity verification call, then a couple months later, was sent a job offer. I thought it was by mistake, so I didn't ask too many questions. I had a thirty-minute call with a hiring manager I otherwise never interacted with, then accepted, flew internationally back to the states to Seattle to start, met him and all my teammates for the first time on my first day of work.
I found the internal documents about this program later on spelunking in the internal wiki.
Interesting, thanks for sharing! Do you know what happened with the program and people hired this way? Was it generally successful?
that is wild! I could certainly see this as an attempt to eliminate hiring bias maybe? that was super popular in that time frame, but never heard anybody taking it that far.
It's funny how people pushed hard for removing hiring bias until they realized that this meant that you'd get even more people from unfavorable social groups.
Anyway, looking from today's perspective, this seems like an obvious attempt at automating the hiring process itself in order to fold the HR department. If only back then they had today's AI technology.
I applied for a management-consulting-ish job a decade ago (I was desperate!) at a big firm and had to take what was basically an IQ test. I have no idea if the test literally calculated my IQ, but the questions were exactly the questions you'd see in an IQ test (e.g. next item in some geometric sequence) so it may as well have.
This was in a group interview for recent university graduates at a very big company. I assume their hiring process was pretty standardized, so there were probably thousands of people taking this test every year in North America.
General IQ tests probably aren't legal in the US; Griggs v Duke Power Co [1] says (more or less) that employment tests have to be job related if the results of the test have disparate impact on protected classes of people.
It's hard to argue that a general IQ test is job related, but they're likely to have a disparate impact on protected classes of people.
[1] https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co.
This is a weird pernicious Internet myth. It obviously can't be true, because there's a big, well-known company that delivers these tests for employment/recruiting settings and they have a logo crawl on their page that include several giant companies. If those tests were illegal, employment lawyers would be making bank off it.
I'm not an expert/lawyer, but this does seem to indicate that the situation is a bit more complicated than either "pernicious myth" or "probably illegal" in general (but much closer to toast0's understanding); my interpretation is that you can either avoid an 80% threshold of "disparate impact" or you can in theory formally validate that a particular test measures/predicts performance at a particular job; that all sounds compatible with "companies do it in the open, but very few, and you can easily get in trouble for doing it wrong" https://www.law.cornell.edu/cfr/text/29/1607.15
The comment to which I responded claimed that IQ tests "probably aren't legal in the US", which is false. They aren't more widely used because they don't work well for candidate selection, but they are used by very large companies that are attractive targets for employment discrimination suits, and wouldn't be if they were legally risky. There are well-known tech companies that up until a few years ago gave IQ tests to candidates!
Empirical observation trumps axiomatic derivation in this case.
I’m from Northern Europe so YMMV, but in this country IQ/cognitive tests are quite common for senior roles or management.
I’ve had to take a few. I don’t mind too much. It’s mostly to test if you are WAY below what they expect for the position.
The personality trait tests are also quite common IME.
The military was pretty big on IQ testing, from what I understand. I’m not sure if this is still the case, but it seems like an efficient first pass to figure out where people should go.
They still do it; it's still very important to them. They maintain their own test, the ASVAB ("Armed Services Vocational Aptitude Battery").
https://en.wikipedia.org/wiki/Armed_Services_Vocational_Apti...
For a vivid picture of why the military is so insistent on IQ tests despite overwhelming political pressure to stop using them, you might like reading https://www.amazon.com/McNamaras-Folly-Hamilton-Gregory/dp/1... .
From what I recall, the ASVAB wasn't really a general IQ test. It was more like an SAT with a broader focus than academics, basically "Are you basically adequate with Words/Numbers/machines/ etc"
That's what a general IQ test is. Those aren't distinct concepts. The SAT is also a general IQ test.
Notice how your score on the ASVAB (and on the SAT!) is a percentile rank, not a count of items you got right.
Here's a sample question targeted at the Wonderlic, which is an IQ test that advertises itself as an IQ test:
> The words PERCEIVE and DISCERN have ___?___ meanings.
> A. similar
> B. contradictory
> C. unrelated
Here's one from the ASVAB:
> Quiver most nearly means
> [ ] shake.
> [ ] dance.
> [ ] rest.
> [ ] run.
Huh. The IQ tests that I was administered as a kid I remember as being primarily focused on abstract pattern recognition, which is what I think of when I think "IQ test". But I suppose due to IQ's definition, if you just give someone a wide enough battery of cognitive tasks, you can derive the IQ score that way.
I guess I always assumed that the ASVAB was used a bit more literally. "Ah, this person is barely literate, but knows all the parts of an engine. TO THE MOTOR POOL"
I had my IQ tested recently and the pattern recognition was part of it, but so were the word associations.
Depending on how young you were, maybe they weight it more toward the abstract patterns, because no kid isn’t going to have a fully developed vocabulary.
> The IQ tests that I was administered as a kid I remember as being primarily focused on abstract pattern recognition, which is what I think of when I think "IQ test".
That's just one specific IQ test, Raven's Progressive Matrices.
> I guess I always assumed that the ASVAB was used a bit more literally. "Ah, this person is barely literate, but knows all the parts of an engine. TO THE MOTOR POOL"
From https://www.officialasvab.com/applicants/faqs/ :
> there is only one exam, the Armed Services Vocational Aptitude Battery, or ASVAB for short. The ASVAB has 10 tests. Your scores from four of the tests—Word Knowledge (WK), Paragraph Comprehension (PC), Arithmetic Reasoning (AR), and Mathematics Knowledge (MK)—are combined to compute your score on what is referred to as the Armed Forces Qualification Test (AFQT). Scores on the AFQT are used to determine your eligibility for enlistment in the Army, Navy, Air Force, or Marine Corps. Scores on all of the ASVAB tests are used to determine the best job for you in the military.
You have to clear an IQ threshold to be eligible to enlist at all. Only after you're smart enough to be in the army will they consider which jobs you might do relatively better in.
> That's just one specific IQ test, Raven's Progressive Matrices.
Not OP but thank you, now I finally know what that test that I took as a kid is called!
If by "IQ test" you mean "any test that correlates well with other accepted IQ tests", then, sure, I guess. But there's a reason clinicians administer the Wechsler and not the SAT, and it's not just that the SAT takes longer.
Sure, the reason is that they want to do something that is "the same" as what they've done in the past, so that previous research will remain as valid as they hope it is.
My mother is an obstetrician, and something that has always bothered her is that American hospitals have women lie on their backs to give birth. This is not a natural position, it's not comfortable for the women, and it can make it more difficult to get the baby out.
So why do we do it?
The answer is that, a long time ago, doctors who assumed that that was the correct way to give birth developed a set of standard measurements that determine what doctors today think of how far into the labor process a woman is. These measurements are only valid for a woman lying on her back - they will change if she shifts positions. They would have to be redone and revalidated for a woman in a natural delivery position. And nobody wants to do that.
The SAT correlates as well with any given IQ test as other, "official" IQ tests do. It is an IQ test. It serves all of the purposes that IQ tests serve, and it cannot serve any purpose that they can't.
It is more accurate than some very standard "accepted IQ tests" such as Draw-a-Man. ( https://en.wikipedia.org/wiki/Draw-a-Person_test )
But it's important to some people not to call it an IQ test. Try not to be one of those people.
Not a hill I'm going to die on, but the SAT has many attributes IQ-ists insist IQ tests are insulated from: it's straightforwardly trainable, culturally loaded, samples only math, processing speed, and verbal reasoning, and tracks prior educational experience as much as it does aptitude.
Draw-a-Person basically isn't an IQ test at all, so I don't see how that comparison clears anything up.
> Not a hill I'm going to die on, but the SAT has many attributes IQ-ists insist IQ tests are insulated from
This is false. In particular:
> it's straightforwardly trainable
No, it isn't. There is an extensive literature on SAT prep, finding that it's worth a couple of points on the test. It is widely described as being trainable, but the opposite was always a design goal, and historically that goal was achieved very well.
You might note that the Raven's matrices are infamous for huge training effects; that test relies on the testee having never seen it before. The SAT doesn't.
> culturally loaded
This claim is true, but nobody claims that IQ tests are insulated from being culturally loaded. The purpose of Raven's is to be a culture-free test. Wechsler makes no such pretense.
> samples only math, processing speed, and verbal reasoning
I'm not sure what you're saying here.
> and tracks prior educational experience as much as it does aptitude.
And this one is false. The point of the SAT is to test only low-level material so that you can be confident the entire test-taking population has been exposed to the material. Aptitude has a very large influence on SAT score; prior education has a negligible influence.
(Prior education will have a larger influence if the population you're investigating includes a lot of people with no education, the kind of people who left school after or before kindergarten. But that scenario isn't relevant to... pretty much any question about the SAT.)
> Draw-a-Person basically isn't an IQ test at all, so I don't see how that comparison clears anything up.
It is an IQ test by the standard you defined: it holds itself out as being "an IQ test", and it is used by researchers to study the intelligence of testees. Did you want to use a different definition?
Now I think you're the one defending a weird hill, because math is like half of the SAT, and trig is a learned skill, not a general cognitive ability.
I've had some IQ tests, one company is now bankrupt, was quite a famous Danish bankruptcy (they offered me the job but I turned it down which looked prescient a couple years later when they went bankrupt), another was Klarna, Boozt also uses a cognitive assessment.
Klarna's seemed like the most proper IQ test although it had at least one question that was wrong.
Just search (both on HN and the internet more broadly) for Canonical (the company that created Ubuntu) hiring horror stories
Yes. Several large companies have been administering them (or equivalent general [non-domain-specific] cognitive aptitude tests) for decades.
It used to be VERY popular in enterprise, where HR professionals often coming from psychology handle most of the hiring.
I have no idea why they did this, I guess that was the idea of a hiring process at the time.
My personal theory is that it's an easy way to say "no" if nothing else is easier.
"sorry but it says here you're stupid"
Military takes the ASVAB which is basically the SAT / IQ Test which qualifies you were certain jobs. For example Nukes in the Navy (which is considered one of the hardest programs for obvious reasons) is 252 score for some combination of scores.
I've worked with at least two such "Nukes". One a researcher, the other a tech startup founder. Both very smart, team-oriented, amiable and down to earth, and inspiring respect (through their ability and how they conduct themselves).
I suppose, on a sub, besides whatever admissions filtering they do for aptitude, you have to take work seriously, and also cooperate and get along with people.
If the ASVAB is basically an IQ test, I’m horrified by how quickly my intelligence has decayed.
When I took it in HS, I scored quite well. I’m almost certain I’m bomb it completely now. Particularly any mathematics I haven’t touched since high school would look foreign to me.
It's exactly what it is. When someone enlists in the US military, they take what is called the ASVAB (Armed Services Vocational Aptitude Battery). It's more specific to the military than Stanford-Binet, but effectively it is still an IQ test. Specific military occupational specialties have minimum score requirements to qualify for that job. Which job you actually end up getting is a combination of scoring high enough to qualify, scoring higher than everyone else who wants the same job, doing well on other things they assess you on, and needs of the branch of service at the time you enlist. Score high enough for cyber but they only need mechanics, then you're becoming a mechanic. Conversely, if they badly need cyber professionals but nobody scores high enough to meet the official bar, they're still putting people into that job. Hence, why the mismatch exists in the first place and why there's a study like this.
My understanding is that IQ tests can measure a type of intelligence, but fixation on its narrow metrics can lead one to overlook other attributes that are just as likely to be relevant, but aren’t something that necessarily shows up in standardized tests. Add to that the fact that IQ tests can be heavily biased, and leave a lot of ambiguity for the proctors to interpret, it’s not a surprise that they’re so controversial.
I personally would be very suspicious if asked to sit for an IQ test as part of a job evaluation. I have worked for places that blindly worship context-free performance metrics, and it was insufferable.
IQ is a correlation variable that pops out when you measure any group of people's aptitude at a battery of tasks that involve thinking. Basically, if someone's good at one task that involves thinking (say, chess), they're more likely to be good at another task that involves thinking (say, reading comprehension). Apply some Bayesian statistics, and boom, you've concluded that there's some confounding variable (that we call IQ). Then you can start measuring how strongly certain tasks correlate with variable. Turns out abstract pattern recognition very strongly correlated with this unknown variable, so we can use that to predict what someone's IQ likely is.
The point is, due to the very definition of IQ, it's not a narrow metric, and selecting for it does tend to find you individuals who are going to be better than average at most anything. That said, it would seem alarming to me for a job to give me an IQ test instead of cutting out the correlation coefficient and just judging me on the task they're hiring me for.
Perhaps measuring how well people perform on IQ tests?
I was a mismatched match for my first job and it led to a career in IT. My qualifications were that I was young, cheap and didn't know any better.
"Combined, these patterns suggest that overqualified individuals are less motivated, but still outperform others in their same job. Underqualification results in a polar opposite set of findings, suggesting these individuals are motivated to put forth more effort, but still struggle to compete when judged relative to others."
Did Captain Obvious wrote this working paper or all NBER research is like this?
Well I worked as a civilian in that branch a decade ago. Purely technical work, and I got the assigned tasks done in about 7 months.
You also have to look at the culture of the place- and although that varies from base to base, whether or not someone is more or less qualified may be completely irrelevant. Cultural fit is very important, for example. Favoritism may be rampant, just like any other workplace. High performers who concentrate solely on the relevant tasks have less time to make the rounds around the office, playing petty politics, just like anywhere.
While I agree with the sentiment, I don’t think it applies to high executive positions in the private and/or public sectors.
Too under qualified is still a continuous train wreck…
Job mismatch can really set someone back early in their career. It's crucial to find the right fit from the start.
But sometimes, those mismatches lead us to discover what we truly love.
Every detour can be a learning opportunity.