I've been thinking about Watson Glaser tests recently.
I find them rather curious (and frustrating, like many of us), especially because I have a particular relationship with the kind of "critical thinking" they are supposed to test.
I have taught critical thinking to students at BA and MA level, and have published several book chapters on the philosophy of interpretation. It is very important that I resist the urge to send long emails to grad recruitment teams saying: "Here is my expert opinion on why WG is a poor way of assessing candidates..." But I don't think that will really help my chances of getting a TC! You've got to pick your battles.
TCLA seems like the place to vent about this kind of thing, though, so, time to get on my soapbox...
WG was famously critiqued by Professor Kevin Possin in the journal
Informal Logic, Vol. 34/4 (2014), and I have to agree that the methodology has some serious flaws. The issue is neatly summed up in Possin's subtitle: "The more you know, the lower your score". WG has a very limited conception of what "critical thinking" actually involves, and often encourages poor judgement.
Although WG candidates are encouraged to identify a single correct answer, it is often possible to make a strong case that several answers are entirely logical and intelligent responses. Sometimes, the most rigorous logical analysis will, in fact, identify a supposedly "incorrect" answer. I would often expect the best lawyer/critical thinker in the room to get the WG test "wrong", at least for several questions each time.
The reason this happens is actually quite simple. The parameters of each question establish a line where inference should be limited. Although candidates who cross that line are penalised (rightly) for assuming too much when making a judgement, the test also penalises candidates with a more developed critical sense who draw that line at an earlier point in the process of assuming relationships and causal connections. Possin breaks this down in some detail, and I would agree with much of it.
When teaching critical thinking, my students would find it easy to undermine the methodology used for many of the standard WG questions. I wouldn't got so far as to say that WG rewards "less intelligent" or less critical candidates. But what does seem to happen is that in order to succeed, candidates have to become acclimatised to the "rules of a game" - the Watson Glaser game - which does not necessarily equate to an ability to rigorously assess data.
(Mind you, that is not a million miles away from what lawyers actually do. The law of England and Wales is a big book of "game rules" that have been evolving for centuries, by a combination of case law and legislation. So maybe there is some logic in it!)
Ultimately, I am glad to know that many firms use WG tests within a more holistic application system, taking other factors into account. We have seen that many times here on TCLA. A low score is not the end of the world.
In the meantime, I'm just going to answer everything with "insufficient data"
...
Any thoughts about this?