It's true! I do enjoy having fun!

The Typealyzer plays around with all that psychology that gets its popularity by telling you about you and convincing you that it's accurate because it tells you so many things that you've thought about yourself.

It's called the Forer effect. (My thanks to Nancy Friedman for having provided the term several months ago in a delightful post.)

To get your pop psychology degree all you have to do is say 'You like to understand things' and the biologist thinks 'Naturally. That's my job' while the musician cries out 'Well yes! Music is very logical after all' and the athlete says 'That's the secret to my winning record!'

So I suggest two standards to see if these tests are worth anything at all. Just because a profiler passes my little tests doesn't mean it's a good analysis based on sound principles. Let's call these bare minimum standards. They're really just watered down versions of validity and reliability.

Test 1: Can I recognize the appropriateness (validity) of the test's profile?
As long as the test isn't wrong about its claims it passes this test. This is why the tests do well consistently. The descriptions are so bland as to be appropriate to almost everyone. Not all that meaningful, but at least it's not providing false results. I call this is a watered down version of validity because these analyses are so general that they fool the subjects into thinking that something has been measured. They pass the validity test on a technicality.

Test 2: Can the profile recognize me?
This one is harder to run. But there's a simple way to verify its ability. As long as the analysis hands out different answers. Give yourself a different identity and see if it says the same thing about you each time.

I have another blog where I don't talk about linguistics, I don't analyze language much, I don't use technical terms, but I'm still writing as myself. At times when writing a post here or there I get confused and can't remember on which blog I've posted, even tho the topics are pretty much in a complementary distribution. This makes for a decent test of the reliability of Typealyzer because I'm not changing the important variable. And yet we find...

Here I'm a Mechanic.

The independent and problem-solving type. They are especially attuned to the demands of the moment are masters of responding to challenges that arise spontaneously.

But over at my other blog I'm a Thinker.

The logical and analytical type. They are especialy attuned to difficult creative and intellectual challenges and always look for something more complex to dig into.

These are compatible, sure. And you might argue that I take on a different persona at each blog. But it's claiming to know how I use my brain. And this is most simply a test of one person administered twice. And it's choosing to give a differing analyses for a reason. Probably because it's not really analysing my personality.

Typealyzer does measure something that gives it the appearance of validity and reliability. It somewhat accurately measures the topics covered based on the words found (maybe it even counts sentence length, who knows) and it categorizes the writer according to the role being played. And if it looks at 20 posts, 50 posts, or every post on a single blog, and if it runs the test again and again it's probably going to reliably come up with the same result. Or a similar result.

But that's where the claims fall apart. Because we were promised a look at our brain and our personality. But it's just telling us about our topic. Our style. If it was really digging into the way I look at things, at the way I think, it should be able to analyze me reliably no matter what I'm writing about. And it doesn't.

But let's back up. These profiles actually do get some things very wrong.

Mechanics:

enjoy working together with other independent and highly skilled people

I hate working with people.

often like seek fun and action both in their work and personal life.

Only if C-SPAN counts as action. And really -- how cheap is that fun part?

enjoy adventure and risk such as in driving race cars or working as policemen and firefighters.

Uhh… No.

Thinkers:

are not very good at seeing and understanding the needs of other people

Oh. Well…fine.

might come across as arrogant

Oh please.

impatient

Now this one I can disagree with honestly.

insensitive to people that need some time to understand what they are talking about.

Well if I haven't made my point by now…

---

I first saw this at Mr. Verb (a fellow Mechanic) then at Language Log where David Beaver tackled it gently.