8 Simple Rules for Choosing an online Personality Test

Psychometric test development is a combination of intuition, inspiration, vocabulary and statistics. A test developer needs to have the intuition to see patterns in human behavior. They need to have the inspiration for seeing the underlying sources of those patterns. They need to have the vocabulary for defining those patterns in tests (through several revisions!). And they need the mathematical background to check the validity of their instruments. The thing is, if you don’t know how any of those steps work, you can make bad purchasing decisions.

Judging the quality of any particular personality test is difficult if you don’t have the right background information. Without that information, you cannot tell the real medicine from the colored sugar water. The normal signals for quality (adoption rates, attractive design, pricing, etc) are almost totally useless in this arena. I have seen tests online that look like they were designed in 1995 that were high-quality and valid instruments. I have also seen very slick web sites and reports that were not much more valuable than a horoscope.

So, how can you, the normal human being who just wants to use a personality test but not spend 6 months becoming an expert, tell a good test from a bad test? This is where I am going to save your bacon, your sanity and quite possibly the future of your company with a few 8 Simple Rules for Choosing an online Personality Test. Read the description of the test and poke around the publisher’s web site. After doing that, let’s look at the following Rules.

Warning sign: Avoid the Woo-Woo

If the ‘inspiration’ for the test came from a hermit on a mountain top, discard.

If, at any point, the description points to some form of “Ancient Wisdom”, discard.

If they try to connect the “magical powers” of numbers (or any magical powers at all) to personality, discard.

Any use of the word “Mystical”? Discard.

Confusion

You have read the description of the test and what it is measuring. Can you explain it to someone else in just a few minutes? I have noticed a trend in ‘bad’ test developers to try to baffle the reader with overblown terminology and contradictory claims. A test should be measuring something specific and what it is specifically measuring should be easy to formulate into a few sentences and explain to someone else. If you cannot explain it to someone else, how are you going to explain it to the candidate, your customer or yourself?

The purpose of personality testing is to take something very complex (human behavior) and explain it in a simpler framework so that it is understandable. If the description or test results are not doing this, discard.

Horoscope

You know how the horoscopes you read in the newspaper are super bland and can be applied to just about anyone? How the claims are so general that if you handed the same one out to 10 different people, 9 of them would say that it was written especially for them? This is a sure sign that the test is, if not wrong, is useless.

The official word for this is ‘Discrimination’, telling people apart. It’s really easy to try this out. If three people take the test and you remove the names from the reports. Can people who know them tell which report is for which candidate? If they can’t, then the test is failing its basic function of describing a person accurately.

If the report reads like a fortune cookie. Discard this test. In bed.

Not even wrong

This is a subclass of the Horoscope kind of report. Every statement in a report can be absolutely true, but totally meaningless.

Here is an example of a statement that is true and meaningless at the same time, “You need to breath air.”

I class these types of tests and reports as not even wrong.

A good test should tell you something meaningful and useful that directly relates to your question.

Too much milk from the cow

The description of the test should describe what the test measures in clear terms and a good exercise is to count up all the things a test measures (x). Look at how long the test is, count up the number of items (y).

If 4x>y, discard.

Just kidding. I won’t make you do math.

Keep in mind that a single test measures one area of human behavior. As soon as someone starts claiming their test measures 5 or 6 different areas and intelligence and your shoe size and your political affiliation and your whatever, discard.

I once saw a publisher offering a 90 page report based on a test with only 4 questions. It made me want to cry, it really did. People base their careers on these results! Real people’s lives are really affected by this information and seeing someone put an instrument out there like that just really upsets me.

Model

A personality test must be based on a model of human behavior. This is the theory or pattern that the test is trying to measure.

The model needs to be understandable and explainable and related to your question. A model is also quite generic, there might be several tests that are based on the same model of behavior. This is a good thing. Having a lot of tests based on your model is not the sole criterion of quality, but having no other tests based on a model is a red flag.

New models are proposed all the time, but it usually takes at least 5 years for it to be fleshed out, challenged, refined and justified/refuted.

A web of relationships

Test publishers publish and they pay attention to what is going on with other publishers. Tests are not created out of thin air, they are usually evolutions of earlier tests and models. There should be a trail explaining where the test came from and what earlier research was referenced. There is not much of a line between “totally unique” and “made up”.

New tests are validated by using other, validated tests. People take the new test and the old test. The model predicts certain relationships between these two tests. If the experimental data doesn’t show those relationships, then there is a problem with the test or the model.

A test should have some documentation describing how it relates to other, high quality, tests. If it’s not published on the web site and the publisher won’t provide that information, discard.

Black and White

Nobody puts Baby in a corner and nobody should try to put you in one, either.

People are variable and when a test tries to force you into a box, red flags should be raised. You are not a binary yes|no bit, you are on a bell curve, just like every other biological system.

So, when the results of a test try to express the totality of you by pointing to one of a few boxes, just say, “Discard”.

Numbers

Behind all of this personality and theory and models and tests, there are a bunch of numbers. I’m not saying that you need to know what all of the numbers mean, but you should know that there are different numbers and a test that puts all of its claims on just one of them is probably hiding the numbers that aren’t so good.

If they don’t have the numbers on the web site, ask for them.

“Can you please share the results of your validity research on test X with me? For example, I am interested in the Cronbach’s Alpha for the traits measured. ”

If they so no, red flag.

If they don’t know what you are talking about, run away!


I hope you have found this information helpful. If you have any questions or suggestions for improving this list, I would love to hear from you richard at oakbox.com.

 

 

Spread the love