Why our test now includes words that don't exist

Since 10 September, some of the words in our test aren't real. They look like English and sound like English, but you won't find them in any dictionary. If you've taken the test recently and paused over a word like "brallow" or "fendicate", that's what was going on. Here's why we added them.

People say yes too easily

A test like ours asks you which words you know. The trouble is that people tick words they've only seen before, words that look like something familiar, or words they feel they ought to know. Almost nobody is lying. Recognising a word is a fuzzy feeling, and it's easy to mistake "that looks familiar" for "I know what that means".

Until now the test had no way of telling the difference, and every extra tick pushed the score up. It mattered most in the step with rare words, because each of those answers stands for a large part of the language. A few generous guesses there could add thousands of words to a result.

Words that don't exist

The made-up words were built to look and sound like real English, and then checked against several dictionaries to make sure they aren't real words or misspellings of real ones. They're spread through all three steps, and nothing marks them out from the real words around them.

Since nobody can know them, each one you tick shows how freely you're ticking. The test uses that to adjust the rest of your answers: the more made-up words you tick, the less your other ticks count. If you tick none, your score isn't affected at all. Each one also costs more than the one before: a single slip barely matters, but ticking a lot of them brings your score down sharply, because those answers no longer say much about your vocabulary.

We didn't invent this idea. Researchers have used made-up words in vocabulary tests since the late 1980s, and the correction we use is based on one published by Huibregtse, Admiraal and Meara in 2002. Our methodology page explains it in more detail.

What happened in the first days

Between 10 and 14 September, 135 tests were completed with the new version. Each one included between 14 and 36 made-up words, depending on how many steps the test-taker went through.

Almost half of the test-takers didn't tick a single made-up word, and another third ticked just one. A single slip like that is common and changes the result very little. Only about one person in ten ticked three or more.

What it means for you

The advice is the same as before: only tick a word if you could explain what it means. If you aren't sure, leave it. Guessing won't raise your score any more, it will only make the result less certain.

If you've taken the test before, don't compare your new result too closely with an old one. The way the score is calculated changed at the same time, so the two numbers aren't on exactly the same scale. From now on, results are comparable with each other again.

Further reading: Meara, P., & Buxton, B. (1987). An alternative to multiple choice vocabulary tests. Language Testing, 4(2). Huibregtse, I., Admiraal, W., & Meara, P. (2002). Scores on a yes-no vocabulary test: correction for guessing and response style. Language Testing, 19(3).