As a seasoned stats geek that is fairly new in the market research industry, I quite often find myself in a unique position that my Masters at university didn’t quite prepare me for. My time at uni taught me how to explore data sets and how to investigate statistical trends with a room full of other statisticians. At no point was I questioned about what I deemed were simple statistical terms. I was never asked to explain the techniques that I used so often they had become as second nature as my ABCs.

Coming from University statistics to the real world took a bit of time of getting used to. All of a sudden, I wasn’t looking at perfect data sets – a number of people were answering ‘don’t know’, or ‘not applicable’ – and I really had to step back, go right back to basics, and re-assess how I interpreted data in general.

An article I recently read in the New York Times really struck a chord with me – it highlighted how people often get hung up on statistical results without thinking about the repercussions of the analysis. Everything, not only in statistics, but in general, has a level of uncertainty attached to it. I’m pretty certain I’ll be cycling into work tomorrow morning, but let’s be honest, there’s a chance I’ll snooze my alarm. If this is the case, I’d expect to catch the usual 08:15 tram into town. If you’re from Manchester, you’ll know there’s a pretty high possibility that the trams will break down. You can only ever guess – nothing is certain.

This particular article I mentioned focused on the uncertainty in the job market predictions, and how this could affect the monthly headlines. Let’s look at the example used in the article – imagine the job market added 150,000 jobs each month (of course, there’s a level of uncertainty added to the monthly estimates). Once you add this level of uncertainty, there is a 4% chance that reports will show the job market added less than 55,000 new jobs. There’s also a 4% chance that estimates will show that over 245,000 new jobs had been added. 

One line sums it all up quite nicely – “we all actually know a lot less than you might think.”

So how do we get around this? I am always saying the same thing to everyone that I work with – it’s all about common sense. With anything from simple cross tabulation analysis to a segmentation to any other technique – it all needs to be run by people who know the ins and outs of the project, understand the research context, and can spot a dodgy looking result from a mile off. Whenever we run statistical analysis, we always ensure that everyone involved in the project is part of a discussion around the results – does anything stand out as being a little odd? 

 

If you’re finding that this is the case, follow the same rules that apply when eyeing up those dishes on that “multi-cuisine all you can eat”. Chances are if it looks dodgy, it probably is.  

Bethan is a Senior Researcher at Mustard – and would be more than happy to geek out or discuss other rules of buffet etiquette on Twitter.