Simple simon says…

Whenever I am in the company of people who I have not met before, the conversation – as it tends to do in situations like these – comes around to what we all do for a living. When I explain my position at Jack in the box people invariably question “So, you do surveys and stuff?”

It is obvious by the tone of these questions people generally feel comfortable enough with research methods such as surveying as to express an opinion about it. The ensuing conversation usually involves people expressing these opinions while I find myself defending the science of properly conducted research. And I have found that the research method most prone to scepticism is the poorly constructed survey.

I fear consumer surveys have been so abused at the hands of inexperienced marketers, the humble survey has now become the laughing stock of the research world. But it’s not just marketers who have garrisoned client surveys to the bottom of the research heap – business owners and managers also have a lot to answer for. I am tempted at this stage to pull out the old expression “If I had a dollar…”, but it’s true. If I did have a dollar for every half baked, unfocused, poorly expressed survey I have seen in my time as a consumer analyst I would be a very rich woman. But disregarding who played the largest part in the demise of the survey, why is it that one of the most useful tools for businesses and consumers alike has been so misused and misunderstood?

I think the abuse of the survey research form is enabled in part by the construction of the survey itself. A well constructed survey is perfect in its apparent simplicity. Unfortunately, this ‘effortless’ simplicity has convinced millions of inexperienced and untrained people that they too can successfully execute a survey based research project with ease – a skill I might add which takes years of experience and study. If I sound a bit grumpy about that, it’s probably because I am. I spent years studying the intricacies of research statistics in order to produce meaningful and actionable research results, so when I meet people who think they can throw together a survey in half an hour, I admit it stings a bit. (See Tony’s blog about his similar frustrations).

Some believe they have seen so many surveys whipping one up themselves shouldn’t be a problem, but that’s like saying you know how to fly because you have once been a passenger in an aeroplane. In both examples, sure you can try – but I don’t like your chances of success! For the apparently least convoluted of research methods the humble survey is really quite complex. How much do you know about sampling design, confidence levels and standard errors? How do you ensure the questions you ask avoid bias? How do you know whether to run a descriptive or experimental study? In fact, what is a descriptive or experimental study?

If you think these considerations are a little too complex for a ‘simple’ survey, let me give you a tip – there is no such thing as a simple survey. Surveys that seek to deliver insights or answer questions, that is surveys which are useful and actionable, are not created by bunching a few questions together and hoping for the best, successful surveys are the product of a researcher with years of expertise and experience. And that folks is as simple as it gets.