We often take for granted how much technology has improved our ability to get things done. Processes that used to be time-consuming or expensive are today so automated that they happen in a flash. One prime example: the way that market research used to be done.
Businesses have always wanted to get input from their customers. Anyone who has a product or service to sell wants to know how satisfied the customers are, the source of any problems, and the attributes that matter most. If the business is contemplating adding a new product, or spending money on a new advertisement, it behooves the company to learn whether the proposal attracts the right people.
Nowadays, it takes only a few minutes to put together a survey using an online tool like SoGoSurvey and to distribute it to email lists. In only a few days, you get a clear picture of the customers’ viewpoints, and you can make better business decisions with accurate data.
But before computers? You could still accomplish this — but it was far more expensive, it took longer (measured in weeks at best), the process was more error-prone, and it required a lot more people to do the research)
For instance, let’s say a well-known food manufacturer wanted to test a new product, such as a new salty snack to add to its line of pretzels and potato chips. The manufacturer’s marketing plan would include the discovery about everything from how the food actually tasted (too salty? Not salty enough?), learning which market segments liked it best (teenagers? Busy homemakers? Which income ranges?), and creating TV ads to help those people discover it.
In each case, the manufacturer would conduct at least one survey. The product marketing department would draw up a list of questions — just as it would today — including some questions to be answered only by a subset of respondents. For example, if the answer to, “Are any children under 18 years of age living in this household?” was No, follow-on questions about the children’s ages would be omitted.
That survey would be printed on legal-size paper, one for each respondent. Many surveys were 20 pages long, and a research project with 1,000 respondents might be conducted in perhaps 10 cities. The manufacturers took this seriously, as there is a lot of money in selling salty snacks!
In any case, this generated several boxes of paper. The market research agency contracted with interviewing services in each of those 10 cities. Each one needed to acquire qualified respondents on an aggressive timeline and to ensure that trained interviewers asked the questions, recording each answer on those pre- Printed forms. While today’s software easily can be programmed to skip questions (if no children in the household, jump over the related questions), an interviewer needed to ask the questions aloud, if for no other reason than to ensure the person didn’t skip ahead (“Oh they are obviously asking about…!”). Back then the staff member of the interviewing service sat in a quiet room, asking questions and taking notes (hopefully in readable handwriting and using complete sentences).
Once the surveys were shipped back to the market research agency (in the US, 90% of these firms were in New York City), the data needed to be collated and “coded” before it was collected in some kind of primitive data gathering process.
For example, an interviewing service might be based in a shopping mall in Miami, and employs a few dozen interviewers. First the interviewers had to stop strangers in the mall to convince them to participate in a market research study (a non-trivial task), and then confirm that a willing interviewee met the criteria (such as buying salty snacks to eat at home and being a female between the age range of 25-35). Then the staff member had to drag the respondent back to the office for a 20-minute interview (often with a squalling child).
Once the surveys were shipped back to the market research agency (in the U.S., 90% of these firms were in New York City), the data needed to be collated and “coded” before it was collected in some kind of primitive data gathering process. The market research agencies employed coders to perform quality checks on the data (e.g. Making sure when children’s ages were entered, the “are there children…” question was marked “yes”) and to categorize the answers to “open ended” questions (such as “What do you think of the taste of this snack?”).
Given the difficulty of gathering data, it was cumbersome, error-prone, and time consuming to analyze the results. Even with basic computer databases (starting with punch cards and rented time on a minicomputer time sharing service), an analyst had to put in a separate request for each possible relationship. With online systems today, you easily can tell the survey tool, “Show me the results cross-tabulated by age…? By income?By number of children?”). But back in the old days, each such request required, separate computer coding to extract the results. That discouraged the “what if?” process from which enlightenment may result.
Have you kept count of the number of human beings involved in this process? Each participant had an opportunity to mess things up… and they often did. Which added yet another step: Quality checks a sample of the surveys. Someone called respondents on the phone (this was before email of course) to ask if they had taken a survey a few weeks ago, and to confirm that the data had some measure of accuracy.
The author is co- founder & CEO of SoGoSurvey