Thursday, July 2, 2015

Survey: The Majority of Americans Believe South Carolina Should Remove the Confederate Flag

CHARLOTTE, NC July 2, 2015 – According to a recent survey by Accelerant Research, nearly two-thirds (60%) of American adults are in favor of South Carolina removing the Confederate Battle Flag from its State House grounds, while approximately one-quarter (27%) of Americans believe the flag should remain. 

Opinions about the flag vary sharply across different demographic, political, and (to a lesser extent) geographic subgroups of the U.S. population.  Only two in five (40%) Republicans support the flag’s removal, compared to more than four in five (82%) Democrats.  Nearly nine out of ten (89%) African Americans want the flag removed, compared to 55% of Caucasians and 53% of other ethnicities.  Generationally, 68% of Millennials (18-34) think the flag should be removed, while 62% of Generation Xers (35-49), 58% of Baby Boomers (50-68), and 57% of Traditionalists (69 or older) share the same opinion.

In terms of geography, opinions are not as different as many might expect.  Residents of Southern and Midwestern states are equally less likely to be in favor of the flag’s removal (58%), compared to those in the Northeast and Western states (65%).

When asked, in their own words, to explain why the flag should stay or go in South Carolina, Americans offer strikingly similar language in their arguments.  Words most often used to argue both in support of and in opposition to the flag’s removal are “history,” “racist,” “people,” “symbol,” and “slavery.”  The most notable difference between the two sides of the argument are that those in favor of keeping the flag are far more likely to use the word “heritage” to explain why, while those who want the flag removed are more likely to use the word “hate.” 

Accelerant Research conducted the South Carolina Confederate Flag survey online among a demographically representative sample of 2,044 Americans 18 years of age or older.  For more information about this study, please contact Accelerant Research at

About Accelerant Research:
Accelerant Research ( is an independent, nonpartisan, full-service marketing research and consulting firm headquartered in Charlotte, NC, which specializes in customer loyalty, product development, customer satisfaction measurement, modeling, tracking, qualitative research, and brand analytics.

Sunday, May 24, 2015

Memorial Day Survey: Americans Without Military Family Members Far Less Likely to Understand Holiday’s Significance

CHARLOTTE, NC May 24, 2015 – According to a recent survey by Accelerant Research, the vast majority (92%) of American adults are at least aware of why Memorial Day is celebrated, however, those with an active or retired military family member are substantially more likely to understand the holiday’s significance (95% vs. only 86% for those without a military family member).  Interestingly, no matter how distant the relationship (whether military family members are immediate family or extended family), any presence of a family member who is serving or has served has the same impact on correctly defining Memorial Day.

Of the one in 12 Americans who do not know why we celebrate Memorial Day, most identify the holiday as the first day of Summer, while a handful confuse Memorial Day with Independence Day.

The survey also investigated how Americans plan to spend their time during the holiday weekend.  Half of Americans are planning a barbecue/cookout, while one in three plan to visit with loved ones and one in four plan to visit a veterans memorial or monument.  Again, the presence of military family members has a substantial impact on Memorial Day weekend plans, with one in three Americans who have active or retired military family members planning to visit veterans memorials, compared to only one in ten Americans without military family members.

Accelerant Research conducted the Memorial Day survey online among a demographically representative sample of 1,459 Americans 18 years of age or older.  For more information about this study, please contact Accelerant Research at

About Accelerant Research:
Accelerant Research ( is an independent, nonpartisan full-service marketing research and consulting firm headquartered in Charlotte, NC, which specializes in customer loyalty, product development, customer satisfaction measurement, modeling, tracking, qualitative research, and brand analytics.

Wednesday, April 1, 2015

The Use, Misuse, and Abuse of Sampling Error, Confidence Intervals and Statistical Significance

Sampling Error 
There are about 250 million adults in America, of every imaginable background and circumstance. So how can a survey of only 800 or 1,000 adults reflect what the entire country is thinking?  How can a thousand voices speak for them all?

Marketing researchers liken it to making a big pot of soup — to taste-test the soup, you don't have to eat the whole pot, or even a whole bowl's worth. You only have to try a spoonful or two. The same is true of marketing research. You don't have to ask every single person in America to find out what Americans think; you only need to ask a few to get the flavor of the population’s opinion. 

This fact is reflected by a survey's standard error of the mean. Specifically, the standard error of the mean is an index of the amount of error that results when a single sample mean is used to estimate the population mean, it is an index of sampling error.  The standard error of the mean equals the standard deviation of the population of raw scores divided by the square root of the size of the sample on which the means are based.

So, a subject highly related to sampling error and standard error is the “margin of error.”  The lower the margin of error, the larger the sample, and the more accurately the views of those surveyed match those of the entire population. 

When marketing researchers report the margin of error for their surveys (usually expressed as something like "plus or minus 3 percent") they are stating their confidence in the data they have collected.

Confidence Interval
You must also remember that every margin of error has a "confidence interval," usually 95 percent. That means that if you asked a question from this survey 100 different times, 95 of those times the results would be within 3 percentage points of the original answer. Of course, this means that the other five times you ask the question, you may get answers that are completely different. 

For example, if 50% of a sample of 1,000 randomly selected Americans said they are satisfied with their bank, in 95 cases out of 100, 50% of the entire population in the U.S. would also have given the same response had they been asked, give or take 3 percentage points (i.e., the true proportion is somewhere between 47% or 53%).

The bigger the sample, the smaller the margin of error, but once you get past a certain point -- say, a sample size of 800 or 1,000 -- the improvement is very small. The results of a survey of 300 people will likely be correct within 6 percentage points, while a survey of 1,000 will be correct within 3 percentage points, a lower margin of error. But that is where the dramatic differences end -- when a sample is increased to 2,000 respondents, the margin of error drops only slightly, to 2 percentage points. 

Despite this, some surveys have sample sizes much larger than 1,000 people. But why ask two or three thousand respondents when 800 will do? Well, it sounds more impressive, but that's hardly worth the cost of interviewing all those additional people. Usually when a study has a large sample, it is so certain subgroups can be isolated and compared to other subgroups or to the total sample. If you want to compare retired people to the general public, for instance, a sample of 1,000 might yield only one or two hundred people who are no longer working, which may not be enough to get a solid grasp on the views of that group. A sample of 2,000, however, will probably yield a larger group of retired Americans and provide a more accurate picture of their views, which can be compared to non-retirees’ opinions.

Sometimes increasing the overall sample size is not enough, if the subgroup you are examining is rare or particularly hard to find. Affluent households, for example, make up only a small percentage of the U.S. population. In a standard random sample, you would have to interview an enormous number of people before you had a large enough subgroup of affluent households. In this instance, you would take an "oversample," purposely seeking out members of the “high-net worth” group you are interested in, and comparing the results to the main sample. 

Of course, in both general samples and oversamples, who is asked is as important as how many are asked. Reputable survey organizations go to great lengths to make sure their interview sample is random and representative of whomever they are surveying, be it retired, affluent, or all Americans.

Statistical Significance
Sometimes, even the best researchers misuse and abuse the concept of significance.  Many in research pour over reams of cross-tabulations and perform a multitude of analyses to find significant differences and formulate their decisions based on statistical significance.  They tend to associate statistical significance with the magnitude of the result.  Their reasoning is something like this:  “The more statistically significant a result, the bigger the difference between two numbers.”  In other words, the fact that one proportion is significantly different than another suggests that there is a big difference between the two proportions and statistical significance is often associated with “bigness” of a result. 

People often think that if the difference between two numbers is significant it must be large and therefore must be considered in the analysis.  It is suggested that when comparing numbers, two types of significance should be considered:  statistical significance and practical significance.  By understanding the difference between statistical and practical significance, we can avoid the pitfall that many in the research industry make.

What does statistical significance mean?  A significance level of, say, 95% merely implies that there is a 5% chance of accepting something as being true based on the sample when in the population it might be false.  The statistical significance of an observed difference depends on two main factors:  the sample size and the magnitude of the difference observed in the samples.

For example, let’s say we do a significance test between two groups of people who are exposed to a product concept and find a 20-point difference between Group A (65% acceptance) and Group B (45%).  Is the difference statistically significant?  Despite the large magnitude of the difference (20 points), its statistical significance will depend on the sample size.  According to statistical theory, we need a sample size of about 50 or more people in each of the groups for the difference to be statistically significant at the 95% level of confidence.  If we meet the sample size requirement, then the difference of 20 points will be statistically significant at the 95% level of confidence.

What does this really mean?  Many marketers will look at this result and conclude that since there is a 20-point difference and the difference is statistically significant, there must be a big difference between Groups A and B.  In reality, if we had done a census (i.e., surveyed the entire population) instead of surveying a sample, the difference between Group A and Group B may have turned out to be smaller.  In other words, what this result tells us is merely this:  given our particular sample size, there is a 5% chance that in the population represented by this sample, the proportions for Group A and Group B are not different.  That’s all! 

Statistical significance does not tell us anything about how big the difference is.  It only tells us the probability with which a difference found in the sample would not be found in the population.  Thus, for this case, statistical significance would allow us to conclude that there is only a 5% chance that in the population the proportion of Group A favoring the product is not higher than Group B; we are taking a 5% risk of concluding a difference exists when there may not be any such difference.  If this difference were significant at the 99% level of confidence, it would not have become larger.  It would only mean that there is a 1% chance that the difference observed in the sample would not be observed in the population.  Thus, we are only taking a 1% risk.
Practical Significance
From a marketing perspective, the statistically significant difference of 20 points may be meaningful or meaningless. It all depends on our research objectives and resources.  If it costs millions of dollars to reach each additional percentage of the market, we may decide to funnel resources toward Group A since it has a higher acceptance rate.  In this case, the difference may be termed a “big” difference because (a) we are reasonably sure (95% or 99% sure) that the difference observed in our sample also exists in the population and (b) each percentage of difference is worth millions of dollars to the client.  Thus, statistical significance should not be used to decide how big a difference is but merely to ascertain our confidence in generalizing the results from our sample to the population.

In another situation, the same difference may be ignored despite the fact that it may be statistically significant.  For instance, if the marketing costs are so low that it makes sense to market to both groups, we can ignore the difference (even though it is significant) and treat both groups as if they were the same.  We may choose to market to both groups as if they had similar acceptance rates (even though our statistical test was significant).

The logic is the following:  although we can be 95% sure that the difference observed exists in the population, given the marketing scenario, the difference is not meaningful.  Thus, the relevance of a statistically significant difference should be determined based on practical criteria including the absolute value of the difference, marketing objectives, strategy, and so forth.  The mere presence of a statistical significance does not necessarily imply that the difference is large or that it is of noteworthy importance.