When is a Sample Size Statistically Significant?

Defining The Term “Sample Size”

Sample size is a count of individual samples or observations in a statistical setting, such as a scientific experiment or a survey distributed to the general public.

When it comes to surveys in particular, sample size more precisely refers to the number of completed responses that a survey receives. 

Sample Populations vs. Target Populations

Samples, or sample populations, represent clusters of people reflecting a larger target population.

In most cases, surveying the entire target population of interest is simply not possible due to the large number of people that comprise it. 

By surveying samples of the population instead, researchers are able to make inferences about and uncover insights into the behaviors and opinions of the greater target population.

For example, a researcher might explore how cat owners feel about different kitty litter brands. Surveying the entire target population of millions of cat owners around the world is simply not feasible, as it would be incredibly time consuming and extremely expensive.

Instead, the researcher would select a sample population of cat owners that represents the larger target population, and ask them about their opinions on kitty litter. 

Hundreds or even thousands of cat owners could make up the sample size. Regardless of the sample size, this sample group must accurately represent the target population in order to yield valid survey responses.

The Fine Line of Determining Sample Size

If too few people make up the sample size, the resulting data will not represent the target population accurately. This means that results will be both inaccurate, and unable to inform decisions.

Conversely, if too many people make up the sample size, analyzing the data will drain the researcher’s time and budget. This especially applies to researchers using panel services or offering incentives to survey takers.

Every researcher that administers surveys must ask themselves the following question while developing their study:

“How many survey respondents do I need to reach in order to maximize my time and budget, as well as ensure that I’m able to take action on the most accurate results possible?” 

Using a Sample Size Calculator to Ensure Statistical Significance

Alchemer’s Sample Size Calculator can be used to determine how many individuals’ researchers need to survey in order to gather results that reflect the target population as precisely as desired.

For more information on the Sample Size Calculator, and how to input the necessary values such as confidence levels and confidence intervals, read our ever-informative documentation.

Questions to Ask While Determining Sample Size

In addition to using the Sample Size Calculator, ask yourself the following questions while determining sample size:

• What type of statistical analysis will I be conducting with this data? Will I want to compare subgroups? If so, a larger sample size is required.

• What is the probability of the event that I’m investigating occurring in this population? If no previous data exists, it’s best to use a 50 percent confidence level for a conservative estimate.

• How precise do I need my survey data to be? In other words, how much error can I tolerate? This affects your confidence interval, also known as your margin of error.

• How confident do I need to be that the true population value falls within my confidence interval?

• What is my budget for this study? Can I afford the sample that I really want or need?

• What is the target population size? Is it large? Small? Finite? If the population size is unknown, it’s best to assume that it’s very large.


---

There is no magic solution or formula that will enable you to determine the appropriate sample size for your study with complete and total confidence.

By leveraging the Sample Size Calculator and asking themselves insightful questions, researchers can have peace of mind that their survey results will be both statistically relevant and informative.