Sampling: When Can a Small Part Tell Us About the Whole?

Sampling uses observations from a part to answer a question about a whole. The decisive issue is how that part was chosen.

Imagine a jar holding several kinds of beans. Taking a large handful only from the top may miss beans that have settled below. Mixing the jar and selecting according to a clear rule gives the sample a better chance of reflecting its contents. Even random selection cannot promise that any one sample will match the whole exactly; size and ordinary variation still matter.

Sampling differs from giving an example. One example can show that something occurs, but usually cannot tell us how common it is. Sampling is also a selection process, whereas representativeness describes how well the selected part corresponds to the population of interest. Whether we discuss surveys, model training or personal experience, we should first ask what the whole is and who had a chance to be included. That sets the limits of what a partial observation can justify.

Source: https://www.itl.nist.gov/div898/handbook/ppc/section1/ppc134.htm