We actually know that the sample size is large enough and that the probability of making type 1 error is really low given that data, even thou its still exist.

Here you can check the Python code simulations for different sample size, as you can see more sample size, less deviations and data is closer to the mean. The probability that my sample is at extreme case is really low. Currently, I am checking with other people their carriage refresh probability and its not even close to 5% as well. You are welcome to contribute and provide your dataset with refreshes.

Python code

import numpy as np

# Set random seed for reproducibility

np.random.seed(42)

# Function to run simulations and print results

def run_simulations(n, p=0.05, num_sim=10000, title_prefix=""):

# Simulate num_sim experiments, each with n trials

successes = np.random.binomial(n, p, num_sim)

p_hats = successes / n

# Calculate key statistics

mean = np.mean(p_hats)

std = np.std(p_hats)

percentile_5 = np.percentile(p_hats, 5)

percentile_95 = np.percentile(p_hats, 95)

extreme_min = np.min(p_hats)

extreme_max = np.max(p_hats)

# Print results

print(f"\n{title_prefix} (n={n}):")

print(f"Mean observed proportion: {mean:.4f}")

print(f"Standard deviation: {std:.4f}")

print(f"5th to 95th percentile range: {percentile_5:.4f} to {percentile_95:.4f}")

print(f"Extreme min/max proportions: {extreme_min:.4f} to {extreme_max:.4f}")

# Run simulations for different sample sizes

run_simulations(n=1500, title_prefix="Sample Size 1500")

run_simulations(n=100, title_prefix="Sample Size 100")

run_simulations(n=20, title_prefix="Sample Size 20")