What are the standard techniques to set cut-off values for analyzing knowledge, attitude, and practice? With scientific references

I am a scientist, that is my reference point.

The standard techniques are to use the reliable Statistics 101 tools.

Look up the statistical t_test, and how to determine a confidence interval of a particular confidence level (95%, 99%, 99.9%). Typically in studies, depending on the discipline, we want a particular level of confidence that our inference is correct; e.g. 95% for a business decision, 99% or 99.9% for a medical study, etc.

Ultimately this depends on the variance of whatever we are measuring, it is used in a formula along with the t-score (the value in the Normal Probability table) that corresponds to your desired level of confidence.

The formula gives you a specific “N” number of samples/measurements you must collect for the average of those measurements to have the desired confidence interval.

Now in this formula, you need to have some knowledge of what the original average and variance are, to detect a change in the original average and variance.

If you don’t have that, and are generating original data, then reworking the formula can tell you, for a running average and running variance (updated with each new sample), what the confidence interval is for a sample so far. You use either the Standard Normal table (if you have more than about 30 samples) or the Student’s T-Table (if you have 30 or less samples) to get the critical values, and then you can compute with 95% or 99% or whatever confidence interval you like that the true mean is within (sampleMean - value, sampleMean + value). You keep collecting values until you have the desired confidence interval.

In this case, when we don’t have a historical comparison, this is how we determine if the sample mean is acceptable or not; if all values in this range are meaningful, it is, if some of the values in this range would mean catastrophic failure, we have a problem.

(Of course in some things like medicine even death is an acceptable outcome if it is rare enough; we must weigh the most likely number of lives saved and not everything can be perfectly “safe”.)

This is how we determine if a drug is likely to be safe, first, and if it is safe, then to determine if it is effective.

Of course the change could be an advertising campaign, we want to determine if a new ad is more effective than an older ad, so instead of spending millions just rolling out the new ad, we compute the minimum number of impressions we must roll out to be 95% certain the average response of the new ad is greater than the average response of the older ad.

We can do the same thing for investment strategies, or electronic circuit function. This is how we compute the likely lifespan of chips. We can use the same tools for just about anything.