The 0.80 kappa threshold on the calibration set should be adjusted in light of the business's tolerance for false rejections and false releases, but once chosen, it should be verified on the holdout set rather than merely achieved on the calibration set.