The MINUIT package provides two ways to compute uncertainty intervals for a fitted parameter, the HESSE method and the MINOS method. Ideally, these intervals should have 68 % coverage probability: if the experiment is repeated with new random data each time, the intervals should cover the true value in 68 % of the cases.
This only asymptotically correct, and if the samples are small, the actual coverage probability can deviate from the expected one. It is often thought that MINOS provides better coverage (closer to the expected value) because its intervals can be asymmetric and seem to retain more information.
As a toy example, I fit the sum of two normal distribution which have equal width but different location parameters. The fitted parameters are the two locations and the shared width. Here is one example fit.
| Migrad | |
|---|---|
| FCN = 3.065e+04 | Nfcn = 31 |
| EDM = 9.53e-06 (Goal: 0.0002) | time = 4.5 sec |
| Valid Minimum | Below EDM threshold (goal x 10) |
| No parameters at limit | Below call limit |
| Hesse ok | Covariance accurate |
| Name | Value | Hesse Error | Minos Error- | Minos Error+ | Limit- | Limit+ | Fixed | |
|---|---|---|---|---|---|---|---|---|
| 0 | mu_a | -0.014 | 0.020 | |||||
| 1 | mu_b | 1.013 | 0.020 |
| mu_a | mu_b | |
|---|---|---|
| mu_a | 0.000412 | -0.2e-3 (-0.386) |
| mu_b | -0.2e-3 (-0.386) | 0.00041 |
I now run this fit many times on newly drawn random samples. To speed things up, the calculation is done in parallel.
10 hesse 0.76 minos 0.75 <corr> -0.388 n_fail=7
20 hesse 0.77 minos 0.73 <corr> -0.450 n_fail=0
50 hesse 0.75 minos 0.67 <corr> -0.458 n_fail=0
100 hesse 0.77 minos 0.69 <corr> -0.461 n_fail=0
70 hesse 0.75 minos 0.67 <corr> -0.472 n_fail=0
200 hesse 0.69 minos 0.66 <corr> -0.435 n_fail=0
1000 hesse 0.67 minos 0.67 <corr> -0.407 n_fail=0
2000 hesse 0.69 minos 0.69 <corr> -0.401 n_fail=0
500 hesse 0.70 minos 0.69 <corr> -0.413 n_fail=0
3000 hesse 0.69 minos 0.69 <corr> -0.404 n_fail=0
400 hesse 0.70 minos 0.69 <corr> -0.422 n_fail=0
300 hesse 0.68 minos 0.67 <corr> -0.420 n_fail=0I plot the actual coverage probability of the HESSE and MINOS intervals. For small samples, both methods overcover, but the MINOS intervals converge faster to the correct value. So in this case, MINOS did perform better. This is not always the case, however.
