Below are the Bonferroni tables for p?s from the new Subset/Method 2- ways for Auq per fold and length interval. Since there are 6 folds and 12 length intervals, the number of Bonferroni entries is 72 per method comparison (1:R1, 2:R2, or 3:R3.)
It?s clear that method 2 does better by far than method 1 or 3. (From the CI plots of R?uq that we were just discussing, it was to be expected that either method 1 or 2 or both would be ?good?, but certainly not method 3; hence, the Bonferroni tables below are not inconsistent with the CI plots previously exhibited.)
It?s also clear from the 2:R2 table that folds a1, b1, c1, c2 all exhibit reliable behavior of Auq in regression Ruq at 2-5 different length intervals each:
a1: 10,11 b1: 6,7 c1: 4,7,9,10,12 c2: 8,10,11,12
To put this point another way, we can use the residuals of Ruq for these folds and length intervals to construct logistic regression predictors with LESS worry that we are merely buidling dream-castles on top of statistically unjustified sand-castles.
On the other hand, the following tables also make it clear folds b47 and a3 exhibit no reliable behavior of Auq in regression Ruq for any length interval.
But - following up on a suggestion you raised some time ago, I want to recompute the Bonferroni tables with just (a1,a3) entries grouped together, just (b1,b47) entries grouped together, and jsut (c1,c2) entries grouped together. Not only will this reduced the number of Bonferroni entries per table from 72 to 24, but will also indicate the extent to which: i) our two alpha folds (a1,a3) are like each other); ii) our two beta folds (b1,b47) are like each other; iii) our two alpha-beta folds (c1,c2) are like each other. I will post these nine new Bonferroni tables (3 pairs of folds times three method interactions) in my next post.
Thanks again as always for your continued consideration of these matters. I hope that you agree with what I?ve said above ...
Bonferroni Tables for p's from Auq Subset/Method 2-way Interactions at Each Fold and Each Length Interval