Result is NOT valid
NOT INDEPENDENT
- 100 % of the test records are the same records the models were trained on. These scores are not an independent test. Use data from another area, event or year.
Results
| Model | ROC-AUC [95 % CI] | PR-AUC [95 % CI] | TSS | Balanced acc. | Kappa | Brier | ECE | CV ROC-AUC |
|---|---|---|---|---|---|---|---|---|
| XGBoost | 0.997 [0.996–0.998] | 0.999 [0.999–1.000] | 0.954 | 0.977 | 0.885 | 0.026 | 0.043 | 0.956 |
| RF | 0.990 [0.988–0.992] | 0.998 [0.997–0.998] | 0.895 | 0.947 | 0.818 | 0.041 | 0.062 | 0.956 |
Confidence intervals: cluster bootstrap over individual records (no group or block size given; intervals may be too narrow if records are spatially clustered). No model was refitted and the threshold was not tuned on these data.
Notes
- Bootstrap resamples individual records: confidence intervals are too narrow when test records are spatially clustered. Provide a block size or a group column.
- NOT INDEPENDENT: 100 % of the test records are identical to training records of XGBoost (same coordinates). The scores below measure how well the model remembers its training data, not how it generalises. Do not report them. Build the test table from another area, event or year.
- NOT INDEPENDENT: 100 % of the test records are identical to training records of RF (same coordinates). The scores below measure how well the model remembers its training data, not how it generalises. Do not report them. Build the test table from another area, event or year.