Event-Wise Validation of Machine Learning for Magnitude-Threshold Classification Using the Turkish Strong-Motion Database (SMD-TR)

Featured Application: The workflow supports retrospective audit and calibration of magnitude-threshold classifiers in strong-motion archives. It is not an operational earthquake early-warning implementation because the predictors are full-record peak parameters. Magnitude-threshold classifiers trained on multiple station records from the same earthquake are vulnerable to event-level data leakage, and comparisons across magnitude boundaries can be misleading because the class definition and prevalence change. We evaluated ten machine-learning classifiers using the official SMD-TR Metadata.csv and IM_RotD50.csv files. Record metadata and orientation-independent horizontal RotD50 intensity measures were joined one-to-one by a waveform identifier (WFID), and earthquakes were grouped by the official earthquake identifier (EQID). The Mw-only cohort contained 35,198 records from 3377 events and 965 stations. Events, rather than records, were assigned to an 80% development subset and a locked 20% independent test subset. Hyperparameters, classifier, and decision-score cutoff were selected within five event-wise development folds at the prespecified Mw = 5.5 boundary using event-level F2. Linear Discriminant Analysis (LDA; lsqr solver with automatic shrinkage) was selected in development (mean F2 = 0.903, SD = 0.043; score cutoff = 0.703). On 676 independent test events, including 23 positives, LDA achieved F2 = 0.779 (95% event-bootstrap interval 0.625–0.896), recall = 0.826, precision = 0.633, balanced accuracy = 0.905, MCC = 0.712, and PR-AUC = 0.865 (TN = 642, FP = 11, FN = 4, TP = 19). An Extra Trees regression sensitivity analysis yielded event-level MAE = 0.171, RMSE = 0.229, and R2 = 0.860. Boundaries from Mw 5.0 to 6.0 are reported as separate descriptive tasks, not as evidence for an optimal physical threshold. Because the predictors are full-record RotD50 peak parameters, the findings support offline calibration and audit rather than operational real-time early warning.