MMHFNet: Multi-modal and multi-layer hybrid fusion network for voice pathology detection

Automatic voice pathology detection using non-invasive techniques that utilize patients’ speech and electroglottograph (EGG) signals play a vital role in diagnosis and early medical intervention. In this paper, a novel deep Multi-Modal and Multi-Layer Hybrid Fusion Network (MMHFNet) is proposed to improve the performance of non-invasive voice pathology detection systems. MMHFNet simultaneously incorporates complementary information of different modalities (speech and EGG signals). It also vertically combines the low-level features, extracted from shallow layers, and high-level features, extracted from deep layers, to take the full advantage of spatio-spectral information of different layers for multi-layer fusion. The features extracted by MMHFNet are then fed into an LSTM classification network to diagnose the voice pathology. Comprehensive experiments are conducted on the publicly available Saarbruecken Voice Database (SVD) to evaluate the performance of the proposed MMHFNet. This dataset is used in two manners; one using its all samples and the other with selected samples to form the largest balanced SVD dataset. Experimental results demonstrated that the proposed MMHFNet achieves accuracy rates of 91% and 96.05% for datasets with all and balanced samples, respectively.