Secure Speaker Recognition System for Higher Education Applications
Keywords:
Speaker Verification, Voice Biometrics, Anti-Spoofing, Voice Cloning Detection, Higher EducationAbstract
Speaker verification has become useful in universities for remote oral exams, identity verification in e-learning platforms, and voice-based attendance. Modern speaker embedding makes verification feasible over variable-length speech, but actual deployment still faces two problems: device mismatch and noise, and spoofing attacks such as replay and voice cloning.
This paper presents a secure speaker verification system designed for university-like settings. The pipeline includes voice activity detection, feature extraction, speaker embedding, and point-based verification. To improve security, we add a random challenge to reduce the usefulness of replay attacks, a counterfeiting module to detect artificial or converted audio signals, encrypted embedding storage instead of raw audio, and lightweight session monitoring.
Our results show that the baseline performs well in clean conditions on the same device with EER = 2.6%, but deteriorates with short phrases where EER reaches 5.5%. When faced with attacks, restarting often succeeds in the baseline setup with ASR = 35%, but drops to 1-2% in the embedded system. Voice cloning is more difficult, with baseline ASR at 48%, dropping to 3-5% after combining challenge-response and anti-spoofing. This security gain comes at a small cost to usability, with response time increasing from 0.45 seconds to 0.78 seconds and the false response rate (FRR) increasing from around 4.1% to 5.0-5.6%.
References
References
(1) Snyder, D., Garcia-Romero, D., Sell, G., Povey, D., & Khudanpur, S. (2018). X-vectors: Robust DNN embeddings for speaker recognition. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).
(2) Raj, D., Snyder, D., Povey, D., & Khudanpur, S. (2019). Probing the information encoded in x-vectors. arXiv. https://doi.org/10.48550/arXiv.1909.06351
(3) Desplanques, B., Thienpondt, J., & Demuynck, K. (2020). ECAPA-TDNN: Emphasized channel attention, propagation and aggregation in TDNN-based speaker verification. arXiv. https://doi.org/10.48550/arXiv.2005.07143
(4) Li, M., Ahmadiadli, Y., & Zhang, X. (2024). Audio anti-spoofing detection: A survey. arXiv. https://doi.org/10.48550/arXiv.2404.13914
(5) Buker, A., Kurnaz, O., Bekiryazıcı, Ş., Demirtaş, S. C., & Hanilci, C. (2025). Evaluating parameter sharing for spoofing-aware speaker verification: A case study on the ASVspoof 5 dataset. In Proceedings of Interspeech 2025. https://doi.org/10.21437/Interspeech.2025-2618
(6) Jia, Y., Zhang, Y., Weiss, R. J., Wang, Q., Shen, J., Ren, F., Chen, Z., Nguyen, P., Pang, R., Lopez Moreno, I., & Wu, Y. (2018). Transfer learning from speaker verification to multispeaker text-to-speech synthesis. arXiv. https://arxiv.org/abs/1806.04558
(7) Wang, Y., Skerry-Ryan, R. J., Stanton, D., Wu, Y., Weiss, R. J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., Bengio, S., Le, Q., Agiomyrgiannakis, Y., Clark, R., & Saurous, R. A. (2017). Tacotron: Towards end-to-end speech synthesis. arXiv. https://arxiv.org/abs/1703.10135
(8) van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016). WaveNet: A generative model for raw audio. arXiv. https://arxiv.org/abs/1609.03499
