Browse by author
Lookup NU author(s): Dr Huizhi Liang
Full text for this publication is not currently held within this repository. Alternative links are provided below where available.
© 2023 Zhejiang University. All rights reserved.An emotion recognition system that integrates acoustic and articulatory feature conversions was proposed in order to investigate the influence of acoustic and articulatory conversions on Mandarin emotion recognition. Firstly, a multimodal emotional Mandarin database was recorded based on the human articulation mechanism. Then, a bi-directional mapping generative adversarial network (Bi-MGAN) was designed to solve the feature conversion problem with bimodality, and the generator loss functions and the mapping loss functions were proposed to optimise the network. Finally, a residual temporal convolutional network based on the feature-dimension attention (ResTCN-FDA) was constructed to use attention mechanisms to adaptively assign different weights to different variety features and different dimension channels. Experimental results show that the conversion accuracy of Bi-MGAN outperforms the current optimal algorithms for conversion network in both the forward and the reverse mapping tasks. The evaluation metrics of ResTCN-FDA on a given emotion dataset is much higher than traditional emotion recognition algorithms. The real features fused with the mapped features resulted in a significant increase in the accuracy of the emotions being recognized correctly, and the positive effect of mapping on Mandarin emotion recognition was demonstrated.
Author(s): Li H-F, Zhang X-Y, Duan S-F, Jia H-R, Liang H-Z
Publication type: Article
Publication status: Published
Journal: Zhejiang Daxue Xuebao (Gongxue Ban)/Journal of Zhejiang University (Engineering Science)
Year: 2023
Volume: 57
Issue: 9
Pages: 1865-1875
Print publication date: 01/09/2023
Online publication date: 16/10/2023
Acceptance date: 02/04/2023
ISSN (print): 1008-973X
Publisher: Zhejiang University
URL: https://doi.org/10.3785/j.issn.1008-973X.2023.09.018
DOI: 10.3785/j.issn.1008-973X.2023.09.018
Altmetrics provided by Altmetric
