Publication type: | Article in scientific journal |
Type of review: | Peer review (publication) |
Title: | Mathematical expression recognition using a new deep neural model |
Authors: | Mirkazemy, Abolfazl Adibi, Peyman Ehsani, Seyed Mohhamad Saied Darvishy, Alireza Hutter, Hans-Peter |
et. al: | No |
DOI: | 10.1016/j.neunet.2023.08.045 |
Published in: | Neural Networks |
Volume(Issue): | 167 |
Page(s): | 865 |
Pages to: | 874 |
Issue Date: | 2023 |
Publisher / Ed. Institution: | Elsevier |
ISSN: | 0893-6080 1879-2782 |
Language: | English |
Subjects: | Mathematical expression recognition; Deep learning; Encoder–decoder architecture; Attention; Scientific documents accessibility |
Subject (DDC): | 006: Special computer methods 510: Mathematics |
Abstract: | In this paper, we propose a novel deep neural model for Mathematical Expression Recognition (MER). The proposed model uses encoder–decoder transformer architecture that is supported by additional pre/post-processing modules, to recognize the image of mathematical formula and convert it to a well-formed language. A novel pre-processing module based on domain prior knowledge is proposed to generate random pads around the formula’s image to create more efficient feature maps and keeps all the encoder neurons active during the training process. Also, a new post-processing module is developed which uses a sliding window to extract additional position-based information from the feature map, that is proved to be useful in the recognition process. The recurrent decoder module uses the combination of feature maps and the additional position-based information, which takes advantage of a soft attention mechanism, to extract the formula context into the LaTeX well-formed language. Finally, a novel Reinforcement Learning (RL) module processes the decoder output and tunes its results by sending proper feedbacks to the previous steps. The experimental results on im2latex-100k benchmark dataset indicate that each devised pre/post-processing as well as the RL refinement module has a positive effect on the performance of the proposed model. The results also demonstrate the higher accuracy of the proposed model compared to the state-of-the-art methods. |
URI: | https://digitalcollection.zhaw.ch/handle/11475/28766 |
Fulltext version: | Published version |
License (according to publishing contract): | Licence according to publishing contract |
Departement: | School of Engineering |
Organisational Unit: | Institute of Computer Science (InIT) |
Appears in collections: | Publikationen School of Engineering |
Files in This Item:
There are no files associated with this item.
Show full item record
Mirkazemy, A., Adibi, P., Ehsani, S. M. S., Darvishy, A., & Hutter, H.-P. (2023). Mathematical expression recognition using a new deep neural model. Neural Networks, 167, 865–874. https://doi.org/10.1016/j.neunet.2023.08.045
Mirkazemy, A. et al. (2023) ‘Mathematical expression recognition using a new deep neural model’, Neural Networks, 167, pp. 865–874. Available at: https://doi.org/10.1016/j.neunet.2023.08.045.
A. Mirkazemy, P. Adibi, S. M. S. Ehsani, A. Darvishy, and H.-P. Hutter, “Mathematical expression recognition using a new deep neural model,” Neural Networks, vol. 167, pp. 865–874, 2023, doi: 10.1016/j.neunet.2023.08.045.
MIRKAZEMY, Abolfazl, Peyman ADIBI, Seyed Mohhamad Saied EHSANI, Alireza DARVISHY und Hans-Peter HUTTER, 2023. Mathematical expression recognition using a new deep neural model. Neural Networks. 2023. Bd. 167, S. 865–874. DOI 10.1016/j.neunet.2023.08.045
Mirkazemy, Abolfazl, Peyman Adibi, Seyed Mohhamad Saied Ehsani, Alireza Darvishy, and Hans-Peter Hutter. 2023. “Mathematical Expression Recognition Using a New Deep Neural Model.” Neural Networks 167: 865–74. https://doi.org/10.1016/j.neunet.2023.08.045.
Mirkazemy, Abolfazl, et al. “Mathematical Expression Recognition Using a New Deep Neural Model.” Neural Networks, vol. 167, 2023, pp. 865–74, https://doi.org/10.1016/j.neunet.2023.08.045.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.