Gated Transformer with Dual Attention for Rumour Category Detection on Social Platforms
DOI:
https://doi.org/10.32628/IJSRST2513104Keywords:
Transformer, Rumour Detection, Dual Attention, Gated Recurrent Unit (GRU), Social Media, Text Classification, Interpretability, Fake News,, Discourse AnalysisAbstract
The spread of misinformation and unverified content on social media has become a major concern in the digital age. Accurate categorization of rumours—into support, denial, query, or comment—is essential to assess the credibility and impact of online discourse. In this work, we propose a simplified yet effective deep learning framework that combines a Transformer-based encoder with a dual attention mechanism. The model captures fine-grained word-level semantics and post-level contextual relevance within conversation threads. By integrating gated recurrent units (GRUs) with word and post-level attention, the framework enhances the ability to distinguish rumour types based on linguistic and discourse cues. Experimental evaluations on two benchmark datasets, PHEME and RumourEval-19, demonstrate strong performance in terms of accuracy and F1-score. Furthermore, attention visualizations provide interpretability, making the model’s predictions more transparent and trustworthy.
Downloads
References
Zubiaga, A., Liakata, M., & Procter, R. (2018). Detection and resolution of rumours in social media: A survey. ACM Computing Surveys, 51(2), 1–36.
Shu, K., Sliva, A., Wang, S., Tang, J., & Liu, H. (2017). Fake news detection on social media: A data mining perspective. ACM SIGKDD Explorations, 19(1), 22–36.
Zubiaga, A., Aker, A., Bontcheva, K., Liakata, M., & Procter, R. (2018). Detection and resolution of rumours in social media: A survey. Information Processing & Management, 54(2), 146–160.
Qazvinian, V., Rosengren, E., Radev, D., & Mei, Q. (2011). Rumor has it: Identifying misinformation in microblogs. Proceedings of EMNLP, 1589–1599.
Ma, J., Gao, W., & Wong, K. (2017). Detect rumors in microblog posts using propagation structure via kernel learning. ACL, 708–717.
Ma, J., Gao, W., & Wong, K. (2016). Detecting rumor threads in microblogging networks. AAAI, 101–110.
Liu, X., Nourbakhsh, A., Li, Q., Fang, R., & Shah, S. (2015). Real-time rumor debunking on Twitter. CIKM, 1867–1870.
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016). Hierarchical attention networks for document classification. NAACL-HLT, 1480–1489.
Zhou, X., & Zafarani, R. (2020). A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Computing Surveys, 53(5), 1–40.
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv:1301.3781.
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. NAACL-HLT, 4171–4186.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., … & Stoyanov, V. (2019). RoBERTa: A robustly optimized BERT pretraining approach. arXiv:1907.11692.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … & Polosukhin, I. (2017). Attention is all you need. NeurIPS, 5998–6008.
Lin, Z., Feng, M., Santos, C. N. D., Yu, M., Xiang, B., Zhou, B., & Bengio, Y. (2017). A structured self-attentive sentence embedding. ICLR.
Kumar, S., Asthana, R., Upadhyay, S., Akhtar, M. S., & Ekbal, A. (2020). Synergistic multiple attention network for rumour verification on social media. Information Processing & Management, 57(2), 102–128.
Wu, L., Rao, Y., Yang, Y., & Yu, H. (2015). False rumor detection on Sina Weibo by propagation structures. ICDE, 651–662.
Khattar, D., Goud, J. S., Gupta, M., & Varma, V. (2019). Multimodal deep learning for fake news detection. WWW, 2915–2921.
Zhang, Q., Zhang, J., Dong, Y., & Philip, S. Y. (2019). Fake news detection with deep diffusive network model. IJCAI, 3812–3818.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.
Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder–decoder for statistical machine translation. EMNLP, 1724–1734.
Ma, J., Gao, W., Mitra, P., Kwon, S., Jansen, B. J., Wong, K. F., & Cha, M. (2016). Detecting rumors from microblogs with recurrent neural networks. IJCAI, 3818–3824.
Bian, T., Xiao, X., Xu, T., Zhao, P., Huang, W., Rong, Y., … & Huang, J. (2020). Rumor detection on social media with bi-directional graph convolutional networks. AAAI, 549–556.
Yuan, C., Ma, Q., Zhou, W., Han, J., Hu, S., & Hu, X. (2019). Jointly embedding the local and global relations of heterogeneous graph for rumor detection. ICDM, 796–805.
Gorrell, G., Kochkina, E., Liakata, M., Aker, A., Zubiaga, A., & Bontcheva, K. (2019). SemEval-2019 task 7: RumourEval. SemEval, 845–854.
Zubiaga, A., Liakata, M., Procter, R., Hoi, G. W. S., & Tolmie, P. (2016). Analysing how people orient to and spread rumours in social media by looking at conversational threads. PLoS ONE, 11(3), e0150989.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 International Journal of Scientific Research in Science and Technology
This work is licensed under a Creative Commons Attribution 4.0 International License.
https://creativecommons.org/licenses/by/4.0
