Web-Based Application for Tongue Shape and Color Detection Using Artificial Intelligence Techniques: Preliminary Results

Ali Al-Naji; Javaan Chahl

doi:10.51173/eetj.v2i2.31

Authors

Ali Al-Naji School of Engineering, University of South Australia, Mawson Lakes, SA 5095, Australia
Javaan Chahl School of Engineering, University of South Australia, Mawson Lakes, SA 5095, Australia

DOI:

https://doi.org/10.51173/eetj.v2i2.31

Keywords:

Tongue diagnosis, Artificial intelligence, Computer vision systems, YOLO deep learning, CatBoost Machine learning

Abstract

Tongue diagnosis is an important method in both Traditional Chinese Medicine (TCM) and Western Medicine (WM), as the tongue's appearance can reflect a person’s overall health. Among the key features observed, tongue shape and color play a major role in identifying certain diseases and tracking their progression. This study focuses on the tongue image analysis method of artificial intelligence (AI) to detect shapes and colors of tongue for fast health screening without any need for human intervention. The proposed system firstly used the You Only Look Once version 10 model (YLOVv10) a deep learning object detection system on 750 tongue images in four tasks. The first task used the YOLOv10 model to detect and isolate the entire tongue region from the input image to ensure that the following tasks focus only on the tongue region. The second task was to accurately classify the tongue into seven shape categories, including normal, geographic, fissured, scalloped, thin, swollen, and deviated tongues. Thirdly the system detected crack types associated with fissured tongue, including side cracks, vertical cracks, deep cracks and irregular cracks. Lastly, the system detected whether the tongue contains ulcers or spots or not. The study also used the machine learning CatBoost model to train 5550 color images captured at different color saturations and under different light conditions and classified into seven classes (red, yellow, green, blue, gray, white, and pink) using several color space models, including (RGB, YcbCr, HSV, LAB, and YIQ) as input features to analyze and extract tongue color. The WebApp was developed using Streamlit to offer an easy-to-use graphical interface and provides an automatic tongue shape and color detection tool and compares results based on both TCM and WM perspectives, thus supporting early screening and medical analysis in a fast and reliable way (https://ai-linguasense-version2025.streamlit.app/).

References

C. Jin, D. Zhang, X. Cao, Z. Zhang, C. Xue, and Y. Zhang, "Lightweight YOLOv8 for tongue teeth marks and fissures detection based on C2f_DCNv3," Scientific Reports, vol. 15, no. 1, p. 1560, 2025. https://www.nature.com/articles/s41598-025-86001-2

S. Balasubramaniyan, V. Jeyakumar, and D. S. Nachimuthu, "Panoramic tongue imaging and deep convolutional machine learning model for diabetes diagnosis in humans," Scientific Reports, vol. 12, no. 1, p. 186, 2022. https://www.nature.com/articles/s41598-021-03879-4

Y. Tang, Y. Sun, J. Y. Chiang, and X. Li, "Research on multiple-instance learning for tongue coating classification," IEEE Access, vol. 9, pp. 66361-66370, 2021. https://ieeexplore.ieee.org/abstract/document/9420449

M. Akrami, A. Jafari, and S. Pedrammehr, Disease Diagnosis and Disease Patterns by Visual Inspection of the Tongue. Cambridge Scholars Publishing, 2024.

Y. Liu, K. Leng, H. Ren, X. Liu, S. Wang, and N. Luo, "Tongue Manifestation Feature Classification and Detection for Chronic Gastritis Based on the YOLO Framework," in 2024 International Annual Conference on Complex Systems and Intelligent Science (CSIS-IAC), 2024: IEEE, pp. 448-453. https://ieeexplore.ieee.org/abstract/document/10919443

T. Jiang et al., "Tongue image quality assessment based on a deep convolutional neural network," BMC Medical Informatics and Decision Making, vol. 21, no. 1, p. 147, 2021. https://link.springer.com/article/10.1186/s12911-021-01508-8

A. K. Abdullah, S. L. Mohammed, and A. Al-Naji, "Tongue Color Analysis and Disease Diagnosis Based on a Computer Vision System," in 2022 4th International Conference on Advanced Science and Engineering (ICOASE), 2022: IEEE, pp. 25-30. https://ieeexplore.ieee.org/abstract/document/10075574

A. K. Abdullah, S. L. Mohammed, A. Al-Naji, and M. S. Alsabah, "Tongue color analysis and diseases detection based on a computer vision system," Journal of Techniques, vol. 5, no. 1, pp. 22-37, 2023. https://journal.mtu.edu.iq/index.php/MTU/article/view/868

A. R. Hassoon, A. Al-Naji, G. A. Khalid, and J. Chahl, "Tongue disease prediction based on machine learning algorithms," Technologies, vol. 12, no. 7, p. 97, 2024. https://www.mdpi.com/2227-7080/12/7/97

Z. Yang, Y. Zhao, J. Yu, X. Mao, H. Xu, and L. Huang, "An intelligent tongue diagnosis system via deep learning on the android platform," Diagnostics, vol. 12, no. 10, p. 2451, 2022. https://www.mdpi.com/2075-4418/12/10/2451

J. Okawa et al., "Developing tongue coating status assessment using image recognition with deep learning," Journal of Prosthodontic Research, vol. 68, no. 3, pp. 425-431, 2023. https://www.jstage.jst.go.jp/article/jpr/68/3/68_JPR_D_23_00117/_article/-char/ja/

G. Kang, Y. Hao, Y. Wang, G. Cao, Z. Ma, and C. Xia, "A two-stage approach for mobile-acquired tongue image with YOLOv5 and LA-UNet," in 2024 IEEE 4th International Conference on Electronic Technology, Communication and Information (ICETCI), 2024: IEEE, pp. 454-460. https://ieeexplore.ieee.org/abstract/document/10594709

T. Chen, Y. Chen, Z. Zhou, Y. Zhu, L. He, and J. Zhang, "Deep learning-based automated tongue analysis system for assisted Chinese medicine diagnosis," Frontiers in Physiology, vol. 16, p. 1559389, 2025. https://www.frontiersin.org/journals/physiology/articles/10.3389/fphys.2025.1559389/full

A. R. Hassoon, G. A. Khalid, A. H. Al Shafeay, and A. Al-Naji, "Artificial intelligence techniques for tongue color detection," in AIP Conference Proceedings, 2024, vol. 3232, no. 1: AIP Publishing LLC, p. 040017. https://doi.org/10.1063/5.0236190

A. Wang, H. Chen, L. Liu, K. Chen, Z. Lin, and J. Han, "Yolov10: Real-time end-to-end object detection," Advances in Neural Information Processing Systems, vol. 37, pp. 107984-108011, 2024. https://dl.acm.org/doi/abs/10.5555/3737916.3741345

C.-Y. Wang and H.-Y. M. Liao, "YOLOv1 to YOLOv10: The fastest and most accurate real-time object detection systems," APSIPA Transactions on Signal and Information Processing, vol. 13, no. 1, 2024. http://dx.doi.org/10.1561/116.20240058

L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, and A. Gulin, "CatBoost: unbiased boosting with categorical features," Advances in neural information processing systems, vol. 31, 2018. https://doi.org/10.48550/arXiv.1706.09516

J. T. Hancock and T. M. Khoshgoftaar, "CatBoost for big data: an interdisciplinary review," Journal of big data, vol. 7, no. 1, p. 94, 2020. https://link.springer.com/article/10.1186/s40537-020-00369-8

M. Khorasani, M. Abdou, and J. H. Fernández, "Web application development with streamlit," Software Development, vol. 498, p. 507, 2022. https://link.springer.com/book/10.1007/978-1-4842-8111-6

M. Khorasani, M. Abdou, and J. Hernández Fernández, "Streamlit use cases," in Web Application Development with Streamlit: Develop and Deploy Secure and Scalable Web Applications to the Cloud Using a Pure Python Framework: Springer, 2022, pp. 309-361. https://link.springer.com/chapter/10.1007/978-1-4842-8111-6_11