Real-time Yoga recognition using deep learning

Santosh Kumar Yadav1 bull; Amitojdeep Singh2 bull; Abhishek Gupta2 bull; Jagdish Lal Raheja1

An approach to accurately recognize various Yoga asanas using deep learning algorithms has been presented in this work. A dataset of six Yoga asanas (i.e. Bhujangasana, Padmasana, Shavasana, Tadasana, Trikonasana, and Vrikshasana) has been created using 15 individuals (ten males and five females) with a normal RGB webcam and is made publicly available. A hybrid deep learning model is proposed using convolutional neural network (CNN) and long short-term memory(LSTM) for Yoga recognition on real-time videos, where CNN layer is used to extract features from keypoints of each frame obtained from OpenPose and is followed by LSTM to give temporal predictions. To the best of our knowledge, this is the first study using an end-to-end deep learning pipeline to detect Yoga from videos. The system achieves a test accuracy of 99.04% on single frames and 99.38% accuracy after polling of predictions on 45 frames of the videos. Using a model with temporal data leverages the information from previous frames to give an accurate and robust result. We have also tested the system in real time for a different set of 12 persons (five males and seven females) and achieved 98.92% accuracy. Experimental results provide a qualitative assessment of the method as well as a comparison to the state-of-theart.

Human activity recognition is a well-established computer vision problem that has imposed several challenges over the years [1]. It is the problem of locating keypoints and the posture of a human body from the sensor data. Activity recognition is useful in many domains including biometrics, video-surveillance, human–computer interaction, assisted living, sports arbitration, in-home health monitoring, etc. [2–4]. The health status of an individual can be evaluated and predicted with the help of monitoring and recognizing their activities [5]. Yoga posture recognition is a relatively newer application.

Yoga is an ancient science that originated in India. According to the Bhagavad Gita, it is the remover of misery and destroyer of pain. Recently, Yoga is getting popular across the globe due to its physical, mental, and spiritual benefits. In 2014, the General Assembly of United Nations has declared 21st June as the lsquo;International Day of Yogarsquo; [6]. Over the last decade, Yoga is getting increasing importance in the medical research community, and numerous literature has been proposed for various medical applications including cardiac rehabilitation [6], positive body image intervention [7, 8], mental illnesses [9], etc. Without the use of medicines, Yoga can completely cure many diseases [10]. Yoga exercises boost physical health as well as help to cleanse the body, mind, and soul [11]. It comprises of many asanas and each of them denotes the static physical postures [12]. Yoga learning and self-instruction systems have the ability to popularize and spread Yoga while ensuring that it is practiced correctly [13, 14].

Computer-assisted self-training systems for sports and exercises can improve the performance of participants and prevent injuries [15]. Many work in the literature have proposed automated and semi-automated systems for analysing the sports and exercise activities such as soccer player ranking [16], swimming [17], tennis strokes [18] badminton [19], rugby [20], basketball [21, 22], vertical high jump [23], hurdles racing [24], etc.

To detect the difference in postures between a practitioner and an expert, Patil et al. [10] have proposed a lsquo;Yoga Tutorrsquo; project using speeded up robust features (SURF). However, to compare and describe the postures approximately by using only the contour information is not enough. Luo et al. [25] have proposed Yoga training system with an interface suit comprising 16 inertial measurement units (IMUs) and six tactors, which is obtrusive to the user and can affect the user to perform asana in a natural manner. Wu et al. [26] proposed an image and textbased expert system for Yoga; however, they have not analysed the practitionerrsquo;s posture.

Chen et al. [11] introduced Yoga activity recognition using features-based approach to design a self-training system. It uses a Kinect for extracting userrsquo;s body contour and capturing the body map. A star skeleton was used for rapid skeletonization to obtain a descriptor for the human pose. The work in [12] provides computer-assisted selftraining system for posture rectification using Kinect. It has taken three postures in consideration, i.e. tree, warrior III, and downward facing dog. However, the overall accuracy is very low at only 82.84%. In [27], a Yoga detection system is proposed for six asanas using Kinect and Adaboost classification with 94.78% accuracy score. However, they are using depth sensor-based camera which generally may not be available to the users.

Mohanty et al. [28] have applied image recognition techniques for Indian classical dance and Yoga postures identification from images using convolutional neural network (CNN) and stacked autoencoder (SAE) algorithms. However, they have evaluated their performance on still images only and not on videos. Chen et al. [15] proposed a Yoga self-training system to assist in rectifying postures while performing Yoga using a Kinect depth camera for 12 different asanas. However, it is using manual feature extraction and making separate models for each asana.

Delegate features, like a human skeleton, are compulsory to extract for describing the human postures. There are various skeletonization techniques in the literature, such as thinning and distance transformation. However, these approaches have a high computational cost and are sensitive to noise [11].

The conventional skeletonization approach has been replaced by deep learning-based methods since the advent of DeepPose by Tos

剩余内容已隐藏，支付完成后下载完整资料

英语译文共 5 页，剩余内容已隐藏，支付完成后下载完整资料

资料编号：[258087]，资料为PDF文档或Word文档，PDF文档可免费转换为Word