Exercise Posture Assistance System

Date: 2022.06.16

Name: ChangMin An JiWoo Yang

Github: Link

Demo Video: Link

I. Introduction

In this LAB, we start a project that tells us how to correct our posture in real time when we exercise at the gym. This program is basically limited to the "lat-pull-down" movement. These days, as interest in health increases due to COVID-19, interest in health increases, and people who exercise alone also increase. However, if we exercise alone, it is difficult to recognize whether you are exercising in an accurate posture, and as a result, a problem that is prone to muscle imbalance is found. To solve this problem, we try to create system that identifies each joint of a person and measures the balance according to both slopes of the upper body joint to give feedback on the balance between the two forces. The tutorial is run by visual studio code(VS code), loading web cam or video source, and processing images in real time using OpenCV.

  • Demo

II. Requirement

Hardware

  • Logitech C922 pro Webcam

  • Lat Pull Down Machine

Software

  • Python 3.9.12

  • Tensorflow 2.9.1

  • numpy 1.21.5

  • OpenCV 4.5.5

  • MoveNet

III. Flow Chart

IV. Procedure

1. Setup

First, installation is carried out using Anaconda Prompt to build the environment. It is important to install something suitable for each version using anaconda to build it to enable image processing.

2. Installation

2-1. Install Anaconda

Anaconda : Python and libraries package installer.

Click here Download the installer on window to Windows 64-Bit Graphical Installer

Follow the following steps

  • Double click the installer to launch.

  • Select an install for "Just Me"(recommended)

  • Select a destination folder to install Anaconda and click the Next button.

  • Do NOT add Anaconda to my PATH environment variable

  • Check to register Anaconda as your default Python.

2-2. Install Python

Python 3.9

Python is already installed by installing Anaconda. But, we will make a virtual environment for a specific Python version.

  • Open Anaconda Prompt(admin mode)

  • First, update conda and pip (If the message "Press Y or N" appears, press Y and Enter key)

  • Then, Create virtual environment for Python 3.9, Name the $ENV as py39. If you are in base, enter conda activate py39

  • After installation, activate the newly created environment

2-3. Install Libs

Install Numpy, OpenCV, Jupyter (opencv-python MUST use 4.5.5 NOT 4.6.0)

2-4. Install Visual Studio Code

Follow: How to Install VS Code

Also, read about

2-5. Install TensorFlow

  • TensorFlow - DL library, developed by Google.

3. Download

  • Download MoveNet model: TFLite model link (Must download in local folder (include main.py))

4. library you need

5. Global Variable

5-1. Main Variable

Definition Body Parts: The output of the deep learning model we use is location information for 17 joints. The position of each joint has an order. The order is as follows.

[ nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, right ankle ]

Since we use the positions of the shoulders, elbows, and wrists on both sides in this application, we defined them as follows.

Definition of body edges: Each joint is tied together to draw a skeleton model.

Thresholding

  • CONFIDENCE_THRESHOLD: Acceptable confidence for each joint position

  • CORRECT_RECOGNIGION:

  • START_THRESHOLD:

  • START_COUNT_THRESH: Define the minimum number of frames for the correct posture

User input: This is a definition for setting the number of user's target exercise in the system.

Flag

  • system_Flag: It is a flag that controls the start of processing.

  • start_Flag: It is a flag that controls the start of processing for counting the number of workouts.

  • finish_Flag: It is a flag indicating that the exercise is over.

  • tk_Flag: It is a flag that determines whether or not to receive new input from the user.

  • up_down_Flag: It is a flag necessary to count the exercise.

For counting: Definitions for the number of good and bad exercises, stack calculations, average values, conditions for starting an exercise, and frame values.

For Balance: Definitions for the balance value, the minimum balance value, and the maximum balance value.

6. Definition Function

6-1. Processing

Get position of 17 each joint

  • Since the position of the joint obtained from the model is relative to the frame, the frame is entered as an input to the function, and the keypoint that has information about the joint points and the minimum confidence are input. As output, 'shape' with the positions of all joints for the frame and 'posebuf' with position information for joints exceeding the minimum confidence can be obtained.

Draw connecting

  • This is a function that can draw a skeleton model for a joint.

Start position

  • If the value obtained by subtracting the calculated slopes between the shoulder and wrist joints is within the allowable range for a certain frame, a text indicating that the user is in the correct posture is given and start_Flag is turned on (start_Flag = True). Otherwise, feedback text about the posture is given.

Calculate balance

  • The balance is calculated using the relative proportions of the positions of both wrists.

Count workout

  • Because there is a noise value, the positions of the elbows on both sides are accumulated for 10 frames each, and the flag is determined based on the average value.

  • The condition for counting is when one or both elbows go down below the shoulder and then rise again.

  • If both elbows go down and the balance value at that time is within the allowable range, the correct posture count is performed.

  • Otherwise, if one elbow goes down or both elbows go down but the balance value exceeds the allowable range, a bad posture count is counted.

  • Finally, when the correct posture count is equal to the target number of exercise, finish_Flag is turned on (finish_Flag = True).

Reset all parameter

  • Reset all parameter when system Flag is off(value = False)

6-2. Show Text

Show text

  • Show text for all count, good count, bad count, balance, feedback and finish

Show start text

  • The text output of the Start_Postion_Adjustment function is displayed on the image.

7. Main Code

Model Interpreter Definition

  • Load the model

  • The model file must be the same as the path where the corresponding code is located.

TKinter for user count

  • When starting for the first time, this is a code that allows you to input the number of target workouts.

Open the Video & Recording Video Configuration

  • Open the video we using and do ready for recording

Start the system by while()

  • Processing is performed for each frame of the video using a while().

  • First, when the user wants to receive input again (press 'r' key -> tk_Falg = True), a section to receive input again was placed at the beginning of while().

  • And use the function (cv.getTickCount()) to get the time to measure the FPS.

  • Get a frame from video.

Resize the frame, Setup input detail and output detail, and Input to model and get output

  • Resized to fix frame size to 1080x1080.

  • Since the input size of the deep learning model we use is 192x192, we change the frame to 192x192 to put it as an input.

    (Using resize_with_pad())

  • You need to know the information of the input tensor and the output tensor in order to transmit and receive data, so setup is done.

  • Input to model and get output

Main code

  • We bring about the joint information we want to use.

    (Wrist, Elbow, and Shoulder)

  • To adjust the flag to start the system, adjust the flag depending on whether the elbow is above or below the shoulder position.

    (flag off(system_Flag = False) if the elbow position is below the shoulder / flag off(system_Flag = True) if the elbow position is above the shoulder))

  • If the system flag is turned on (system_Flag == True), the finish flag is checked, and if it is off, the function to adjust the start flag is executed.

  • If the flag is turned on through the function (start_Flag == True), the balance is calculated and the exercise count starts.

  • Otherwise, the feedback on the starting posture adjustment is output as text.

  • When the exercise done with the correct posture is equal to the target number of exercises, the finish flag is turned on (finish_Flag == True).

  • When the finish flag is turned on, the system flag is turned on and the exercise result window appears.

Left
Bad Count

Right
Bad Count

8. Show the result of workout

  • show the result(worst pose, best pose, good pose count, bed pose count, and all count)

V. Result

1. Adjust Correct Starting Position

Adjust
Complete

2. Exercising

Contraction
Relaxation

3. Unbalance

Left
Right

4. Show Result

VI. Evaluation

Since we used the pre-trained model, we analyzed the algorithm we implemented, not the analysis of the model itself. The adjust correct starting position part and the experimenting part were largely divided and analyzed.

  • Adjust Correct Starting Position

    For evaluation, another Lat-Pull Down machine tested "Adjust Correct Starting Position" 20 times per person and 40 times in total. In this case, Positive means Correct Position, and Negative means a state in which movement to right or left is required. Accordingly, the heat map is as follows, and based on this, Accuracy, Precision, and Recall are analyzed.

    - Accuracy: 87.5%

    - Precision: 94.1%

    - Recall: 80.0%

    Looking at the above results, Recall is lowered, which means that negative is frequently recognized (FN) when positive. In other words, it can be seen that the threshold value should be adjusted so that it can be clearly recognized as positive when it is positive.

  • Workout Counting

    This time, an experiment on "Workout Counting" is conducted 20 times per person, 5 sets, and a total of 200 times. At this time, Positive means exercising in the right posture, and Negative means exercising in the wrong posture. The heat map accordingly is as follows, and based on this, Accuracy, Precision, and Recall are analyzed.

    - Accuracy: 94.0%

    - Precision: 89.3%

    - Recall: 100.0%

    Looking at the above results, the precision is lowered, which means that there are many cases (FP) that are perceived as positive when negative. In the experiment, a mirror is present and the precision is lowered due to the recognition of the person in the mirror. In other words, when using this program, it should be executed in an environment where there is nothing else that can be recognized as a person other than the surrounding me.

VII. Reference

PreviousSudoku Program with Hand gestureNextPeople Counting Embedded System

Last updated

Was this helpful?