This week, I have been working on improving the zoom-in gesture. Before, the dynamic gesture consisted of 2 static ones: “fist” + “thumb_index”. After testing this gesture out, even though it was recognized relatively high number of times, I decided to improve it by implementing a new metric. I decided to calculate the distance between the thumb and the index finger as an indicator of sooming in and zooming out.
The gesture is triggered, when the user show a “fist” gesture. It zooms in if the distance between the thumb and index_finger is more that 0.13 and zooms out if the distance is less than 0.05. That way, the gesture is smoother and requires less movement. In order to stop the soom mode, the user is supposed to show the “palm gesture”.
This is the result:
The pinch in gesture is more intuitive and requires less movement. To achieve that, I did not have to retrain my model. Instead, in the script of the camera and the model, I added the metric of distance between thumb and thumb_index that is calculated using Mediapipe framework.
Besides, this week I was working on refining my thesis, editing parts of it and creating chart to visually understand the pipeline.
I also created a flowchart that presents a user walkthrough and possible error handling. Even though not all of the error handling was implemented in the coding part of the project, it is important to identify them for future enhancements
Fully working pipeline video demonstration: https://drive.google.com/file/d/1V1UNRyJT2IpGC7y2KxzfDJDr3JU5lnRu/view?usp=sharing
Leave a Reply