Gesture Controlled Flower Art with ml5.js PoseNet and p5.js.
Using just body gestures to create artwork is really captivating .The combination of artwork created in p5.js and using the power of ml5.js PoseNet to make it gesture controlled,take a simple artwork to the next level.
This tutorial is inspired by Daniel Shiffman Hour of Code PoseNet tutorial .This tutorial provides the platform to build ,extremely interesting projects.It helped me come up with this idea .Hope you enjoy it.
Create a new project using p5.js Web Editor. The canvas size set to windowWidth and windowHeight and the background colour is 220 .
First thing to start with is to make a global variable of video. Use this variable in function setup(),to call the createCapture(VIDEO) and adjust the size of the video in width and height.In the function draw(),use variable video in the image() function ,to make the the video appear on the canvas.
Now the important step to add the ml5.js library extension in the index.html .You can find the extension link on the ml5.js website.
Next ,we are going to follow the basic tutorial for ml5.js poseNet by Daniel Shiffman Coding train tutorial to upload the pre-trained poseNet models (the code which is explained in the video is given below).
Once the model is loaded,the body part as explained in the tutorial is selected and used ,in this case we are using the keypoint [0] for “ nose ”. Use the link provided on the ml5.js website for parts selection.
Time to make some interactive flowers using poseNet. First step, is to comment the code for background and image , this will stop the video from appearing and the background is disabled .Then create a square(instead of a circle) with the rect() and set is origin to CENTER using rectMode().Set the fill to noFill() and just give the stroke() and strokeWeight().
Next step, is to make a global variable of ‘angle’ and set it to 0. In the function draw , increment the angle. Use this angle variable inside the map function and the rotate function .The local variable ‘b’ for map function can now be used in the last two parameters of the rect(),it will change the size of the square. To make the rotate function work in 2D we need translate(),which will set the origin according to the parameters given .
CHALLENGE:
Try ,using the mouseX and mouseY in the translate() as parameters and see what happens.Use the variables to set the stroke.
Finally replace the translate() parameters with the noseX and noseY of the poseNet model .
It will use the camera function to detect the x and y position of the nose and form a flower shape when remain static .Otherwise ,squares of different sizes with pulsating rotation can be observed ,when moving across.Congratulations ,you made your own interactive artwork.Happy Coding!!
Resources:
1-Getting Started with p5.js- A book about programming for in p5.js.
2-p5.js Reference:References for all the p5.js functions.
3-ml5.js Reference: Reference for all the ml5.js functions
4-Daniel Shiffman Coding train Tutorial- for ml5.jsPoseNet.
