November 12, 2018 by Muhammad Aqib
Learn how to make an FM synthesizer that creates noise based on your face using Wekinator, openFrameworks, and ChucK.
| 1 | Wekinator | ||
|
|
|||
| 1 | ChucK | ||
|
|
|||
| 1 | openFramworks | ||
|
|
|||
In this article, learn how to use the face tracker made in openFrameworks to control an FM synthesizer in ChucK. The data will then be trained by Wekinator machine learning software.
Inputs will be given to Wekinator from openFrameworks. The face tracker made in openFrameworks will send the x, y position and width of your face to Wekinator. Wekinator will be trained according to these inputs and will send the output to ChucK where the FM synthesizer will be controlled. The FM synthesizer will receive five continuous outputs from Wekinator and will make different sounds according to these.
We will record samples for the three gestures: the first will be with a face right in front of the camera, the second will be a face at a further distance from the camera, and the third will be with no face or other objects in front of the camera.
If you are new to openFrameworks, you need to first install it on your computer.
The executable file for Mac is available from Wedinator's examples page. If you use Windows, get the source code from the examples page and then you need to compile it.
A screenshot of where you'll find a link to the source code for openFrameworks.
Download the file according to your OS from there. Unzip the file and run it. It will use the laptop's webcam (or any webcam attached to your computer) to track your face. The output window will look like this:
The program automatically starts to track your face complete with sunglasses!
On the output side, we will use ChucK. ChucK will receive 5 continuous outputs from Wekinator and will make the sounds accordingly.
The project code for ChucK is available for download and is titled ‘FM Synthesis.’ Download for the operating system that you are using.
Once downloaded, open the terminal and go to the directory where you have placed the project code and write the following line:
chuck FMSynth_5ContinousOutputs.ck
The opening terminal directory.
ChucK will begin listening for outputs from Wekinator and, upon receiving the output, will change the pitch of sound.
Open Wekinator and make the settings match the ones shown in the below figure.
Set up the fields of your new project identically to the ones shown in this screenshot.
Set the inputs to 3 and the outputs to 5. Select the output type to “all continuous” and leave other settings as they are.
Wekinator will receive 3 inputs from openFrameworks and, after training, will send 5 different outputs to ChucK.
ChucK will produce different sounds according to Wekinator's outputs.
Once those settings are complete, click "next" and this window will appear:
Position your face in front of the camera and click "randomize." Set the output in the Wekinator window to 1 and start the recording for a second. It will record some samples.
Place your face closely to the webcam for output 1.
Next, sit further from the webcam, select output 2, and click on randomize. Record for a second and to get some samples.
Sit farther away from the webcam to record output 2.
Now, select output 3, move away from the camera, and click on randomize. Start the recording for half a second.
For output 3, record without your face in the camera's frame.
After setting these three outputs, click "train" and then click "run." Now, when you will drag the green box in the processing window, ChucK will produce music according to the output.
The video below shows how to set up this project and get it working.
For custom projects, hire me at https://www.freelancer.pk/u/Muhammadaqibdutt
