Face Tracking
Face Tracking accesses the camera image (and on iOS also the TrueDepth sensor data, if available) to detect head rotations and face features.
On iOS, there are two modules for this: The ARKit Face Tracking module and a Mouth Tracking module. The first one only can be used if your device has a TrueDepth camera. If available, this can even track your tongue movement.
If your iOS device doesn't have a TrueDepth camera, you can only track the mouth using the Mouth Tracking module (the head tracking on iOS with the normal camera can only track head movements in 45 degree changes, so it is not very useful for MusiKraken).
On Android, you can use the MLKit Face Tracking module to do the same. It supports head rotation and mouth tracking.
ARKit Face Tracking
iOS only, and needs device with TrueDepth front camera
This module collects value from iOS ARKit Face Tracking. This means you can use your face to generate MIDI events. So you can use your head, mouth, eyes and even your tongue movements to generate MIDI Control Change (Modulation, Volume, Expression and so on) or generate notes.
Ports:
x:
This is the rotation along the x-axis of your face. Can be changed by moving your head to look down or up.
Range: -1.0 to 1.0
y:
This is the rotation along the y-axis of your face. Can be changed by moving your head to look left or right.
Range: -1.0 to 1.0
z:
This is the rotation along the z-axis of your face. Can be changed by turning your head sideways.
Range: -1.0 to 1.0
Mouth Open:
This value is controlled by how much you open your mouth.
Range: 0.0 to 1.0
Left Eye:
This value is controlled by how much you close your left eye. Note that open is 0 and closed is 1 (the opposite of the mouth open value).
Range: 0.0 to 1.0
Right Eye:
This value is controlled by how much you close your right eye. Note that open is 0 and closed is 1 (the opposite of the mouth open value).
Range: 0.0 to 1.0
Tongue:
This value is controlled by how much you stick out your tongue.
Range: 0.0 to 1.0
