Face Tracking and Facial Recognition

Challenges

face tracking with p5

Lot of projects in Github, I decided to fork this repository: face-tracking-p5js and now it is available at here: face-tracking-p5js. And then in Bashm I wrote these commands:

cd git
git clone https://github.com/talonendm/face-tracking-p5js.git

and tested e.g. 002_points starting local server at: http://127.0.0.1:8887/002_points/

Python - DLib

Similar package for Python is available, read more e.g. here It seems that both libraries detect almost the same points:

Jaw Points = 0–16
Right Brow Points = 17–21
Left Brow Points = 22–26
Nose Points = 27–35
Right Eye Points = 36–41
Left Eye Points = 42–47
Mouth Points = 48–60
Lips Points = 61–67

Note that e.g. point 27 and 32 are irises in clmtracker, check image here.

Face Substitution

Face substitution where facemodel has fitted and try out different masks. More examples available at Auduno’s Github Repo. HELI.exe FaceOff

Ideas to do

I already made first experiments in my forked repo, see face-tracking-p5js. TODO:

Combine points by lines and add additional points between the points like shown here.
Check ratios between points, e.g. eye distance divided by max(y) - min(y), some ideas available here IBM Research Releases ‘Diversity in Faces’ Dataset to Advance Study of Fairness in Facial Recognition Systems.

Hand

P5 hand detection webcam collection

Examples

image transition

Text to Speech

Code example to convert Finnish text to speech (youtube) (original code by Hao) and updated code in p5 editor :

In local experiments, you need p5.js speech library or use from CDN as

<script src="https://cdn.jsdelivr.net/gh/IDMNYU/p5.js-speech@0.0.3/lib/p5.speech.js"></script>

let speech;

var speaking = false;

function setup() {
  createCanvas(600, 800);
  speech = new p5.Speech(voiceReady); //callback, speech
  speech.started(startSpeaking);
  background(0);

  // shiffman has these inside setup:

  function startSpeaking() {
    background(0,255,0);
    speaking = true;
  }

  function endSpeaking() {
    background(255, 0, 0);
    speaking = false;
  }




}



function voiceReady() {
  // https://www.youtube.com/watch?v=v0CHV33wDsI
  console.log(speech.voices);
}

function mousePressed() {
  // speech.setVoice('SpeechSynthesisVoice');
  speech.setVoice('Microsoft Heidi - Finnish (Finland)');

  // 3 things can be changed: rate, pitch and volume
  // https://www.youtube.com/watch?v=v0CHV33wDsI
  speech.setVolume(0.7);
  speech.setRate(2);
  speech.setPitch(0.2);

  speech.speak('aluskate tulee asentaa siten että vesi valuu ulkoseinälinjan ulkopuolelle'); // say something
}
  
function draw() {
  fill(120,100,200);
  //print(frameCount);
  if (speaking) text(frameCount,40,40);
  
  if (frameCount == 100) {
    speech.setVoice('Microsoft Heidi - Finnish (Finland)');



    speech.speak('aluskate juu juu');
  }
  // speaking = false;
}

Tags: clmtrackr face tracking p5 text2speech

p5 - Face Tracking

javascript library for fitting facial models