How to get face embedding / encoding?

[josephernest]

Congrats for this nice project @ipazc!
I see the output of your algorithm is something like:

[
    {
        'box': [277, 90, 48, 63],
        'keypoints':
        {
            'nose': (303, 131),
            'mouth_right': (313, 141),
            'right_eye': (314, 114),
            'left_eye': (291, 117),
            'mouth_left': (296, 143)
        },
        'confidence': 0.99851983785629272
    }
]

i.e. it gives the bounding box, the keypoints (nose, mouth, eye, etc.).

But how to get an face embedding / face encoding, to be able to do face identification?

[MattyB95]

I may have misunderstood what you are asking, but this project isn't about getting face encodings for recognition purposes. For that, you will need something else, such as https://github.com/ageitgey/face_recognition.

[josephernest]

@MattyB95 Maybe I misunderstood this project, but I thought MTCNN + FaceNet not only allows face detection, but also recognition / identification, by assigning a 128D vector embedding to each face.

https://github.com/davidsandberg/facenet

Is it correct @ipazc?

[MattyB95]

@josephernest I'm not completely familiar with FaceNet but wouldn't that be providing the face recognition/identification encodings. This project would be more about cropping the image to the facial region for that purpose, but I will let @ipazc give his verdict :)

[imnimn]

MTCNN is only used to detect faces in an image,
The value in the the "box" are the corners for the face box in an image
To get the face encoding you will need to pass the the pixels resulted from cropping the image to a face corners to one of the models that are used for face encoding, as facenet, deepface,....

[josephernest]

@imnimn Is there a Python implementation that packs together: MTCNN to get the face box + facenet or deepface for the encoding?

[imnimn]

@imnimn Is there a Python implementation that packs together: MTCNN to get the face box + facenet or deepface for the encoding?

@josephernest please check this article:
https://arsfutura.com/magazine/face-recognition-with-facenet-and-mtcnn/,
It is implementation is in this repository
https://github.com/arsfutura/face-recognition