FaceNet: A Unified Embedding for Face Recognition and Clustering

A single picture is surprisingly effective in recognizing that person in another photo. I've been playing around with a python wrapper on dlib's facial recognition [0] and it's astonishing how it can identify people from just one photo. It's not like other machine learning methods that include training a classifier or even fine tuning a classifier. It works by encoding the persons face to a 128 dimension vector and then using that for comparison. The accuracy from most methods is between 99.2% - 99.8%, but the problem is that the training samples are too easy and controlled. It's sensitive to lighting. Google's most recent paper [1] on Facenet found 99.63% on the easy Labeled Faces in the Wild (LFW) dataset, and an impressive 95.12% on the Youtube faces dataset, presumably a much more difficult dataset. [0] https://github.com/ageitgey/face_recognition [1] https://arxiv.org/abs/1503.03832

The authors didn't try very hard to solve the problem (although it's still slightly interesting). Many networks can detect if two never-before-seen objects are the same or different. Simple example is Koch's Siamese Neural Networks[1]. More useful example is Facenet[2], which can tell if two faces are the same or different, even if never trained on these specific faces. Facenet learns an n-dimensional embedding and similarity is estimated by euclidean distance. These are cool papers by the way. The Omniglot dataset[3] used in Koch is a sort of an "inverse MNIST". Omniglot has 20 examples each of 1600 different characters, vs MNIST's 60,000 examples of 10 different characters. [1] https://www.cs.cmu.edu/~rsalakhu/papers/oneshot1.pdf [2] https://arxiv.org/abs/1503.03832 [3] https://github.com/brendenlake/omniglot

They would probably use FaceNet[0] then, if they only want to detect faces, as that should give better results. [0] https://arxiv.org/abs/1503.03832

A single NN can predict more than one class of object. The ImageNet competition has 20,000 classes. There's also image segmentation as another poster has pointed to. In the case of FB face tagging, they'd have learn an embedding space for faces, and when a new image comes in they'd place it in the embedding space along with all the person's connections and find the nearest neighbors. See https://arxiv.org/abs/1503.03832 or the implementation https://cmusatyalab.github.io/openface/

The Google FaceNet and other similar system typically perform an approximate nearest neighbor search in a ~128 dimensional embedding of faces. Here is the paper for the reference: http://arxiv.org/pdf/1503.03832v3.pdf Also 70% accuracy is like really really bad. Here are results from MegaFace Challenge, which involved identifying face in presence of Million distractors: http://megaface.cs.washington.edu/results/fgnetresults.html