Apple has detailed how it used machine learning advancements to introduce significantly improved people recognition in iOS 15, including in situations when a face isn’t clearly visible.
The company lists “improved recognition for individuals” as a new feature in the iOS 15 version of the Photos app, though the web page is sparse on details.
However, a new blog post on Apple’s machine learning site reveals that the Photos app can identify people in a variety of scenarios, including if their faces aren’t clear to the camera.
One of the methods Apple uses to achieve this is to match the faces and upper bodies of specific people in the images reported AppleInsider.
“Faces are frequently occluded or simply not visible if the subject is looking away from the camera,” Apple wrote.
“To solve these cases, we also consider the upper bodies of the people in the image, since they usually show constant characteristics — like clothing — within a specific context. These constant characteristics can provide strong cues to identify the person across images captured a few minutes from each other,” it added.
The company takes a full image as an input, and then specifically identifies the detected faces and upper bodies.
It then matches the faces to the upper bodies to improve individual recognition in situations where traditional facial recognition would be impossible.
The mechanism uses on-device machine learning to ensure privacy. Apple has also taken steps to ensure that the process minimizes memory and power consumption.
“This latest advancement, available in Photos running in iOS 15 significantly improves person recognition. Using private, on-device machine learning, we can correctly recognize people with extreme poses, accessories, or even occluded faces and use the combination of face and upper body to match the people whose faces are not visible at all,” Apple wrote.
