Reverse engineering Apple Photos to extract photo metadata and do fun things with it
May 21, 2020 9:58 PM Subscribe
Reverse engineering Apple Photos to extract photo metadata and do fun things with it
I've been building tools to liberate my photo metadata from Apple Photos. In addition to machine learning labels (it automatically tags my photos with categories as detailed as lemur, pelican and seal) I also found an intriguing collection of quality scores, with names like ZPLEASANTCAMERATILTSCORE and ZSHARPLYFOCUSEDSUBJECTSCORE. I've used them to identify my most aesthetically pleasing photographs of pelicans according to Apple's fancy machine learning algorithms!
I've been building tools to liberate my photo metadata from Apple Photos. In addition to machine learning labels (it automatically tags my photos with categories as detailed as lemur, pelican and seal) I also found an intriguing collection of quality scores, with names like ZPLEASANTCAMERATILTSCORE and ZSHARPLYFOCUSEDSUBJECTSCORE. I've used them to identify my most aesthetically pleasing photographs of pelicans according to Apple's fancy machine learning algorithms!
Role: Creator
This project was posted to MetaFilter by filthy light thief on June 15, 2020: Digging into the AI ratings in Apple Photos
So, first of all: you are entirely correct - the pelican close-up is a much better pelican photo than the one in flight. (Although that's really good, too.)
Second: I somehow had no idea that Photos does all this tagging. I kind of hate it, in a way - I hate the facial recognition stuff, because I am a privacyphile and I don't want it grouping my photos of people in any way, shape, or form, even if it DOESN'T actually know who they are or plan to put them all in my Permanent iRecord. I just wish I could turn it off. Location info, too.
But sheesh - searching for "trees" or "plants" gets me loads of photos, as does "birds" - wow, as does "heron". (But nothing for "egret," which most of those supposed "heron" photos actually are.)
It's fascinating that there doesn't seem to be a way to see all the categories that have been assigned (is there?). I tried typing in "white" to get all the photos of white egrets, but it doesn't categorize things by color, apparently - but typing "green" shows me two buildings it erroneously things are greenhouses, and that "white" search did find 14 photos categorized under "white wine" (one of which is a glass of water, and one of which is an office building).
I am now super excited to check out osxphotos, and very intrigued to find out what ELSE the Photos app thinks I've been taking pictures of.
Congratulations on this fabulous project, simonw, and thank you for sharing it with us here!
posted by kristi at 8:18 PM on June 4, 2020
Second: I somehow had no idea that Photos does all this tagging. I kind of hate it, in a way - I hate the facial recognition stuff, because I am a privacyphile and I don't want it grouping my photos of people in any way, shape, or form, even if it DOESN'T actually know who they are or plan to put them all in my Permanent iRecord. I just wish I could turn it off. Location info, too.
But sheesh - searching for "trees" or "plants" gets me loads of photos, as does "birds" - wow, as does "heron". (But nothing for "egret," which most of those supposed "heron" photos actually are.)
It's fascinating that there doesn't seem to be a way to see all the categories that have been assigned (is there?). I tried typing in "white" to get all the photos of white egrets, but it doesn't categorize things by color, apparently - but typing "green" shows me two buildings it erroneously things are greenhouses, and that "white" search did find 14 photos categorized under "white wine" (one of which is a glass of water, and one of which is an office building).
I am now super excited to check out osxphotos, and very intrigued to find out what ELSE the Photos app thinks I've been taking pictures of.
Congratulations on this fabulous project, simonw, and thank you for sharing it with us here!
posted by kristi at 8:18 PM on June 4, 2020
Good post
posted by ScienceFeed at 6:43 PM on June 7, 2020
posted by ScienceFeed at 6:43 PM on June 7, 2020
« Older A Chrome extension for managing your tabs... | London's coronavirus street ar... Newer »
posted by kathrynm at 11:24 AM on May 22, 2020