Extensis Logo SUPPORT | FORUMS | KNOWLEDGE BASE

Facial Tagging


#1

Hello!

I have a client with a current Mac Portfolio Server installation who would like to utilize facial tagging akin to what they did in Mac OS X Photos. Has anyone done anything like this either internal to Portfolio or in an efficient workflow external to Portfolio that could be easily and consistently utilized within Portfolio. We are trying to avoid exporting out of Photos which houses the facial tags in the IPTC:Keywords field but requires EXIFTool to reset the creation dates of the files before cataloging in Portfolio. Any ideas would be greatly appreciated and we would welcome a development project of reasonable means. You can contact me via this forum or directly at hrgoldstein@gmail.com. Thanks in advance!

Howard Goldstein
HR Goldstein Consulting- Digital Imaging Strategies


#2

Hi Howard,

Very interesting post. We have discussed adding facial recognition to the product in the past. A few questions:

  • Have you looked at webservices such as lambdal.com/? If so, do you have one or two that you like?
  • Are you happy with the quality of Mac Photos facial recognition? If the creation date problem were solved, would using Mac Photos be fine, or are there other issues as well?
  • Can you tell me a little about the population of people you are interested in identifying?
  • Have you considered using the Person In Image IPTC tag for identities?

I’m not experienced with Mac Photos face recognition - I’ve only read about how it’s supposed to work. It looks like you need to build up a set of identities. If you switched to something else, you would need to start over. We might want to think a little about what that process would be and whether we could use the existing data from the embedded keywords to jumpstart it.

My initial thought is that integration with a web service would have two main parts: 1) A way of submitting pictures and retrieving matches, 2) a way of telling the service which identities to associate images with. The first part seems more straightforward, but the second would take more thought.
I look forward to hearing from you. Best regards, Brooks Brown, Sr. Software Engineer


#3

Brooks-

Thank you for your reply!

I looked at both of the services you noted and they both seem fine though Lambda didn’t offer as useful a demo other than showing how you can recognize celebrities. Would either of these be able to be integrated into Portfolio so that a user could ingest image files, have those with faces recognized and then tag them, maintaining a database of tags to be utilized automatically for the next ingest of images with already tagged faces. This is how I would like to see it work.

Mac Photos works fine and if the creation date issue were solved that would be better but the idea of having to utilize two products separately to accomplish the result does not seem to be the most efficient workflow.

This is a wealthy family with close to 400,000 photos, many of them of family events.

Did not consider using that tag, but it is almost as easy to use Keywords, which is where the initial ingest of files has the facial tags from Photos stored. It’s not a question of where to put the tag, but of making the process of tagging more efficient.

You are basically correct in how Mac Photos works. At this point it is not so much a question of where we have been but where we might be able to go. What we have now is just a number of tags in the Keywords field for image files. If an integration could re-scan those files and somehow associate the tags anew and then utilize those tags going forward, great, but we would be happy with a new system going forward and drawing a line in the sand.

Yes, the first part seems to already be doable if we see the workflow as a web site to submit pictures to, but the second part would need to allow tagging and retrieving of the files with the tag embedded and maintenance of a database for subsequent submissions. The better solution would seem to be one integrated in the Portfolio Web Client somehow, even if the calls were being made to an outside service with a usage payment to enable the feature.

Would love to continue this conversation and find a solution for my client. Feel free to contact me directly at (917) 842-5754 to discuss (I am on the east coast).

Howard Goldstein
HR Goldstein Consulting- Digital Imaging Strategies


#4

I see that James Grace is looking at this, so we may be duplicating efforts.

Yes, I don’t see a big advantage to using Person In Image. It would keep things a little more tidy and could be helpful when using images with other applications, but that is speculative.

The part 1 of the process, the way I was thinking about it, would include embedding the tags into the assets. Portfolio would submit assets to the facial recognition app, get back the tags and embed them.

A first stab at integration would look something like:

A) The FR app is trained: Files are manually uploaded and tagged within the FR app, just like you are doing currently with Mac Photos.
B) A selection is made in Portfolio, a menu item is invoked, behind the scenes:
i) Assets are again uploaded to the FR app.
ii) The trained FR app recognizes the faces and sends back the tags/names.
iii) Portfolio then embeds the tags into the asset.

A more complete integration would give the customer a way of executing part A within Portfolio. This might work by:

  1. A selection is made in Portfolio, a menu item is invoked.
  2. Portfolio then sends the assets to the FR app, and brings up a web page from the FR app that allows the customer to train the FR app for those assets.

-Brooks