The online 'look': L’Oréal files trio of patents for AR virtual makeup app

Writing in its first international patent ​[I​​], L’Oréal said the digital makeup artist app offered consumers beauty routine information; common makeup looks to choose from; cosmetic product recommendations based on skin type and ethnicity; and the ability for users to store look preferences. Users interacted with the platform via image capture and a dialogue box, with machine-learning used to analyse individual faces.

The result? A digital cosmetic ‘look’ that could be stored and used as an online profile picture across a range of social media platforms, including Instagram, LinkedIn, TikTok or Snapchat, or used as an overlay to live video streaming on the likes of Microsoft Teams, FaceTime or Google Hangouts.

The digital makeup artist​

A significant part of the app, L’Oréal said, was a digital makeup artist – either presented in the form of an avatar or image of a real person.

“There is a need to provide a custom try-on experience for a particular user that allows interaction in a manner that is comparable to the experience that the user may have with a personal makeup artist,” ​L’Oréal wrote in its patent filing.

“There is a need for a makeup experience in which the personal makeup artist teaches a user the steps to get a desired look on their own face,” ​it said.

Whilst many social media apps had been developed enabling consumers to create cartoon avatars with customised hairstyles, hair colours and makeup, L’Oréal said many users would likely prefer posting a made-up image of their own face – a desire this app aimed to plug.

Machine learning for better personalisation​

The makeup artist system had been designed to work on a personal mobile device or tablet computer via an app, named MyFace in the patent documents, with the possibility to link several devices via cloud services.

The digital device enabled users to take and upload photos or selfies but also interact with a dialogue box of the digital makeup artist, either by typing or speaking into a connected microphone. The goal of this dialogue was to log consumer needs, current beauty routines and desired looks.

The system then relied on machine learning to analyse images of the user’s face to identify face parts and facial characteristics like eye colour and skin tone before making recommendations on makeup looks via audio directions or video, which could include a tutorial sequence for the user. In one embodiment of the invention, this digital makeup artist could be considered a “conversational agent”​ – responding in real-time to individual questions and interacting with a user.

In any case, L’Oréal said these tutorials and recommendations were adapted to the user’s experience in applying makeup.

Digital makeup palette and fashion matching​

In two separate international patents​ [II​​ & III​​] related to this app, L’Oréal outlined two other important add-ons: a virtual makeup palette and a system that made ‘look’ recommendations based on a user’s outfit.

The company said the virtual makeup palette could include colour cosmetics but also application tools like brushes and sponges. This meant that users who selected a ‘do-it-yourself’ look could use physical gestures to apply digital makeup to their own facial image in the app.

The fashion-matching aspect, it said, enabled users to opt for makeup looks that were in “aesthetic agreement”​ with what they were wearing on a specific day or evening.

Beyond traditional virtual try-ons​

Taking virtual try-on technology to the next level was something other brands and specialists in the field were also focused on. Perfect Corp, for example, had now developed NFTs [non-fungible tokens] of digital watches and jewellery​ that consumers could purchase.

Coty had also recently filed a patent on a ‘dynamic’ virtual try-on platform for colour cosmetics​, that could also be used across Augmented Reality [AR] images or video streaming.

[I] ​WIPO International Patent No. WO/2022/146766​​
Published on: July 7, 2022. Filed on: December 21, 2021.
Title: “Digital makeup artist”​
Inventor: L’Oréal – MC. Troutman, FD. Cruz and S. Gadol

[II] ​WIPO International Patent No. WO/2022/146615​​
Published on: July 7, 2022. Filed on: December 2, 2021.
Title: “Digital makeup palette”​
Inventor: L’Oréal – S. Gokhale, R. Downs and JD. Mason

[III] ​WIPO International Patent No. WO/2022/144232​​
Published on: July 7, 2022. Filed on: December 21, 2021.
Title: “Method for determining at least one colour compatible with an outfit of a user”​
Inventor: L’Oréal – F. Zapalowicz et al.