Social AR 101 – An Insider’s Guide to Social Augmented Reality

The Basics – What is AR?

Augmented reality (AR) is an interactive digital experience viewed within our real-world environment. You see and interact with digital visuals and objects in the real world. This is often done through a camera view on your mobile phone or superimposed on the lens of a pair of goggles you look through.

AR truly needs an App for the user to have a premium experience. However, most people do not want to have to download another App. So we’ve focused on what people already have on their phones – Social Apps. Social Apps allow you to tap into a built-in audience without downloading any other App. All Social Apps have AR capabilities, and the platforms are open for business.

What is Social AR?

Social AR is Augmented Reality displayed through various social Apps like Facebook, Instagram, and Snapchat. It comprises of a series of camera filters that are fun, engaging, and can provide a transaction between a customer and a brand or retailer.

Other social platforms have AR capabilities, but I am going to focus on the top 3 – Facebook, Instagram, and Snapchat.

Who is Engaging with AR

Facebook has more than 2.7 billion people across its family of apps, according to its earnings report in May of 2019, and more than 1 billion of them are using AR experiences. Facebook’s AR may not be as central to the app as it is on Snapchat, but it’s available on Facebook, Messenger, Instagram, and it’s home hardware camera system – Portal.

Instagram, they have 500 million active daily users, with 88% of those users outside the US 71% of Instagram users around the globe are under the age of 35. There is not any published data on their AR use.

In Q1 of 2019, Snapchat reported that it reached 190 million daily active users. They skew to a younger audience as Snapchat reaches 90 percent of all 13-24 year-olds and 75 percent of all 13-34-year-olds in the US. Average time spent on Snapchat: approximately 25 to 30 minutes a day. Their AR lenses have been viewed 35 billion times!

More brand awareness: a great AR experience is highly shareable. Meaning if you create amazing AR experiences for your audience, people will want to share the results with their audience – this is incredible news for brands and marketers as it’s word-of-mouth and UGC (user-generated content) on steroids.

History Lesson – Who Was First?

Snapchat introduced its “Lens” camera effects after acquiring Looksery (for $150 million) on September 15, 2015. This allowed users to add real-time effects to their snaps by using face detection technology. The first effects were the Robot Scanner, Hearts, Zombie, and the ever-popular Puking Rainbow.

Facebook boosted its AR capabilities after it purchased a technology startup Masquerade (for an undisclosed amount) in March of 2016, which had created a standalone face filter App called MSQRD.

Facebook released its first face filter camera effects about a year later, during the 2016 Summer Olympics in Brazil. The first filter allowed you to add a country’s flag to your cheeks.

A lot has changed since the original face filters were launched.

Filters and Lenses – What’s the difference?

They are the same. However, different social platforms have taken a different approach to what they call them. For instance, – Snapchat calls its AR camera effects “Lenses.” Whereas Facebook and Instagram call them “Filters.” These terms are interchangeable.

Through users experimenting, playing, and social sharing, the level of brand engagement increases, generating greater awareness.

 

---

The AR Ecosystem

Social AR – Face Filters

Face Filters allow users to express themselves uniquely and creatively by adding computer-generated effects to their face. Face tracking technology allows the camera to track various facial features (eyes, mouth, eyebrows) with a mobile camera in real-time in a three-dimensional space. So this allows for a wide range of fun and exciting visual and interactive experiences.

Face Filters are by far the most popular and shareable in the Social AR space.

Social Platform Scorecard – AR Face Filters Capable
How do the different social platforms compare?

• • Snap – Yes
  • Facebook – Yes
  • Instagram – Yes

---

Social AR – Voice Filters

Voice filters can either be used to change or augment the user’s voice, or it can be used as a trigger to add an interaction or effect. So a voice can be used as a control within the filter. Snapchat has an AI component, so it understands certain words.

While limited, this shows that filters are becoming smarter and may be able to understand a much larger library.

Social Platform Scorecard – AR Voice Filters Capable
How do the different social platforms compare?

• • Snap – Yes
  • Facebook – Yes
  • Instagram – Yes

---

Social AR – Animal Face Filters

In 2018, Snapchat introduced the ability to add AR camera effects to your animals. Initially, the effects worked on certain breeds of dogs, but it had been updated to track cats now.

Social Platform Scorecard – AR Animal Face Filters Capable
How do the different social platforms compare?

• • Snap – Yes
  • Facebook – No
  • Instagram – No

---

Social AR – Hand and Body Filters

As part of the ability to create interaction with users, Social AR platforms are now evolving to track your hands and body. (This took a while as facial tracking has been around for years.) This allows the camera to track your hands and body (to an extent). So now you can create experiences where the user can video themselves interacting with AR using their hands.

Social Platform Scorecard – AR Hand and Body Filters Capable
How do the different social platforms compare?

• • Snap – Yes
  • Facebook – Yes
  • Instagram – TBD

---

Social AR – Objects

AR Objects are a broad category. It is often an item or an animated object/character that looks as if it is actually in your real world. These “objects” appear to be real because they look like they are anchored to the floor or tabletop, as if they are aware of their location in the world around them. This is often accomplished by a technology called SLAM. SLAM is short for Simultaneous Localization And Mapping. The camera is using SLAM technology to create an invisible map of its surroundings and orient a 3d object properly within the map in real time.

So, that means I can place a dinosaur in front of me, and when it walks around, it looks as if it is aware of the floor it is walking on. See the case study of the dinosaur filter we created for the Perot Museum of Nature and Science. – https://www.groovejones.com/perot_museum_snapchat_ar_lens/

Or it can be combined with Image Recognition technology, and it can be anchored to an actual real-world physical item. This is similar to the Dirk Nowitzki AR experience we created for the Dallas Mavericks, where we used the season ticket for Dirk’s final game, where he announced his retirement. Game-goers could then scan the ticket using Facebook or Snapchat to activate an AR experience that turned the key into the Dallas Mavs basketball court with an oversized 3d jumbotron that played a highlight video of Dirk’s career. See the case study – https://www.groovejones.com/dallas_mavericks_dirk_nowitzki_ar/

Social Platform Scorecard – AR Objects Capable
How do the different social platforms compare?

• • Snap – Yes
  • Facebook – Yes
  • Instagram – TBD

---

Social AR – Portals

AR Portals are 360º spaces that can be viewed through the person’s device. People engage with portals by entering the portal through a digital doorway. Doorways are placed by the person holding their mobile device/camera. The door is anchored to the floor through SLAM technology. Once you walk through the doorway, you can stand inside a 360º space and view the area through your camera.

Social Platform Scorecard – AR Portals Capable
How do the different social platforms compare?

• • Snap – Yes
  • Facebook – Yes
  • Instagram – Yes

---

Social AR – Games

AR games are usually simple and casual interactive experiences where the user uses gesture controls to play a game where they interact with AR objects.

Social Platform Scorecard – AR Games Capable
How do the different social platforms compare?

• • Snap – Yes
  • Facebook – Yes
  • Instagram – Yes

---

Social AR – Landmarks and Places

Some innovative and experimental things happen with time. Most recently Snap has released a new toolset that allows developers to augment famous landmarks, allowing users to take fun AR videos with some recognizable buildings. Imagine taking the Rainbow Vomit Face Filter and having it work on the Eifel Tower.

Well, now you can! Snapchat’s AR Landmarks technology can recognize specific landmarks, and now you can turn the Eifel Tower into a dancing rainbow vomiting video. This is cool. However, it is limited. This technology ability was promoted at Snaps Developer Summit this year, where they launched the ability for approved developers to add AR effects to the Eiffel Tower (Paris, France), Buckingham Palace (London, UK), the Capitol (Washington DC, USA), the Flatiron Building (NYC, USA) and the TCL Chinese Theatre (Hollywood CA, USA).

At the Facebook F8 event in April 2019, there were some hints that they might have the ability to create AR Places, but nothing materialized.

Social Platform Scorecard – AR Landmarks or AR Places Capable
How do the different social platforms compare?

• • Snapchat – Yes
  • Facebook – No
  • Instagram – No

---

Social AR – Murals, AR Billboards, and AR Posters

The OOH (Out Of Home) advertising industry woke up when we launched the largest AR mural on Facebook for the Dallas Mavericks last year. An AR Mural is an experience where we take a traditional printed medium, like a mural, billboard, or poster, and allow users to engage with it through their mobile device. The AR Filter uses image recognition to identify the printed media piece and activate a larger-than-life advertisement.

For the Dallas Mavs, we created a series of AR experiences that were activated initially through Facebook and then later through Snapchat. See the case study – https://www.groovejones.com/dallas_mavericks_luka_ar/

Social Platform Scorecard – AR Murals, AR Billboards, and AR Posters Capable
How do the different social platforms compare?

• • Snap – Yes
  • Facebook – Yes
  • Instagram – TBD

---

Social AR – Shopping

Every retailer is looking to use AR to introduce their products to users through AR, and both Snapchat and Facebook are aware of this. AR is a way of optimizing the consumer shopping journey. From Awareness > to Interest > to Consideration > to Intent > to Evaluation > to Purchase. Many of AR’s capabilities fall in the Evaluation portion of that journey. By allowing 100% of the consumers to see how the product looks on them or in their living room, retailers can increase the metrics to push consumers into making that purchase decision.

Snapchat can add a Shop Now button to an AR experience through its Shoppable AR functionality? This is promoted through a media buy. (Snapchat has partnered with Amazon to include visual search through their Apps camera to find a product on the retailer’s site for purchase. While not an AR camera effect but part of the XR world.)

Facebook worked with Michael Kors by creating an AR ad that let Facebook users “try on” sunglasses and then make a purchase through the platform.

Nike was involved in Facebook’s beta-AR Shopping, which integrated an AR experience in Messenger. Sneaker fans had to obtain and line up a sequence of four emojis distributed by Messenger-based social influencers working with Nike. Customers that correctly entered the four emojis were given a link that, when clicked, opened the phone’s camera and allowed the user to see the sneakers atop a pedestal surrounded by red carpet via AR. The Red Carpets are a limited edition shoe branded by basketball star Kyrie Irving. The user could then examine the sneaker by walking around the pedestal, viewing its superimposition over a real environment through the camera screen. When the user exited the experience, a screen offered the opportunity to purchase the shoes. The limited shoes sold out in 1 hour.

Social Platform Scorecard – AR Shopping Capable
How do the different social platforms compare?

• • Snapchat – Yes
  • Facebook – Yes
  • Instagram – Yes

---

Social AR – Scavenger Hunt

At the recent F8 conference, Facebook released a new feature to their platform – Scavenger Hunt AR. They activated it by creating a series of Mural AR experiences and tied them together so guests could unlock each mural, then receive directions to the next mural. Complete them all, and you are given a prize code. This code could then be used to redeem a prize given out on-site. This is unique for the Facebook/Instagram platform. We are already working on productions that can use this.

Social Platform Scorecard – AR Scavenger Hunts
How do the different social platforms compare?

• • Snapchat – No
  • Facebook – Yes
  • Instagram – Yes

---

What Are the Differences

Each social platform has AR capabilities, but they all have taken a different approach. One platform may have a unique feature for a while, but we are sure the others will be fast to follow as the space is highly competitive. They each have their own development platform. Snapchat uses Lens Studio, and Facebook/Instagram uses Spark AR Studio.

Here are a few of the general production differences.

Files Size?

For instance, all the platforms have different file size constraints, and while they all can record and share, they all have different lengths of time that you can record and capture.

File Size Constraints

• • Snapchat – 3 MB
  • Facebook – 2 MB
  • Instagram – 4 MB

Record and Share Video Capabilities?

All the social platforms allow users unlimited time to interact and play with the AR filters. However, they have different limitations to Record and Share the videos that users make.

Video Recording Time

• • Snapchat – 3:00 minutes (or unlimited, as we have never tested the limits past 3 minutes)
  • Facebook – 25 seconds
  • Instagram – TBD

Tracking and Analytics

They all provide similar tracking and analytics. Tracking analytics are updated every 24 hours. Facebook / Instagram gives a little more detailed data. Neither platform offers individual unique user information.

Facebook/Instagram

• • Impressions – The number of times an effect was displayed on the screen. This includes when an effect is displayed while a person is trying it in a camera, as well as when the effect is displayed in a shared story or post.
    • Camera impressions
    • News Feed impressions
    • Story impressions
  • Captures – The number of times someone took a photo or recorded a video that used this effect. This includes captures that were not shared. Captures do not include products used in a live video.
  • Shares – The number of times someone shared a photo or video that used this effect. Shares can be to a story or post.

Snapchat

• • Views – The number of times a “Lens” has been viewed.
  • Scans – The number of times a “Snapcode” has been scanned.
  • Shares – The number of times a “Lens” has been shared in Snapchat.