Invesco QQQ wanted an interactive and technology-driven fan experience to highlight their sponsorship at this year’s NCAA Final Four Men’s and Women’s Championships. Groove Jones worked with their agency, 160Over90, to design and develop an experience so sports fans could see and feel what it’s like to be on the court where they can recreate a moment during a pivotal NCAA championship game.

Invesco QQQ is the Official ETF of the NCAA and March Madness. Their idea centered on putting fans on a basketball half-court, surrounded by LED walls displaying a raucous crowd within a stadium arena, reacting to your every move.

Groove Jones focused on technologies inspired by innovations from companies within the Invesco QQQ ET. The experience was activated at the Men’s Final Four Tip-Off Tailgate in Phoenix, AZ, and the Women’s Final Four Tourney Town event in Cleveland, OH. 

Sizzle Video of the Experience

 

Take A Shot At Greatness – Multiple Games To Choose and Play

Groove Jones created three game experiences that college basketball fans choose to see if they can step up to greatness and recreate a moment in NCAA Final Four history displayed on the LED walls.

• The Buzzer Beater – You have 5 seconds to recreate a game-winning shot.
  • Kris Jenkins – Buzzer Beater, 2016 – A legendary shot during the ’16 Final Four.
  • Arike Ogunbowale – Buzzer Beater, 2018 – Tap into GOAT mode with this shot.
• The Clutch Free Throw – The clock’s at zero, and you can win the game by sinking two free throws.
  • Kyle Guy – Clutch Free Throw, 2019 – A championship comes down to free-throw.
  • Monique Reid – Clutch Free Throw, 2013 – Miss one, and it’s all over.
• Hot Hands – You have 30 seconds to make as many baskets as possible.
  • Candace Parker’s – Hot Hands, 2007 – An unstoppable force.

After registering to play, players enter the Innovation Arena – a physical space of a half-court basketball court, surrounded on 3 sides by 15-foot-tall LED walls, which display a replica of the Final Four arena. This made users feel as if they were walking onto the court of the big game. 

Once inside, the experience begins with a highlight video of their chosen Final Four game-winning moment. This played out on the overside walls within a giant NCAA Final Four arena. After the highlight video, a hologram appears – recapping the moment and prepping the user for their big game-winning moment. 

When the game begins, the lights come up, revealing an arena filled with cheering fans, photographers, and cheerleaders. As you move around, you realize that everyone in the arena is looking at you, flashing taunting signs and oversized graphics of your actual face to distract you. Then, the voice-over announcer says your name, and he broadcasts your play-by-play performance. The pressure is on!

The crowd simulation is powered by AI Computer Vision, which tracks your movements, performance, and position on the court. This data controls the CGI crowd, directs them to where you are in real time, and directs their reactions. The AI crowd reacts with cheers when you make a shot and taunts when you miss a shot. 

Leaderboard System Tracks Performance

At the end of each game, the performance data recorded on the court during the game made it to the leaderboard. As players exited, they received Invesco QQQ swag and took photos and selfies of their leaderboard standings. A highlight reel of users’ performances was also displayed on the exit screen as an idle loop.

Inside the Game – An LED Volumetric Stage 

The Innovation Arena leverages the same techniques as a volumetric stage. A volumetric stage is a technologically advanced environment traditionally used in the entertainment industry. It displays video in three dimensions, allows the recording of performances from all angles, and has LED walls and ceilings that provide visuals of background environments. This visual data is processed in real time to create a three-dimensional digital model of the scenes or actions. National Experiential was brought on board to provide the AV hardware system to bring this to life.

When the experience is activated, it looks like you are standing in a basketball arena. A 5.1 sound system rumbles the floor and captures the excitement of a real game.

Scenic Route designed and fabricated the physical build based on concepts from the 160Over90 team, which included an entrance for guests to queue up to register and an onboarding area for brand ambassadors to assist them when it was their turn.

Behind the Scenes Technology Innovations

The Groove Jones team incorporated AI technologies into multiple production components. Here is a breakdown of each one.

AI Advanced Computer Vision for Player and Ball Tracking

We needed to obtain the player’s real-world position and properly trigger animations and events on the LED walls within the game engine Unity. The detection module utilized a single 1080p webcam strategically positioned above the backboard to capture comprehensive views of the court area. This setup allowed for the effective use of AI to analyze real-time video feeds, recognizing player movements and ball positions within a predefined region of interest on the court.

Monocular Bird’s Eye View Detection

The AI algorithm could track the 2D position of players from a top-down perspective, a complex task due to the unusual angle and the presence of human avatars on the surrounding LED walls.

Ball and Player Tracking

The system adapted traditional object detection models to accurately identify and track the basketball and players from the top-down view. The tracking was refined by developing a custom dataset with top-down footage to train the models effectively.

We prototyped the system within the Groove Jones studio and created a halfcourt identical to what would appear onsite at the events.

Homography and Real-World Coordinate Mapping

After detecting objects in the video feed, their pixel coordinates were converted to real-world coordinates using a homography matrix. This transformation was critical for accurately triggering interactive animations on the LED walls in response to player movements and actions in the game.

AI Intelligence Crowd Simulation

The CGI people around the player, displayed on the LEDs, needed to react to the player. A sophisticated crowd SIM was created so the individual people within the arena could look at the player on the real-world court. Each character was then programmed with unique animations – idle, celebratory, and taunt so the crowd felt natural and real.

AI Advanced Computer Vision and Custom Hardware Sensors for Score Detection

The camera-based and sensor-based detection results combined for accurate score detection with the hardware sensors on the hoop. We designed and fabricated a set of custom camera and sensor rigs that we integrated into the actual backboard and rim.

Here, you can see the custom cameras placed above the rim and a sensor below the rim to detect and track the ball.

AI-Powered Multi-Camera Social Share Video Production

One of the primary goals of the activation was to deliver a bespoke video for each participant so they could share it on social media. We developed a multi-camera system that leveraged the AI logic of the game to serve as the director and select key action shots of the player moments before they shoot and the results of their performance to produce a slick hype video of your experience. This was then compiled and produced in the cloud and then sent to the guest to share on social.

This was compiled and produced in the cloud and then sent to the guest to share on social media.

AI-Powered Announcer Voice

Every participant needed to register to play. This enabled our team to collect the guest’s name to feed the leaderboard system displayed on the wall. It also was a data point for our text-to-speech AI system. During production, we hired voice talent as the announcer’s voice. This was then used to train our AI engine to call play-by-play action in real-time on the court. This included the ability to announce the player’s name when the game starts, as well as to add colorful commentary and even create nicknames for the guests based on their performance. 

This comprehensive approach not only enhanced the fan experience by making them feel like they were part of a real NCAA basketball game but also showcased the potential of AI in creating interactive and engaging sports environments.

Fan Favorite

The experience was a breakout at both of the events. It was the most technically advanced program that connected with fans. It provided a personalized, hands-on experience that put fans into an immersive experience while simultaneously creating a crowd attraction where they participated by cheering the player on. Collectively pulling everyone into a moment of greatness.

Thousands of people saw and experienced the activation, and it helped Invesco QQQ stand out in a crowded field.