The Different Types and Uses of XR (Extended Reality)

9th March 2021

XR means Extended Reality, and encompasses any kind of digital media that builds upon or replaces a viewers reality. It is an umbrella term for all 360°, Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) media.

Read on to understand more about each of these fast emerging technologies, and examples of their use.

 

360˚ Media

360°: Live Streams

These are just like normal live streams, but thanks to omnidirectional 360˚ cameras, viewers can now move their view and see the entire room. As opposed to the classic 16:9 view where the camera man and director are in total control of what you see, 360° cameras record the entire environment around them and consist of multiple lenses pointing in different directions. The resulting footage is then live stitched into a 360° sphere and streamed online, where users can move their view around to expose different parts of the room. The best part? It's watchable through phones, desktops, tablets and VR headsets, meaning everyone gets a chance to be immersed in something happening at that very moment, anywhere in the world. It's known as both 360˚ and VR live streaming.

We work with artists, promoters and event organisers to capture concerts and events with our 360° cameras - check out this example we did in lockdown with Kate Stapley. These can be live streamed, and/or released at a later date online. When an event happens in both a physical capacity and a digital capacity, it is known as a hybrid event.


 

360°: Music Videos

Also known as VR music videos, these immersive experiences are almost incomparable to the music videos we are used to. 360° filmmaking revolves around the viewer, giving them control over the viewer and making them a character in the story. By imitating our natural way of looking around to see things, our brains become partially convinced this is our reality - even if your mind is not - and it's a therefore a totally different way of creative film-making. Whether it's live action, animation, or both, there is no limit to what an experience might be. 360° films have been used to provoke empathy, understanding and awe, putting people in positions they could never be in in real life.

The Music Trip applies this to music videos. We create 360° music videos for artists, taking any kind of concept or basis of a song and transforming it into a liveable experience. The fans and viewers are no longer simply watching a music video, they are in it ‑ fully immersed and able to look around in a world created entirely around a song. It's perfect for artists that have a story to tell and a strong message or concept they want to express.

Just like 360° live streams, they can be watched on phones, desktops, tablets and VR headsets. Check out this example from Mac DeMarco.

 

 

Virtual Reality

Virtual Reality (VR) media differs from 360° media in that it's not filmed as it must be interactive and responsive to the player's actions. Think games, creative apps and immersive experiences where you can dictate the story. It’s often confused with 360˚ media as not only they are very similar in nature, but they can both watched with a VR headset. Incorrectly though, 360˚ media does not react to the viewer, where VR does. There are hundreds of VR headsets on the market, with the most popular consumer headset currently being the Oculus Quest 2.

 

VR: Creating

Bespoke VR apps are designed everyday by a range of different organisations looking to give audiences an interactive experience of their brand. By making complex procedures easy and rewarding with the simple actions, any one can be a professional in Virtual Reality.

For artists and labels, it's stand out way to get anyone ‑ not just professional music makers ‑ engaging and creating with music on their roster. See how this creative app has allowed everyone to effortlessly become a producer through VR! Virtual Reality creatives apps add a huge amount of value to a brand by providing experiences that your fans will genuinely thank you for; it’s not a just piece of content they’ll see once and scroll past, it’s an activity that can revisit and enjoy time and time again.

 

VR: Gaming

What does Fortnite's Travis Scott event reveal about the future of  entertainment?VR was created with gaming in mind, and you can understand why. We physically and emotionally become the characters, and live the game’s story after being absorbed into its world. In 2020, Travis Scott performed via a digital avatar in the game Fortnite - it drew 12 million gamers to the virtual space and was one of the most successful virtual concerts to ever take place. The use of the whole of Fortnite island as the stage, and the incredible numbers it drew and PR it generated is testimony to the gaining traction of music in VR.

Numerous music VR games have also been developed over the last few years, like Beat Saber (below) who blended drumming with Star Wars ‑ becoming one of the most popular VR games to date. Players chop blocks with their sabers and can choose which song they ‘play’ - artists like Eminem, Linkin Park and Imagine Dragons have all created special runs for their tracks.

 

 

Augmented Reality

Augmented Reality (AR) is when Computer Generated Imagery (CGI) sits on, interacts and blends with real world environments. Where Virtual Reality completely replaces it, Augmented Reality only partially alters (augments) it.

 

AR: Live Streams

Augmented Reality can be seen in any live media, as its key element is CGI interacting with live footage. Last year, we saw it on Strictly Come Dancing during one of Bill Bailey and Oti Mabuse’s dance, when an AR elephant appeared on the dance-floor with them.

Just like this, we work with artists to add an extra dimension to their live streams. We layer CGI over the stream to create a more exciting experience that couldn’t be possible in real life. It can be videos, audio, animation, graphics and 3D images, and can be integrated into both traditional live streams and 360˚ live streams.

 

AR: Social Media Effects

How to Unlock The Most Stylish Filters On Instagram Stories | Teen VogueThe most popular use of this is on social medias like Instagram, where users can take videos and photos that are digitally altered as they're taken. Known as effects and filters, these might be altering the look of someone; transferring their face into your media (as shown by Glass Animals); or placing objects or stickers in their real environment ‑ to name a few. We love creating interactive album artwork: when people scan an album cover with their phone on a social media app it comes alive and shows them exclusive content.

We're seeing more and more musicians creating effects for social media that help promote a new release or tour, as fans play with the effect and share it to their 100's (or 1000's) of followers, organically leading everyone back to the creator's profile.

 

AR: Web Applications

Augmented Reality isn't limited to social media, and more complex designs need their own Web space.
For Sam Smiths release of 'Diamonds', they worked with Spotify to create an AR Web page that people scanned on another phone to reveal a digital Sam Smith dancing on their phone. Concepts like this mean that we can have global superstars dancing on our own kitchens.

 

AR: Smart Lenses

We've been hearing a lot about AR glasses recently and their predicted future in our day‑to‑day lives. The idea is that one day we'll be walking around seeing the time, date and all of our notifications floating in our vision as and when we want them. Companies like Mojo Vision are taking the concept from glasses to contact lenses, hoping that one day we might not all be constantly looking at our phones.

As these hardwares become more consumer available, expect to see your current song playing in the corner of your eye, or billboard adverts coming alive as you look at them.


 

Mixed Reality

MR: Education

Mixed Reality (MR) is where digital and physical objects co‑exist and interact in real‑time. It's not too dissimilar from AR and is a term associated mainly with the Microsoft Hololens. The Hololens is Microsoft’s head mounted hardware that is primarily focused on education and training as it allows people to integrate their computer with their vision and collaborate, design and and study through just wearing the headset.

It's yet to be adopted by the music industry, but that won't be for long. Similar to VR apps, it could be a great tool for musical education and training.
As with most XR technology ‑ the sky is the limit to what this tech can be applied to!

 

XR Production

Virtual Sets: Film-Making

Forging new paths for filmmakers on The Mandalorian - Unreal EngineXR is taking off behind the scenes, too. Instead of using a green screen on set during filming and inserting the backgrounds in post production, more and more production companies ‑ including The Music Trip ‑ are using Virtual Sets that are made entirely of LED screens. By having a pre-produced CGI environment on the screens, it means there's no bleeding of the green onto the actors and objects, and the cast can respond more authentically to the scene as they can see it. Camera tracking means that as the cameras move, so does the display on the LEDs to accommodate the change in perspective ‑ essentially using the basics of VR to replace green screen technology. The virtual environments are built in advance using the Unreal Engine software.

See the above example of how virtual sets were used in Disney’s award-winning The Mandolorian.


 

The Future of XR....

The future of XR technology really is a world of possibility. With digital media, anything can be created and consumers are always waiting for new XR media to come and blow their minds. At The Music Trip, we are dedicated to uniting the Music Industry with the world of XR, and we work with musicians creatives everyday to make sure no one gets left behind as we head into the digital future.

 

Interested? Get in touch! Email Sophie or Murray at hello@themusictrip.co.uk to start the process of creating 360˚, VR, AR media for your brand.

Other News