MWC2019 Recap

One tiny piece of MWC19

Late last month I went on a fascinating adventure into the global telecommunications industry at Mobile World Congress 2019 in Barcelona, and saw where they see themselves over the next few years. Smartphones and LTE networking are both very mature markets, so everyone is looking for what’s coming next. And it’s coming sooner than a lot of people think.

One thing that kept coming up throughout the conference and across the expo floor was about what the different generations of networking enabled, and what that means for (consumer) adoption of 5G: 2G brought messaging, 3G the web anywhere you go, 4G/LTE made video explode, and the most compelling vision of 5G so far is spatial computing.

Spatial computing is how I and many others prefer to think about Virtual/Augmented/Mixed/etc. Reality experiences — anything where we can interact with digital objects in three dimensional space, as if they are actually in front of us. Sometimes that means you are completely transported to a new world like in VR, sometimes those digital objects appear next to you in the real world as in AR, but fundamentally the UX paradigm is the same: computing will no longer be on a 2D screen but all around us.

What follows are my notes covering most everything I saw at MWC. These notes are arranged into general categories of tech seen on the expo floor, but first I’ll cover the conference portion — each day of the show there were multiple talks and panels with everyone from major leaders in these industries to startup founders to AI researchers to marketing agencies general counsels, to teenagers building real businesses.

Conference Keynotes

The first thing I did at the event was head to HTC’s keynote: their Chairwoman Cher Wang explained their mission designed around the 3 sides of their triangular logo: humanity, technology, and imagination. HTC is obviously big on VR and had some newer Vive hardware to show off, so she shared stories of rural Chinese children using Vive equipment in school, and showed some concept videos around how these kids might grow up in a spatial-first world. She also announced their 5G Hub hotspot device.

Microsoft CEO Satya Nadella spoke a lot about company culture and how they’ve steered Microsoft into focusing on services, spatial computing, AI, etc. My main takeaway was what he described as the 3 leadership signals that he looks for on his teams. Great leaders usually:

1. Bring clarity into a situation where none exists.
2. Create energy.
3. Drive success in over-constrained problems.

The last keynote I was able to attend was ostensibly about ARM, but they had a panel of 4 awesome kid founders! These younger generations are building the future they want to live in, not waiting for adults to do anything for them. Very inspiring, and also pretty humbling.

Conference Sessions

There was of course lots of discussion on how to leverage 5G for various industries. But it was interesting to hear from media, content, and consumer-focused companies on things to think about. The slides above are pretty good summaries of the main talking points.

Also lots of technical discussion on how to think about a 5G world and what it will enable. Super-low latency is going to be the main benefit of 5G, but also threaded frequencies will mean lots of parallels bands on the same frequencies, all with full speeds.

One of the best examples of this I heard was about a live event; a major bike race in Europe. Drones filming the event with control & the video streams, event operations & video broadcasting, and attendee wireless networks can all run on the same network with effectively zero latency and gigabit speeds.

Thin clients (workstations with all compute in the cloud) are a prime use case of this so they came up a lot, as did streaming 4k/VR multiplayer games. But it was pretty clear that it was all aspirational: these things *seem* like the obvious use cases from what we see now, but ultimately products and services will have to be built and grow organically based on customer value.

We also heard about AI impacting enterprise work and security/privacy, gamification and physical experiences, etc.

Cloud XR Summit

“Spatial Computing” is officially a thing, and a major focus for the big players — in enterprise and as part of the 5G pitch. This ‘Summit’ was a special session focused on how these players are building a cloud for these experiences:

• Microsoft spoke mostly about the Hololens 2 in Enterprise, but shared a lot about how they think about this next era of computing. Their vision & focus going forward is Mixed Reality, Artificial Intelligence, and Quantum Computing, though there wasn’t much of anything to talk about for the latter. They are seeing real results across many industries, and are working to enable it everywhere, from the cloud with Azure and 5G networks to the ‘edge’ with the actual hardware. And they aren’t just talking about mixed reality headsets like Hololens, but AI perception — computer vision and speech recognition — as well.
• Huawei similarly talked about supporting the full networking lifecycle, mostly via cloud computing infrastructure streaming everything to whatever device you want. The performance of the end device won’t matter when you can stream 4K multiplayer experiences with no latency.
• Intel touched on a lot of the same points, but also talked about the actual experiences that will come from this tech on a spectrum between passive and interactive, across different scales. So not just interactive single-player experiences, but more dynamic and smarter experiences across massively multiplayer scales.

Between and after the conference sessions we explored the numerous and massive expo show floors. There were thousands of different vendors across 6 massive conference halls, ranging from a single tiny desk pitching some new app, to enormous multi-level booths with demos and meeting rooms and auditorium seating and much, much more. Here are my highlights and takeaways.

AR Hardware

Hololens 2 was obviously the biggest news and most exciting hardware demo (that I wasn’t able to try due to 2+ hour waits!), but Qualcomm showed off some promising sunglasses made by nReal with a screen/cameras “hidden” in the lens (powered by their new Snapdragon processor of course), and LetinAR showed off amazing lenses with tiny mirrors reflecting microdisplays into your vision. And more vendors are already on the way.

The tech is here for very lightweight glasses tethered to another device for compute power (or eventually streamed over 5g?), and in the same week of the event Apple was granted some very interesting patents — they are easily 2 years ahead of Qualcomm on these chipsets. I would bet Apple will announce their glasses in 2019, shipping early 2020.

AR Software

Microsoft (and many others) are very clearly focused on enterprise applications as a way to get these early devices into the market and make sure developers start working on these new platforms while the light-weight consumer versions are still to come. So spatial development/UX is flourishing.

For consumers, the UX is currently focused on camera-as-UI, since users need a viewfinder via their phones. Google Lens was the biggest example of this at MWC: point your phone at something, Google understands what it is and what kind of information you likely need on it. Examples in Google’s large experience were pointing at art in a gallery (learn about the artist, explore super-high res scans of the painting, see it come to life in AR), records on a wall (pull it up on Google Music/Play, get the lyrics, etc), and even text in a book (highlight and translate it in realtime).

Other examples were a virtual price tag ‘attaching’ to physical boxes in a store demo, a multiplayer AR ‘dodgeball’ game made by Niantic, and an SDK for mapping and recreating the world around you in 3D using the depth data iPhones already gather. People are no longer just mobile-first or mobile-only: they are increasingly camera-first, so having a better understanding of what the camera is looking at is vital.

VR: Standalone, Eye Tracking, Retina

While MWC is focused on mobile network connectivity and devices, there was a lot of VR technology as well, usually running some kind of streamed experience via 5G. But there was some new and/or low cost tech of note:

• HTC had several standalone headsets for demos, which is a game changer for ease of use and adoption (no cumbersome cables or gaming PCs required). They also combined these with their 5G hub to stream the games.
• I noticed some low-cost add-ons that turn 3dof headsets into 6dof, which would help users upgrade their experience without breaking the bank.
• There were also a few different eye-tracking solutions, which when done well make it so much more intuitive to navigate UI (especially in higher-resolution headsets, like the “Retina” Varjo headset in the 2nd pic. They added 1080p microdisplays and a mirror system to make the middle of the picture much sharper than the peripheral. This kind of pixel density will come down in price, enabling 8k+ displays in a couple years).

Projection Mapping

This is a slight aside, but there were several booths/demos of some really interesting projection mapping that I wanted to mention.

• One booth had a consumer system the size of a small lamp: the projector and camera system hang from the ceiling to detect what is on the table and project a game on top. Lots of interesting potential depending on how smart the camera system can get.
• A fairly large model of a port city came to life by projecting imagery, movement, and other information onto the model.
• Notifications and other information projected onto the wall of a bedroom

Motion Capture

All these camera systems detecting depth are able to use that extra data to do motion capture — translating a real persons movements to virtual characters. There were standalone systems like Microsoft’s Azure Kinect, and component suppliers selling tiny, phone-sized systems. Several of the big guys (Samsung, LG, Huawei) showed off these cameras in their consumer products as well.

The camera is arguably the most important part of a phone these days, and they are advancing extremely fast in hardware and software into computers that see and understand the world. And sometimes translate that into digital assets or experiences.

5G Live Demos

A lot of show floor space was of course devoted to 5G, showing off potential use cases of all types. There were tons of autonomous systems, industrial applications, etc, but also some fun and interesting consumer experiences. As mentioned earlier multiplayer 4k/VR gaming, and lots of ways to try to get people to stream many HD videos at once: choose 3 out of 6 camera angles of a live concert, bullet-time scrolling around a live stream, and new file formats like 3d animations/objects/videos.

This stuff all looks super cool and exciting, but the reality is no one really knows what’s going to actually happen. Usage will eventually expand to take advantage of all the extra capacity/latency, but that doesn’t mean we know what that usage will look like. But it does seem like once 5G really rolls out at mainstream scales over the next couple of years there will be a lot of exciting new things possible.

Robots

Because who doesn’t love robots? Powered by 5G of course.

Oh yeah, and Phones

Folding phones got the most obvious press, but lots of interesting other stuff:

• More/better cameras were everywhere: depth cameras, wide angle lenses, notches and pop ups, filters, bokeh tricks, etc. Nokia had a 5-lens system that will be DOA when software is already better at the tricks that enables. Depth and wide angle are very interesting, especially related to the AR progress mentioned above.
• Smartwatches were everywhere, with full connectivity for video calling etc. Lots of kids models and outdoorsy ones, and a ridiculous folding wraparound screen one.
• Screens are getting crazy good. Sony showed off a 4K, 21:9 screen phone. That’s stupid high for a phone, but would be awesome in a headset.
• Niche/concept phones: the supply chain is out in full-force, trying different things with all the same basic components. For example phones with a second screen on the back for better selfies, or weird pop-up cameras, but Energizer won the viral prize with their 20+ new phones all with massive batteries: from 4k mAh to 18k! (iPhones are less than 3k). Obviously a powerbank with a phone on it isn’t a realistic product, but as a stunt it’s pretty brilliant.
• Back to the folding phones: they were all behind glass, so there are clearly major concerns about reliability and construction quality. Samsung and Huawei will probably actually sell them, but it’s too early for these to be actual products people can use. Several other vendors had very clearly slapped together fake phone cases around their bendable screens to look like it’s ready.

And everything else

Some cool other tidbits:

• Awesome concept cars. Mercedes, BMW, Audi all had electric concepts on display and various experiences promoting what they can do with 5G. BMW specifically had a private ‘Mixed Reality’ experience where you get to drive their electric autonomous concept in a future city in VR, but in a physical room with a car seat, steering wheel and pedals, and more. Very cool.
• Two different flying taxi VR experiences, where you literally sit in the helicopter-like concept vehicle and use VR to experience flying around a city.
• Other weird tech like a multi-touch e-ink keyboard/tablet on a laptop, transparent TV panels, tons of niche phone add-ons and bluetooth/qi accessories, 360 cameras, smart home devices, and so much more.

It was incredibly obvious through the entire event that the smartphone market is very mature, and everyone is trying to figure out what the Next Big Thing will be. 5G will enable new products and experiences that were never possible before, but it’s so early everyone is taking wild guesses at what those might be. Some Next Big Things are further along than others, though, so I came away with unanswered questions:

• What will people actually do with ultra-high wireless speeds and ultra-low latency?
• How will we interact with computers when they can see and understand everything around us?
• Will spatial computing be compelling enough to consumers to drive adoption of both 5G networking and new hardware devices like headsets or glasses? And if so, what will those compelling experiences be, and what will they do to existing businesses/cultures?
• Or will the fad subside and people only care about better/faster slabs of glass in their pockets?

Thanks for reading!