• Versioning scheme
• Simulators instead of emulators
• VS Code instead of deprecated IDE
• Chromium 87
• No new formats / codecs supported

The LG webOS operating system (OS) versioning scheme is now using years. The previous OS version introduced in 2021 was called webOS TV 6.0, but the new 2022 version is called webOS TV 22.

And, even though the new OS is called "webOS TV 22", its SDK is called 7.X.X.

webOS TV 22 no longer supports emulators, and relies on simulators instead. Note that simulators don't support DRM, and that the audio and video specifications differ from real devices. Additionally, some native APIs aren't support on simulators.

The webOS TV IDE is no longer supported, and the VS Code extension should be used instead.

This version of webOS uses Chromium 87 instead of Chromium 79. The new user agent is listed below:

Mozilla/5.0 (Web0S; Linux/SmartTV) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36 WebAppManager

(You might want to check the relevant Chromium release notes.)

When comparing supported media formats between webOS 22 and webOS 6.0, it seems that no support for new codecs or formats has been added to webOS 22. The same statement is true for MSE and EME specifications, or DRM systems.

Interested in the changes visible to consumers? Check the links below.

• https://www.techweekmag.com/news/tv/webos-22-and-how-it-differs-from-webos-60/
• https://www.gearrice.com/update/webos-22-proposes-big-changes-to-lgs-new-oleds-and-qneds/
• https://beebom.com/lg-new-smart-tv-lineup-webos-22-ces-2022/

Google I/O is Google's yearly developer conference. If you're a video streaming enthusiast, I'd recommend you to spend 30 minutes watching the presentations on "What's new with Android TV and Google TV" and "What's new in Android media". If you can spare two hours, you could watch the keynote – or skim a recap.

The next two sections outline these first two presentations. Click the timecode in "the timeline" (below the video) to be taken to that part of the presentation. (You might need to click the Play-button again on some devices to resume video playback.)

What's new with Android TV and Google TV (in 2022)

This talk focuses on reviewing their recent progress, recapping Android 12, and highlighting the (upcoming) Android 13 updates.

The timeline:

• 02:23: Majority of the newly activated devices run Google TV instead of Android TV. Remember: Google TV is the successor of Android TV.
• 02:40: Profiles & Kids profiles (personalized)
• 04:49: Virtual remote (with the Google TV app)
• 05:30: Cast from Google TV app (which aggregates content)
• 07:58: Android 12 recap (framerate switching; 4K UI; text scaling; kids profiles)
• 09:00: Android 13 (performance & quality; input & accessibility; multitasking)
  • 09:12: AudioManager (useful to find best audio format for the active device)
  • 11:09: HDMI state changes (e.g. pause content if HDMI device is no longer the active source)
  • 11:26: Keyboard layouts
  • 12:08: Audio description (system-wide preference that allows users to enable audio descriptions across apps through AccessibilityManager API)
  • 12:45: Picture-in-Picture (including expanded PiP and Keep-clear API)
  • 16:18: Media Controller Test
  • 16:45: MediaSession Validator

What's new in Android media

Media3 is Android's collection of libraries related to media playback, including ExoPlayer. This talk focuses on the new Android 13 capabilities, as well as updates to ExoPlayer.

The timeline:

• 00:50: New platform capabilities in Android 13
  • 01:06: HDR video (for use cases related to editing)
  • 04:52: Spatial Audio through the Spatializer Class
    • ExoPlayer 2.17 has support for spatial audio
  • 07:20: MIDI 2.0
• 07:48: Media3 & ExoPlayer
  • 09:01: New features in ExoPlayer
  • 09:43: Performance Class and Jetpack Core Performance Library
  • 11:55: Using AV1 for content delivery

That's it. That's the post. 🏳

Krishna Rao Vijayanagar (Founder at OTTVerse) shares his 2021 thoughts on The Fight for Eyeballs.

The final bit that interests me is seeing how streaming companies grow and retain their subscriber base.

Are they going to invest heavily in original programming?

Or, ink better content licensing deals?

Niche-down to a specific genre or sport?

Or, are they going to build immersive experiences using a second screen (phone/tablet)?

And let's end the 2021 series on this philosophical note... :-)

Krishna Rao Vijayanagar (Founder at OTTVerse) shares his 2021 thoughts on Ad Tech.

The second thing that I am eagerly tracking is how the Ad Industry will evolve with the phasing out of 3rd party cookies. How will this impact data collection, data quality, and advertisers' ability to target prospective customers?

My take

Companies like Firefox and Apple are going all-in on privacy to protect their users. Firefox even owns https://trackthis.link/ which is a website tool which opens 100 random websites to confuse trackers.

While this is applaudable, it does give the ad industry less ammo to tailor ads.

Should we care about this?

Yes, but with certain duality.

1. Privacy is important. There's no question about it.
2. If "less-than-ideal-ad-personalization" implies that less viewers will click ads, then the profitability of ad revenue goes down, which means that ad-supported businesses become less profitable – potentially even going bankrupt. (And, yes, that includes ad-supported streaming services.)

🤷‍♂️

Krishna Rao Vijayanagar (Founder at OTTVerse) shares his 2021 thoughts on the Codec Wars. 👇

Due to the COVID-19 pandemic and people being stuck indoors, the number of hours spent watching videos has surged. Adults and kids alike, are consuming a lot more video than before, and, this provides a situation ripe for innovation and some bold moves.

So, what do I find most exciting and what am I looking forward to in 2021?

The Codec Wars.

With the decade-old H.264/AVC still ruling the roost, HEVC's patent problems, AV1 slow and steady progress, and three exciting new codecs (EVC, LCEVC, VVC) coming through, it is going to be amazing to watch how the compression landscape evolves in 2021 and beyond!

Context

The "Codec Wars" is about the battle of supremacy among the video codecs. When comparing video codecs, people usually like to compare the following parameters:

• Compression efficiency: which one is objectively better than the other one?
• Device support: which devices support the video codec Today?
• Royalties and costs: what's the patent pool and licensing situation like? What are the encoding costs like?
• Complexity (for encoding and decoding): how difficult is it to build an efficient encoder and decoder?

Searching Google for "Codec Wars" gives you a couple of cool hits.

The oldest mention I could find dates back to the 4th of August in 2004 at http://biostatisticien.eu/www.searchlores.org/destream.htm. This article is more about the War On Codecs than the War Between Codecs though. :-)

Another one from 2006 reads:

It participates in geopolitical codec wars (e.g., China’s AVC codec, versus the increasingly popular H.264, versus other versions such as Microsoft Windows VC- 1— Windows Media 9).

Another fun one is at https://bloggeek.me/av1-vs-hevc-webrtc-codec/.

And, finally, let's not forget about Krishna's article at https://ottverse.com/vvc-evc-lcevc-mpeg-video-codecs/. 🙌

Negar Hajihoseini (Innovation Engineer at THEO Technologies) shares her 2021 thoughts on how low latency is becoming available to anyone through cheap, scalable HTTP technology. 👇

2020 ended, but the Covid-19 era is not completely gone yet. More than 80% of the organizations worldwide have encouraged - or required - employees to work from home due to Covid-19 pandemic.

Almost all academic and business conferences around the world were held online – if not postponed to the post-Covid time.

Online Streaming is playing a main role in the educations at all levels and forms. Yes, in 2021, Online Streaming is not limited to sports and entertainment events anymore. A lot of educational and professional events will be streamed online. Thanks to ultra-low latency HTTP streaming protocols, saying goodbye to cable and antennas is now closer than it has ever been.

With such an ultra-low latency HTTP Streaming protocol, video is split further than segments to CMAF (Common Media Application Format) chunks. This allows an encoder to push small video chunks of few MegaBytes rather than large video segments, making possible the delivery of content in almost real time over DSL networks.

CDN companies are adapting their network with innovative caching methods to handle the ultra-low latency delivery of content to their customers all around the world. It is now possible to cache as small as byte ranges of an object to make sure that the content is delivered with the least possible latency.

Scalable HTTP adaptive streaming based protocols remain cheap in scale, efficient in bandwidth usage, and open a door to an exciting future for the online streaming industry!

(One example of such a low latency HTTP streaming protocol leveraging CMAF is LL-DASH. More info can be found at https://www.theoplayer.com/blog/low-latency-dash and https://bitmovin.com/live-low-latency-streaming-p2/.)

Adriaan Bloem (Head of Digital Infrastructure at MBC) shares his 2021 thoughts on how our job will never be done. 👇

We've made great strides in the past few years in terms of what we can deliver to our users in terms of video streams. Things that seemed almost impossible to achieve not that long ago, unless you had your own huge team of rocket scientists to build it, became productized and available as libraries, systems and cloud services. Server side ad insertion is doable. HD and 4K streams are feasible. Streaming it out to a plethora of clients in the required different streaming formats and DRM using just in time packaging is possible. Audio track selection and subtitles are supported by the players. Live streams can have low latency.

Until you have to do all of these at once. That's still an impossible combination of technologies, where multiple features exclude each other and you hit several roadblocks. Every time something hard becomes doable, we can have a two second breath of air, and then our life gets difficult again. "Just when I thought I was out, they pull me back in!" This is both the frustrating and the exciting thing about working in streaming media; every time we standardize and simplify one thing, the complexity explodes again somewhere else, and 2021 will be no different.

(My view on this? => amen. ‍🙏 )

Nicolas Weil (Product Manager at AWS Elemental) shares his 2021 thoughts on metadata in video/media streams. Keep reading to get 1) context and 2) my take.

2020 brought an interesting set of new specifications for CMAF, with AOM's ID3 Timed Metadata in the Common Media Application Format and MISB's ST1910 for the carriage of KLV Metadata over CMAF, opening new perspectives for building data-intensive video workflows through Event Message Boxes.

This is however not the end of the road, as MPEG is working on new specifications like the CMAF Timed Metadata Tracks, the Event Message track format (MPEG-B part 18), with a corresponding Event Stream and Timed Metadata Processing model in the core DASH spec.

While all these new specs create interesting perspectives in terms of bandwidth optimization (by centralizing all the events into a dedicated track instead of duplicating it into all audio/video tracks), it also creates an additional level of implementation complexity for packager and players, with the nested timelines of the Timed Metadata Track and the contained emsg boxes.

So far we have seen a very limited industry support for the Timed Metadata Track in the context of the new CMAF-based Ingest Protocol developed by DASH-IF/MPEG. It will be interesting to see in 2021 if the ecosystem finds enough benefits in the new set of specs to cope with the associated complexity, for ingest or player-centric use cases.

Metadata is the new fronteer, let's be trailblazers. For more information on KLV metadata in streaming workflows, please check this blog post by Matt Carter: https://aws.amazon.com/blogs/media/misb-st-1910-and-klv-changing-the-game-in-intelligence-entertainment-and-beyond/

Context

We already wrote about the Common Media Application Format (CMAF), but it should be added that "CMAF containers" can also contain timed metadata. An example of timed metadata would be a signal (coming from the container) signalling that an ad break starts at second 15 and stops at second 45.

There are (generally speaking) two types of timed metadata:

1. In-band timed metadata which is contained by a container, for example ID3 in the TS containers of an HLS stream, or Event Message (=emsg) in CMAF containers.
2. Out-of-band timed metadata which is contained outside of the container, for example the timed metadata in the manifest of an HLS (=> #EXT-X-DATERANGE) or MPEG-DASH (=> EventStream) stream.

Note that you typically have "faster access" to out-of-band timed metadata, because you first fetch and parse your HLS/DASH manifest (and acquire the out-of-band timed metadata). Only after processing your manifest, you start downloading  (and processing) your segments (=containers) which contain in-band timed metadata.

Nicolas is explaining that two new types of in-band metadata are available:

1. ID3 metadata in CMAF containers.
   The benefit of this new type ... .
   The added challenge is ... .
2. KLV metadata in CMAF containers. The benefit of this new type is ... . The added challenge is ... .

Additionally, he's mentioning that 3 new types are in the pipeline:

1. CMAF Timed Metadata Tracks. This is a new type of in-band timed metadata.
   The benefit of this new type is ... .
   The added challenge is ... .
2. Event Message track format (MPEG-B part 18), with a corresponding Event Stream. This new type relates to both in-band and out-of-band timed metadata, as "Event Message" is in-band timed metadata in CMAF containers, and Event Stream is out-of-band timed metadata in MPEG-DASH manifests.
   The benefit of this new type is ... .
   The added challenge is ... .
3. Timed Metadata Processing model. This is ... ?
   The benefit of this new type is ... .
   The added challenge is ... .

My take

LG first announced this 2021 edition on the 7th of January 2021 at https://lg.com/us/press-release/lgs-webos-6-0-smart-tv-platform-designed-for-how-viewers-consume-content-today.

The home screen got a big redesign, and the TV comes with a "Magic Remote" with voice recognition.

An OLED TV branded the "LG SIGNATURE OLED R 65'' Class Rollable 4K Smart TV w/ AI ThinQ® (64.5'' Diag)" is the most expensive 6.0 model priced at $99,999.99. A 4K TV branded the "LG 49 inch Class 4K Smart UHD TV (48.5'' Diag)" is the cheapest 6.0 model priced at $379.99.

So what's new for webOS app developers? In a nutshell: not much, but keep reading below.👇

Chromium 79

webOS TV applications use Chromium (i.e. Chrome) as their development framework. This means that web app developers use HTML, CSS and JavaScript to develop webOS TV applications, as well as the "native" Chromium APIs.

webOS TV 6.0 uses Chromium 79. The 2020 model (5.0) was using Chromium 68.

There are no major updates hidden in these patches, which is similar to what we wrote in our Tizen 6.0 article. Things. Regardless – feel free to skim through Chrome's "New in Chrome" articles listed below.

The new user agent is:

Mozilla/5.0 (Web0S; Linux/SmartTV) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.79 Safari/537.36 WebAppManager

webOS TV SDK 6

The release notes for the webOS TV SDK 6.0.0 were published to https://webostv.developer.lge.com/sdk/installation/release-notes/ in May 2021.

MSE and EME specification

Media Source Extensions (MSE) and Encrypted Media Extensions (EME) are the browser APIs that web developers use to implement video and DRM playback.

webOS 6 uses the MSE W3C Recommendation 17 November 2016 and EME W3C Recommendation 18 September 2017, which is also what webOS 5 was using. These are the latest "stable" specifications, but there are more recent "editor drafts".

Audio and Video formats

The AV formats for webOS 6.0 TV are documented at https://webostv.developer.lge.com/discover/specifications/supported-media-formats/webos-60/.

There are no changes compared to 5.0, except for some notes on the "8K Ultra HD" model. The 6.0 "8K Ultra HD" model seems less "advanced". It's missing support for VP9, and AV1 is limited to 40Mbps instead of 50Mbps. (However: there are also some models that do support VP9 for the 6.0 model, similar to how the 5.0 model has some models that lack VP9 support.)

Design

There are new resources related to the design at https://webostv.developer.lge.com/design/webos-tv-system-ui/home/ and https://webostv.developer.lge.com/design/style/icon/.

Andrea Fassina (Senior Software Engineer at Sky Italia) talks about video summarization in his 2021 prediction.

I think an interesting trends for 2021 is media/video summarization. For example,
- video + text analysis for detecting offensive memes on FB
- AI in detecting <13 year olds account on social networks, through computer vision and age estimation

According to Wei et al, "Video summarization is the process of distilling a raw video into a more compact form without losing much information". Surya Remanan also references this quote on her blog post on Deep learning-based video summarization.

So, it looks like video summarization is a popular research field. There are 150+ hits on Google Scholar. An Adobe blog post on Leveraging AI and Deep Learning for Video Summarization is a good read. Andrea also links a recent blog post by AWS on "Saving time with personalized videos using AWS machine learning".

My personal interest in video summarization would be related to recommendation systems, and how it can be used to enhance related search results. Additionally, like Andrea said, it would also be useful to detect and flag (offensive) content.

Nick Krzemienski gives us a couple of 2021 takes, more specifically about CMAF and CBCS, about Vod2Live, and about SDR-to-HDR.

CMAF. Will this actually be the year organizations can finally stop using CENC (CTR) and only CBCS? I’ve done the math, and we’d only loose about 3% of customer device support as the year progresses, so I plan to finally drop CENC (CTR) on dash.

So now we know Nick's view on our article on CMAF and CBCS!

Vod2live. When channel X is playing a re-run of episode Y and you also carry the VOD  for Y, you can shutdown your linear encoding, and point customers to the static VOD content. I’ve begun exploring this project, but there are ad restrictions from content owners. That said, I believe we will see a serious trend of content owners themselves leading the way with these kind of efforts.

I see the value in this, and this is something people have been doing in the past. I'm curious about the ad restrictions though!

SDR-to-HDR upscale. Personally, I believe that this is hands down the most useful way to increase a customers perceived video quality. With the costs of HDR-enabled screens being extremely reasonable now (latest series 5 50in TLC 4K RokuTV w/ DoVi & HDR is only $399), this year could definitely be the time to explore what you could do for your customers experience.

This is probably a research field I'd devote myself in if I had the intellectual capacity. SDR-to-HDR algorithms not only enhance the video quality, but they would also be super valuable for upscaling archived (read: old) videos. Throw in some Machine Learning and AI, and do this at run-time on the client-side, and you got yourself the future of video streaming.

Magnus Svensson (VP at Eyevinn Technology) expects that local broadcasters and distributors will band together – with ads  Keep reading to get 1) context and 2) my take.

The year 2021 will be the deciding year for the local broadcasters and distributors. Cooperation and mergers will be the best way to survive together with an increased focus on local content. With giants as Netflix and Disney already established and HBO Max, Discovery+, and Paramount+ growing globally, the subscription budget for the consumers are filling up. The lean-back TV experience will be fulfilled as ad-supported services such as the FASTs together with the device and TV manufacturers. The free ad-supported TV services with virtual channels or playlists will grow globally and become a good replacement for traditional TV services.

So, the local broadcasters and distributors must cooperate on the technical and infrastructure side and instead focus on the local content that still will be attractive for consumers. In the same way as the global media companies have been going all-in with the shift towards streaming, the local broadcasters and distributors must go all-in to adapt to the changing environment and find ways to attract Gen Z.

Context

In case you didn't know:

• What is FAST? Free Ad-supported Streaming TV. (Streaming services with ads!)
• What is a lean-back TV experience? A viewing experience where you "sit in your couch" and lean back to consume the content.

My take

Local broadcasters and distributors have a hard time competing with the content inventory, originals and appeal of the Hulu's and Netflixes. Collaboration sounds like the logical move for these "traditional", local companies.

This kind of collaboration between local companies is already happening. For example, in Belgium (shoutout to my home country!), Streamz is the joint effort of a local telco and local media company.  Joyn, the most popular free, German streaming service, is the joint effort of a local media company and Discovery. They have the bundle (and create) local content to target their geographic market. Additionally, they also need to be aware of the audience of streamings services, and tailor their catalog.

The average American has 3 streaming service subscriptions. If you're already paying for Netflix, Disney+ and Amazon Prime, you most likely don't want to pay an additional $10/month for a local streaming service. Hence, these local streaming services need to figure out how to maximize their advertising model.

Collaboration is not easy, and ad-monetization isn't easy, but they are most likely necessary for local companies to survive in a global internet.

Shane Afsar (Engineering Manager at LinkedIn) got us excited about low-level media access in 2021.

In 2021, there’s going to be a proliferation of products and tools that will help people easily create and stream media.

Any technology that makes media creation more accessible and approachable will allow developers to focus more on what makes their product unique. I’m excited about libraries that provide turn-key low-level media access, like ffmpeg.wasm (https://github.com/ffmpegwasm/ffmpeg.wasm), LiTr (https://github.com/linkedin/LiTr), and newer browser APIs like the WebCodecs API (https://web.dev/webcodecs/).

Keep reading to get 1) context and 2) my take.

Context

Having "low-level access" means "having access to the (deepest) nuts and bolts of the system" – instead of touching the surface.

Currently, video players outsource a lot of the complexity to the platform. Typically, when software engineers build (or interact with) a video player, they feed "tiny videos" into (the media buffer of) the platform. This platform (e.g. a browser, a smartphone) renders the videos (in its buffer). Don't get me wrong: doing this correctly, across different platforms, is challenging.

However, sometimes, you don't wanna outsource the complexity to your platform. For example, having access to low-level media access may allow you to workaround bugs of the platform (and its "native rendering pipeline"). Alternatively, maybe you want to implement something which isn't yet supported by the platform.

Low-Level Media Access projects, like the ones linked by Asfar, allow us to challenge the platform, and do things which are normally not possible. And that's awesome.

• https://github.com/ffmpegwasm/ffmpeg.wasm allows you to run FFmpeg (the most popular software encoder/decoder) directly in the browser.
• https://github.com/linkedin/LiTr offers hardware-accelerated audio and video transcoding for Android.
• https://web.dev/webcodecs/ allows you to drill down into the frames in the browser.

My take

I love these type of projects – they really bring power to the people. ✊

That being said, when you switch from "medium-level media access" to "low-level media access", you might get more than you bargained for. Maintaining and working with these APIs isn't for everyone.

I'd only leverage low-level media access when you really, really, really need it. And I think that's exactly the reason why having access to a low-level media API should be a possibility. Because you might just really, really, really need it for your use-case.

Anna-Katharina Reykowski (Product Manager at Verizon Media Platform) gives her 2021 take on Watch Together and Pre-Integrations. Keep reading to get 1) context and 2) my take.

Stay-at-home orders across the globe not only increased the consumption of video streaming hours but also left many people craving social interaction, looking for new ways to interact and share experiences with their friends and family.

These converging trends created both a boost for Connected TVs as well as a strong demand for interactivity within video streams and features such as virtual “watch parties” like Verizon Media’s Watch Together. While I am hopeful that COVID-19’s grip will steadily loosen as this new year progresses, I believe that demand for these  new interactive features will remain. Content providers are thus facing the daunting task of building out support for new interactivity on a growing number of platforms that may rely on a variety of player implementations, turning feature development into a cumbersome, lengthy process.

To tackle these challenges, I expect that we will see a strong push this year to standardize how video players and underlying streaming services interact and exchange data with one another. Achieving simple, repeatable implementations will be key to faster time to market and increased reach for the ever growing number of viewers across the globe.

Context

Let's explain Connected TVs, Watch Together, and Pre-Integrations.

• Connected TV (CTV) and Over-the-Top (OTT) are closely related. Connected TV devices are your Smart TVs, your streaming sticks (e.g. Roku, Chromecast, Apple TV) and your gaming consoles which you use to consume OTT video streaming apps. (A laptop or a smartphone isn't a CTV device, because you typically also use those for other purposes than entertainment/streaming.)
• Watch Together (or Watch Parties) is an OTT experience where a group of people watch an OTT stream in sync. Anna's company, Verizon Media, has a Watch Together use-case with Yahoo Sports.
• Pre-Integrations are integrations between two (or more) components/products/services. Video streaming services combine many different components, and connecting the dots isn't always as easy. If you can leverage existing pre-integrations between components/products/services, you can speed up your time-to-market (and benefit from them maintaining it).

My take

There's an audience for Watch Parties, but I have three "problems" with this feature.

1. Few streaming services offers it. (Boo!)
2. I don't know which of my friends want to join my party. (I know, that sounds pretty sad.) For example, I've got Disney+, which offers GroupWatch, but I don't know which friends have Disney+, and I don't know which of them share my taste.
3. Not all types of (experiencing) content are suited for Watch Parties. A football game in the weekend – yes. 30' of an episode I'm binging on a Monday – no.

I also agree that the industry needs pre-integrations to make these kind of products work at scale.

When you're implementing Watch Together for a sports game, you're working on 1) low latency, 2) metadata insertion, 3) synchronization, 4) cross-platform, 5) discoverability, 6) UX/UI, ... A couple of pre-integrations would definitely come in handy – not every streaming service needs to reinvent the wheel.