This article is part 12 of the "21 Thoughts on Video Streaming in 2021"-series.

Shane Afsar (Engineering Manager at LinkedIn) got us excited about low-level media access in 2021.

In 2021, there’s going to be a proliferation of products and tools that will help people easily create and stream media.

Any technology that makes media creation more accessible and approachable will allow developers to focus more on what makes their product unique. I’m excited about libraries that provide turn-key low-level media access, like ffmpeg.wasm (https://github.com/ffmpegwasm/ffmpeg.wasm), LiTr (https://github.com/linkedin/LiTr), and newer browser APIs like the WebCodecs API (https://web.dev/webcodecs/).

Keep reading to get 1) context and 2) my take.

Context

Having "low-level access" means "having access to the (deepest) nuts and bolts of the system" – instead of touching the surface.

Currently, video players outsource a lot of the complexity to the platform. Typically, when software engineers build (or interact with) a video player, they feed "tiny videos" into (the media buffer of) the platform. This platform (e.g. a browser, a smartphone) renders the videos (in its buffer). Don't get me wrong: doing this correctly, across different platforms, is challenging.

However, sometimes, you don't wanna outsource the complexity to your platform. For example, having access to low-level media access may allow you to workaround bugs of the platform (and its "native rendering pipeline"). Alternatively, maybe you want to implement something which isn't yet supported by the platform.

Low-Level Media Access projects, like the ones linked by Asfar, allow us to challenge the platform, and do things which are normally not possible. And that's awesome.

My take

I love these type of projects – they really bring power to the people. ✊

That being said, when you switch from "medium-level media access" to "low-level media access", you might get more than you bargained for. Maintaining and working with these APIs isn't for everyone.

I'd only leverage low-level media access when you really, really, really need it. And I think that's exactly the reason why having access to a low-level media API should be a possibility. Because you might just really, really, really need it for your use-case.