Nvidia is applying its RTX graphics card’s artificial intelligence smarts to remove the need for streamers’ green screens. Thanks to the Tensor Cores within the Turing architecture, Nvidia is reportedly able to ditch green screens altogether and let its AI figure out where you end and the background begins.
There are actually three new SDKs being introduced by Nvidia in the run up to Twitchcon, as a part of its newly-announced RTX Broadcast Engine: RTX Greenscreen, RTX AR, and RTX Style Filters. The suite of features will soon be available to developers, who can sign up for access today, with a public release within popular streaming apps expected at a later date – popular open-source streaming app OBS is already signed up.
RTX Greenscreen will utilise the clusters of Tensor Cores found within RTX 20-series graphics cards to remove backgrounds in real-time, leaving just your mug on-screen. Nvidia is planning on demonstrating RTX Greenscreen in OBS at TwitchCon, which runs from the 27-29 September, 2019.
Whether the total number of Tensor Cores within a GPU will change its effectiveness, we don’t yet know. Tensor Core counts vary all the way from 240 within the RTX 2060’s TU106 GPU to 544 within the RTX 2080 Ti’s TU102 chip.
Also leveraging those cores is RTX AR, a facial tracking feature that can apply augmented reality effects in real-time.
And lastly, Nvidia is rolling out RTX Style Filters. These will transform the image from your webcam into the style of another image. You pick an image, load it in, and the clever AI algorithm will make you look one and the same. A Rembrandt perhaps?
This is similar to a technique Nvidia implemented with its GuaGAN AI painting app – which you can try for yourself here – that also utilises reference images to change the style and tone of images.
“The new RTX Broadcast Engine is an exciting advancement that will allow developers in our app store to create powerful new tools for streamers with NVIDIA RTX GPUs,” Ali Moiz, CEO of the now Logitech subsidiary Streamlabs, says. “We’re thrilled to continue working with NVIDIA as they introduce new features to the Streamlabs developer community, and look forward to implementing this new technology.”