Highlights of Twitch streams, or Clips as Twitch decided they should be called when they introduced their own way of creating them earlier this year, are quite popular. Used for everything from identifying cheaters at a card game tournament to celebrating a particularly sweet Counter-Strike ace, they’re pretty useful. Now a company, Videogami.tv, are trying to see if they can automatically generate highlights based on algorithms that analyse everything from the speed of chat to the streamer’s tone of voice.
Related: Try the highlights of PC gaming past and present.
“Our company got started a year ago doing something similar to an advanced version of [third party clipping tool] Oddshot” says Slava Sakhnenko, part of the team working on Videogami. “Through early conversations with some Twitch folks we could see the writing on the wall and that soon all those Clipping features would become native to the platform. So instead we focused on our R&D to develop AI bots that can ‘watch’ eSports streams and identify, rate and clip interesting moments automatically.”
It does this by analysing three key things:
- Audio. An audio filter monitors “certain frequencies associated with human speech, shouting or audience cheers, while filtering out other ambient noise.” If things sound like they’re reaching a peak, the software makes a note.
- Video. They’ve developed software that can tell the difference between a pause screen, people talking in an interview (or presumably a shot of a caster desk) and actual gameplay. They’re further developing it to be able to recognise some text, such as scoreboards.
- Social. As you might expect, this “monitors the chat for patterns in conversations that go beyond the normal ebb and flow of discussions on that channel, to identify which moments are the most talked about.” Rather a lot of Kreygasm? Probably a highlight.
If you hook it up to your account, anyone can have a check of what you’ve been up to during a session as identified by these algos, mainly the sound and social ones. Once you’re done, it’ll mail you all the ones it found. This is totally free at the moment, though presumably they’re exploring monetisation options so as not to go the way of Vine.
They’re pitching it as a time-saving tool. While I’m not sure how much success they’ll have getting general use, it’s fairly fascinating technology. Even if it doesn’t work, seeing what it accidentally picks up and why is going to be hilarious – from players falling off stages to particularly cringeworthy crowdshots, if my knowledge of Twitch chat is anything to go by. Give it a try on the website.