What effect does the Steam Workshop actually have on concurrent player numbers?

steam_workshop_mod_investigation

We always take it as read that mods are good for PC gaming – that giving the gaming community tools to create new content for their favourite game should extend the life of that game. The problem with mods previously is that distribution and installation of the final content was a hard problem – one that Steam’s Workshop has made a good stab of fixing. Now, installing a mod is as easy as clicking a link.

I wanted to investigate what effect the Steam Workshop is having on the games that have adopted it. To do so, I’ve used SteamGraph to track player numbers over the past year for most of the Steam Workshop games. The only game that isn’t included is Naval War: Arctic Warfare, as there too few players online for it to be tracked.

The hypothesis is, surely, that Steam Workshop will bring more players into your game, and keep them playing for longer. Is that true?

Team Fortress 2 was the first game to introduce the Steam Workshop. In it, players can create new weapons and new cosmetic items. It was introduced in part to help Valve assess and prioritise the best items for introduction to the game. So, a big success?

If your criteria for success is increasing player numbers… not really.

According to the data, the introduction of the Steam Workshop in October 2011 had a minor impact on player numbers, but the player population declined substantially in the months following. It only recovered after the introduction of the Pyromania update.

Not a great start.

So. Let’s try again. DOTA 2’s entire business model is based around the Steam Workshop. Players will be creating great items, which will then be sold in the Steam store. New items means new content, which should translate to new players, right?

No.

Dota 2 is growing, but the introduction of the Steam Workshop on the 1st June seems to have had little impact on the trend. There is a dip immediately before the launch of the Workshop, but I wonder if that is down to Valve holding back invitation waves before launch, creating negative churn. We alsoknow that Dota 2 is still in closed beta, and that its growth is artificially limited. So it’s perhaps not the best test case.

Let’s try the last Valve game to use the Steam Workshop: Portal 2.

Ooh, shiny. The introduction of the workshop to Portal 2 had an immediate and significant impact on player population, and it continued to raise the number of active players post launch to beyond that was seen previously. There is a caveat: the launch of the Workshop with Portal 2 coincided with a flash sale. But the previous jumps are most explained by sales, too. So we can at least say that the introduction of the Workshop, when introduced with a price discount, increases the player population.

Now let’s look at the numbers across third party games. Skyrim is by far the most popular game on the Workshop – with 8600 items uploaded at time of writing. It’s also one of the most popular games on Steam. Let’s see how it did.

Umm.

So, let’s make some excuses for the lack of a spike. Skyrim already had a successful and busy modding community hosted at the Skyrim Nexus – personally, I believed that Skyrim’s success on PC (Bethesda said that in the US, the PC version outsold the console version 3 to 1) was due to players knowing it was a highly moddable and extensible game. So the introduction of the Workshop didn’t necessarily mean there was a sudden influx of new content – it just meant easier sharing of what was already available.

One more reason: it may well be that players had already burned through what most of Skyrim had to offer in the months previous to the Workshop launch; jumping back into the game with a fresh install wasn’t immediately appealing.

But there is some success to note: Skyrim’s player population has held steady for the six months following the launch of the game – that has to be down to players installing and sampling new content.

Now, let’s try Civilization V.

Blimey.

So, the chaos at the far end is fascinating. The launch of the Workshop had an immediate and dramatic impact on the Civ Vi player population. It then continued to grow, helped, presumably, by it being available early in the Summer Steam Sale, and the launch of Gods and Heroes, and the new expansion. Civ V players seem to really like mods. It also appears that mods keep them coming back to the game.

Here’s what it looks like when we zoom in on that mess.

So: the Workshop boosted player numbers, but not quite as much as the expansion pack, but the Sale continued to keep numbers up.

I think there’s something interesting going on, though. I wonder if the success of the Workshop with Civ and Portal suggests that there are more valuable types of mod for retaining players. The most popular mods for Civ are either total conversions or maps – Tamriel, Westeros, Planet Earth, Europe and Middle Earth all feature highly. Downloaded maps give the player a complete experience: a start, middle and end.

Compare that to a downloaded mod for Elder Scrolls. The most popular content on the Elder Scrolls workshop is mostly complementary to a player’s experience – new weapon and armour sets, new mounts and player housing. Similarly, TF2 and DOTA items are supplementary to what you play: but they don’t supplant or extend the base game’s content.

And finally, that would explain the success of the Portal 2 Steam Workshop. There, players download complete puzzle chambers – they are again complete experiences.

There are two indie games on the Steam Workshop that are worth considering.

The first is Dungeons of Dredmor, it’s an indie fantasy Roguelike with a small but passionate playerbase. If you like Dungeons of Dredmor, you tend to really like it.

Ooh.

First note: the bit where it appears that no-one is playing can be ignored – what that really means is that it’s dropped off the top 100 games on Steam. The second point is thatthe Steam Workshop really does work. Steam sales work too.

What I really like about this graph is it demonstrates how sticky Workshop content can be: for a full month, it looks like a decent proportion of the Dungeons of Dredmor playerbase took to downloading and playing mods as a hobby. The most popular mods for DoD are new player classes – the goal is to play until you die. With new player classes, like the amazing Philosoratpr, or the Kanadian Goalie. He’s a defensive wizard “who really likes it when stuff gets blocked”.

PC gaming is best.

The last game on our list is Dungeon Defenders – it’s a successful indie tower defense game in which you level characters, earning new weapons and armour sets as you go.

Woo! Success!

The immediate impact is clear – more people playing the game. Combine that with a sale, and the Steam Workshop appears to have rejuvenated the Dungeon Defenders playerbase. And what are they downloading? The vast majority of mods are new maps, designed in Trendy Entertainment’s version of the Unreal Engine 3 map tool.

There’s a final question. Are there any games coming up that would benefit significantly from the Steam Workshop?

I think there’s a huge missed opportunity within the implementation of the workshop – I’m extremely surprised that Steam doesn’t distribute new levels for Team Fortress 2, Left 4 Dead 2, and the Counter-Strike games. I’ve spoken to Valve about this in the past, and they’re clearly working on the problem: apparently there’s a limitation on the size of files that the workshop can distribute. If Valve can create a market for new FPS levels within the Workshop for TF2 and L4D2,I’d expect to see a greater jump in player numbers than was even seen in Portal 2.

More generally,strategy games and tower defense games would seem to benefit the mostfrom both the Workshop and a good map editor. You could easily see how Orcs Must Die 2, Company of Heroes 2, XCom and more could all benefit if the dev teams can find the time to create and support level editor. That’s not an easy task. Developers I’ve spoken to have explained that while the Workshop support is relatively easy to implement, turning the tools they use to create the game into something that players can use is not a trivial amount of work.

Finally: there are a class of sandbox games that rely on community content to extend their lifespan: consider serious racing simulations like Live for Speed, or the forthcoming Project CARS, ArmA and it’s vast modding community, Garry’s Mod… The impact of the Steam Workshop on those games could be staggering.

TL:DR

Some games benefit from the Steam Workshop. Some games benefit more than others. There were graphs.