This week, Team Fortress 2 became one of the last games to switch over from Steam’s old content delivery system to SteamPipe. Once the switch-over’s complete, the game will boot more promptly and maps will load a little faster – but you probably won’t register the difference any more than you do the face of the man who brings your Amazon parcels to the door.
Before then, though, you’ll have to sit through a 20 to 30 minute “conversion process”. But what is SteamPipe? What’s it for, and what does it mean for your games?
There’s very little documentation available around SteamPipe: just a couple of wiki posts. But talking to developers and those close to Steam, we can get a clearer idea of what SteamPipe, is, and what it means for PC gamers going forward.
The first thing to understand is that SteamPipe uses HTTP to transfer game data, replacing the old Steam2 content delivery system. This is important for a few reasons: it means that Steam data can fly more easily between firewalls, and makes it easier for Valve to use third party HTTP caches and content delivery networks to speed up downloads.
At a more granular level, SteamPipe also allows for ‘delta updating’ – meaning that only changes to code need to be updated, which should make for quicker and easier patching. You’ll also notice that you’re able to browse the file structure of games downloaded via SteamPipe, rather than having them hidden away in .gcf files.
SteamPipe has also allowed for a new feature within Steam: the ability to set download rates. But it’s on the developer side of things that the new system will really make itself known.
In the traditional Steam update process, there’s a period of ‘approval’ – after developers submit their patch to Valve, and before it arrives on Steam. You might imagine that a Microsoft-style certification process is happening in the interim, in which Valve set about testing the updates for bugs. But Garry ‘Garry’s Mod’ Newman says that’s not the case at all:
“When I send the update to Valve they’re putting it on Steam,” he explained in a blog post last year. “The time delay is the delay between me uploading the update – sending the email – then someone at Valve getting into work, checking their email, and pushing the update out to the hundreds of Steam content servers.”
This manual phase of your average Steam update is the job of Valve’s technical account managers, who assign content servers to updates based on their geographical location, game size and popularity.
It’s a bottleneck and barrier: one in which Valve stand between game developers and their audience.
SteamPipe changes that. In practical terms, the new delivery system will mean that developers are able to upload and issue patches themselves, without any manual intervention whatsoever. They’ll be able to push out updates at opportune moments and in line with announcements. They’ll be able to warn players about the larger ones beforehand, and monitor and fix any problems at their own pace.
SteamPipe also gives flexibility within Steam for developers to branch their code and manage access to betas without hosting their own content servers. And code is slightly more secure: old versions of their games are now unavailable to curious eyes.
But SteamPipe comes with an inconvenience. If you’re a regular user of Steam, you’ll have seen the pop-up explaining that Steam has to update the game to a new content system. That is the conversion process; a new version of the game is created from the old files, before the previous version is deleted. The update process can take a long time: Valve themselves quote at least a 15 minute wait when you next start Team Fortress 2.
But it’ll be worth it. SteamPipe is the future of patching on PC. Now every time Dustbowl loads a little bit faster than you expected, you know what to thank.