Update: Changes To VODs On TwitchAug 06 2014 · 380 comments · Engineering, Tech
[UPDATE 2:45 PM PT Aug 7]: The YouTube Exporter is back in service. Thank you for your patience.
[UPDATE 9:20 AM PT Aug 7]: Today’s AMA with Emmett Shear will now be hosted on /r/IAmA. Please join us at 10:30 AM PST.
[UPDATE 10:50 AM PT Aug 6]: We need to temporarily disable the YouTube Exporter so that we can resolve increased capacity issues. The fix will take a day and should be ready for use again tomorrow.
Our goal at Twitch is straightforward: deliver the highest quality video. This includes the ability to watch video on demand (VOD) on all of our platforms, not just the website.
In order to create a system that supports live and VOD across the globe and on multiple platforms, we need to make significant changes to the way we’re currently storing video. Today, we’d like to discuss what these changes are, why they’re necessary, and how they benefit the entire Twitch community now and in the future.
VOD: Past Broadcasts and Highlights
First, some language clarification. Twitch VODs include both past broadcasts and highlights. After you finish your live broadcast, the complete broadcast is saved in Past Broadcasts. Historically, we’ve offered you the option to save a past broadcast forever.
Past broadcasts can also be edited into Highlights and watched on the website or exported to YouTube for promotional use. Currently, a highlight cannot exist without saving the entire broadcast as well.
Shortcomings in the Current System
The current system has a few notable limitations:
- VODs are saved in FLV format, and platforms like mobile and console devices can’t playback that format.
- As past broadcasts are saved in 30-minute chunks across multiple media servers, a single server maintenance or failure could prevent proper service of an entire VOD.
- Extremely limited availability of transcodes on VODs.
- Due to current storage requirements, past broadcasts can only be saved for 3 days without an explicit “save forever.”
- We must save an entire past broadcast indefinitely if a highlight from it exists.
Looking at Viewership Data
We found that the vast majority of past broadcast views happen within the first two weeks after they’re created. On the days following, viewership reduces exponentially.
We also discovered that 80% of our storage capacity is filled with past broadcasts that are never watched. That’s multiple petabytes for video that no one has ever viewed.
Highlights, on the other hand, have much more value and longevity. Over their lifetime, highlights get 9x as many views as past broadcasts.
Here’s the New Deal
This data, along with feedback we’ve heard from the Twitch community, made it obvious that our VOD system needs to be improved. Today, we start to move in that direction with an update to how we manage your VODs.
This update is the first step in a process to build features that you’ve asked for:
- Better VOD quality of service for our international viewers.
- Ability to watch VODs on mobile and other platforms.
- Secure storage of your past broadcasts (with triple redundancy, we make three copies of your past broadcasts).
- An easier way to export highlights of any length to YouTube.
- A step toward conceptualizing new features like DVR, matching quality options from live to VOD, and better VOD discovery.
Going forward, we’re increasing default rolling storage for past broadcasts from 3 days to a maximum of 14 days, for everyone who has opted in (i.e., enabled Archive Broadcasts). For Turbo subscribers and members of the Twitch Partner Program, that storage is increased to a maximum of 60 days.
Unfortunately, increasing the amount of storage this way comes at considerable cost. In order to enable this, we have to remove the “save forever” option entirely for past broadcasts. Given the viewership patterns on past broadcasts, we believe the tradeoff is better for everyone. To be clear: this is not a move to economize on space. Due to the triple redundancy, it will actually require us to substantially increase our total amount of storage.
Highlights will be saved indefinitely; however, they will now be limited to 2 hours in length. All prior highlights that you’ve made will be saved regardless of length.
As for existing past broadcasts, beginning three weeks from today, we will begin removing them from Twitch servers. If you would like to keep your past broadcasts, we encourage you to begin exporting or making highlights of your best moments so that they’re saved for posterity.
New Video Manager
To help make managing your past broadcasts easier than ever, we’ve released a new Video Manager. This is your one-stop shop for:
- Creating highlights
- Exporting past broadcasts
- Expiration countdowns
The following video walks you through how to use the new Video Manager:
The new Video Manager also helps you prepare for the upcoming changes to the way we’re storing your past broadcasts.
How to Prepare for These Policy Changes
- Use the new Video Manager to highlight the content you love so it will be saved indefinitely.
- Export past broadcasts you want to save to YouTube. Note: Past broadcasts will be exported in 2-hour segments.
- We will remind you regularly over the next three weeks of the impending changes so you’re prepared.
We understand that you will have questions. And, we will do our best to answer any concerns you have. We will create a FAQ as a follow up to this post. Please send your questions to firstname.lastname@example.org and we will add some of these to our FAQ.
We want to hear your feedback and questions. Tune in to the following events to ask us (almost!) anything:
- Reddit AMA on /r/Twitch: Thursday, August 7, 10:30am PST
- Twitch Weekly: Friday, August 8 at 2 PM PST.
And, as always, please feel free to leave your comments below. We will answer as best we can.