posts displayed by category
This is part two in a three part series on our statistics pipeline. The first part described our pipeline. In this part, we’ll go into detail concerning particular design decisions. The third part will cover the history of analytics at Twitch. Head over to our team page to get to know more about what we do.
When bringing together our Data Science team we established a clear goal:
Make our co-workers want to give us their data.
Editor’s Note: This content is also mirrored on the author’s personal blog. So check that out, too, for this and other musings.
This is part one of a three-part series covering all aspects of our data analysis. This part describes our pipeline. The second part will go into detail concerning particular design decisions. The third part covers this history of analytics at Twitch.
In the beginning, we logged all of our data to Mixpanel. As our user base grew, we logged an increasing number of events; this growth in data points vastly outstripped our user growth to the point where we were sending Mixpanel billions of events per week. As our growth continued, we have needed to make better decisions based upon joining different events to gain really deep insight into our users’ behaviour. Count based metrics, such as those provided by Mixpanel or statsd, are insufficient when it comes to this and given the ever increasing cost of Mixpanel, we brought together a team to work on storing our event data in an economical fashion while providing the tools to query the data without these downsides.
Twitch Engineering is very excited about our new Group Chat feature – not only because it’s friggin’ awesome, but also because we have been using it to facilitate broader improvements to our chat architecture. These changes are aimed at improving our QoS – we hate dropped messages as much as you. For this post, I’d like to give an overview of some of the things we’ve been doing to improve overall chat reliability over the past few months.
One of Twitch Engineering’s long-term goals is to move away from a monolithic Ruby on Rails application toward a Service Oriented Architecture, where each feature of the site is a separate component. Our Group Chat design facilitates this drive by creating a new service that will warehouse all chat data (mod lists, Chat colors, etc), data that has historically existed in databases owned by our Rails system. This data migration is currently a work-in-progress.
Look! A snazzy video walkthrough!
UPDATE: This has now been rolled out to Twitch Turbo subscribers as we ramp up our scale.
One of the most beloved features on Twitch is the ability to chat with the broadcasters and your fellow gamers. In many ways, it’s what makes Twitch, Twitch. It’s where community memes begin. Where you express your glee, outrage, befuddlement, or amusement, as the case may be. Twitch is a social place, and chat is where the social exchange happens.
Fun with Stats: How many chatters do we average a day? ~700,000 uniques, hooray!
Today, we’re proud to announce a new beta initiative called Group Chat. Group Chat is a way to create invite-only chat rooms that live separately from the traditional Channel Chat.
Without further ado, here’s the one-and-only djWHEAT to take you through the particulars.
Spring is here and love is in the air. So we flew to Paris with some servers, a few cables, and our dreams. Ah, Paris.
We’re proud to announce our newest infrastructure addition in Paris, France. We’ve been hard at work delivering on a promise to bring more capacity to Europe. More capacity means more Twitch, and we want you to Twitch all day and night. In a good way of course.
Our Paris point of presence (POP) follows four POP updates and the addition of Prague in just over 3 months. We’re so proud of what our infrastructure and video teams have been up to, we wanted to recap recent activity:
NOTE: For the full press release, click here.
Today we announced the Twitch Mobile Software Development Kit (SDK). This new game developer tool will enable the live broadcasting, capturing and archiving of mobile games. This latest innovation will help mobile gaming companies reach the Twitch community directly from their devices and across other platforms as well.
A couple of weeks back, we completed an upgrade of the Twitch London Point of Presence (POP). As a result, we can serve you with an additional 200% of capacity and accept more ingests (broadcast) through LHR.
We’ve got more infrastructure upgrades and additions on the way, so stay tuned for the future.
As chat is undoubtedly a VERY important feature to our community, we’re working on a series of chat updates to improve server-side and client-side performance and functionality. These updates will roll out over the coming weeks.
Starting today, you may begin to see a notice on Partner channels when an advertisement can’t be loaded. This message is dismissible and will only last the length of the advertisement(s) you didn’t see. This is the first in a series of experiments on partner monetization based on direct partner feedback.
We recently upgraded our Twitch Los Angeles Point of Presence (POP), increasing its serving capacity by roughly 400%. This upgrade also increased ingest (broadcast) capacity.
We may also have installed a Sound Blaster 16 sound card, a vintage Anderson-Jacobson acoustic modem for backwards compatibility, and a cutting edge Courier External 56K* V.92 Global Dial-up Business Modem.
We continue to work on capacity additions and upgrades, so keep an eye out for future announcements.
FIRST: We’re keenly aware and sympathetic to the notion that less latency will make social interaction better. We care very much about interactivity and continue work to deliver less overall latency. We assure you we have not directly imposed any latency on top of what the system currently delivers.
As the new video system has been 100% live for a week, we wanted to provide an update on its performance and address the feedback and concerns some of you have posted.
Previously only available from the “Following” directory in the left navigation of the site, you now have access to all your followed channels in a persistent right-hand navigation across all Directory pages.
You’ll see all the channels that you follow, with live (“Online”) channels at the top and non-live (“Offline”) channels in a list at the bottom. Navigation through all of your followed channels is now much easier and more fluid.
Over the past year, we’ve been hard at work on massive improvements to the video system. Our goal is to improve the video system’s stability and scalability so that we can offer the best quality video to as many users on as many platforms as possible, all while we continue to experience explosive growth.
For over a year, we’ve been building a new in-house video system that will allow us to scale to the rapid growth of our platform. Part of this process involves updating the current video player. The update to the video player affects anyone out there that embeds the Twitch player on your websites.
If you embed Twitch content on your site, make sure to check your embed code. If you use:
You’ll notice this is basically a change from Justin.tv to Twitch.tv. (The latter player is already live!) With this one simple change, your embed experience will be seamless when the new player goes live and the old player is decommissioned.
We’ll have more to tell you very soon, so thanks for tuning in and utilizing all Twitch has to offer!
Editor’s note: This is the first in a new, ongoing series of blog posts dedicated to keeping our users informed of what we’re doing to improve service around the world. We’ve received a great deal of feedback from you all on Twitter, Facebook, reddit, Team Liquid, in person, et cetera. We want you and your friends to use our service because it’s the best. We want you to know what we’re doing to make it the best.
It’s been a while since we talked about either the eSports communities or European audience here on Twitch.