Pinterest engineering blog

  • Back to all posts
  • Feb 14, 2017
  • Share

Distributed tracing at Pinterest with new open source tools

Suman Karumuri

Suman Karumuri is a tech lead on the Visibility team

The Pinterest backend consists of hundreds of microservices deployed across tens of thousands of machines. A simple request to the Pinterest home feed can make hundreds of network calls to dozens of backend services. While we know how long a request takes to execute, previously, we didn’t know why a request was slow. To collect this data, we built Pintrace, a distributed tracing pipeline that tracks a request as it’s processed across our Python and Java backend services. We also built Pintrace collector to help us debug performance issues.

Pinterest engineering blog

  • Back to all posts
  • Jan 30, 2017
  • Share

Evolving MySQL Compression - Part 2

William Tom

William Tom was an intern on the SRE team during the 2016 summer. He’s currently completing his degree at UC Berkeley.

Generating a predefined dictionary

This post follows a previous one, Evolving MySQL Compression.

Tags: 

Pinterest engineering blog

  • Back to all posts
  • Jan 27, 2017
  • Share

Building Pinterest’s video platform

Norbert Potocki

Norbert Potocki is the tech lead for Video & Image Platform

Building Pinterest’s video platform

As video content becomes more ubiquitous on the internet, it’s an increasingly important part of Pinterest. Pinterest is one of the largest visual platforms in the world, where people have saved billions of pieces of rich media, making it ripe for further video disruption. As part of our progress with video, we launched native video Pins last year to help Pinners more seamlessly watch videos right on Pinterest. Here we’ll cover how and why we built a video platform behind it. Read on!

Pinterest engineering blog

  • Back to all posts
  • Nov 18, 2016
  • Share

How we switched our template rendering engine to React

Jessica Chan

Jessica Chan is a Software Engineer on the Core Experience Web Platform team

In 2015, we made the decision to migrate our legacy web experience to React to keep up with our fast growth and perform better with increased developer velocity. Ultimately, we found React rendered faster than our previous template engine, had fewer obstacles to iterating on features and had a large developer community.

Pinterest engineering blog

  • Back to all posts
  • Nov 4, 2016
  • Share

Open-sourcing Rocksplicator, a real-time RocksDB data replicator

Bo Liu

Bo Liu is an engineering lead on the Serving System team, which is part of the Infrastructure team

Pinterest’s stateful online systems process tens of petabytes of data every day. As we build products and scale billions of Pins to 150 million people, we need new applications that work in a way where computation co-locates with data. That’s why we adopted RocksDB. It’s adaptable, supports basic and advanced database operations with high performance and meets the majority of requirements for building large-scale, production-strength distributed stateful services.

Pinterest engineering blog

  • Back to all posts
  • Oct 18, 2016
  • Share

Evolving MySQL Compression - Part 1

Rob Wultsch

Rob Wultsch works on stateful services for the Site Reliability Engineering team

Pinterest Infrastructure engineers are the caretakers of more than 75 billion Pins–dynamic objects in an ever-growing database of people’s interests, ideas and intentions. A Pin is stored as a 1.2 KB JSON blob in sharded MySQL databases. A few years back, as we were growing quickly, we were running out of space on our sharded MySQL databases and had to make a change. One option was to scale up hardware (and our spend).

Pinterest engineering blog

  • Back to all posts
  • Sep 23, 2016
  • Share

Migrating Pinterest profiles to React

Imad Elyafi

Imad Elyafi is a software engineer on Web Core Experience team

Since 2012, we’ve scaled our web framework Denzel (named after the greatest actor of all time) on top of Backbone. But nowadays, React is a golden standard. It has a large developer community and enables excellent engineering velocity and performance. Here we’ll look at techniques we tried and challenges we faced while migrating to React, starting with Pinner profiles. 

Tags: 

Pinterest engineering blog

  • Back to all posts
  • Sep 16, 2016
  • Share

Auto scaling Pinterest

Jinru He

Jinru He is a software engineer on the Infrastructure team

At Pinterest, infrastructure efficiency is one of our top priorities. During peak hours, Requests Per Second (RPS) can be twice that of RPS during off-peak hours. In the past, we maintained a fixed number of instances in the fleet to serve during peak hours and ensure the fleet wouldn’t be under capacity. However, as RPS decreases during off-peak hours, most of our instances run under utilization. Since Pinterest is built on top of AWS, we decided to apply Amazon Auto Scaling to our service.

Tags: 

Pinterest engineering blog

  • Back to all posts
  • Aug 16, 2016
  • Share

Tracker: Ingesting MySQL data at scale - Part 2

Rob Wultsch

Rob Wultsch is a database engineer on the SRE team

In Part 1 we discussed our existing architecture for ingesting MySQL called Tracker, including its wins, challenges and an outline of the new architecture with a focus on the Hadoop side. Here we’ll focus on the implementation details on the MySQL side. The uploader of data to S3 has been open-sourced as part of the Pinterest MySQL Utils.

Pinterest engineering blog

  • Back to all posts
  • Mar 11, 2016
  • Share

Under the hood: PinQueue, a generic content review system

Yuan Yao

Yuan Yao is a software engineer on the Infrastructure team

People use Pinterest to discover, save and do things that inspire them. Every day, we connect millions of people to billions of Pins, and it’s our mission to present the highest quality content to Pinners. We use machine learning to filter out content that violates our policies for everything from Promoted Pins to spam, but manual moderation is still needed since algorithms can’t cover all of our sophisticated and ever-evolving content policies.

Tags: 

Pages

Subscribe to RSS - infra