Engineering Leadership Summit: Juan Pablo Buriticá, Head of Engineering, LatAm at Stripe

As part of this year’s Engineering Leadership Summit: Virtual Edition, we spoke to Juan Pablo Buriticá, Head of Engineering, LatAm at Stripe. He discussed strategies for implementing delivery metrics, and shared how he got buy-in from his team. Below is an excerpt from the fireside chat portion of Juan Pablo’s session, edited for length and clarity. 

Hillary Nussbaum, Content Marketing Manager, Code Climate: Welcome to the latest session in our Engineering Leadership Summit I’m here with Juan Pablo Buriticá, Head of Engineering, LatAm at Stripe. I’ll let you take it from here, Juan Pablo — introduce yourself, and tell us a little bit about what you do and how you got to where you are.

Juan Pablo Buriticá, Head of Engineering, LatAm at Stripe: Of course. Hello Hillary, and hello everyone. Welcome to my kitchen. I figured since we were going to talk about shipping and delivery and metrics, why not try to ship a product? I’m Juan Pablo Buritica, I live in New York City. I’m from Bogotá, Columbia. I’ve been in the U.S for a little bit over a decade, and in New York for a decade. I’m a former chef, turned engineering leader, and I’ve been leading product engineering organizations for a decade or so as well. 

More recently I was at Splice, a music company, for three and a half years. I was the VP of Engineering, and we built a distributed team of about 60 engineers within the Americas. I started working with delivery metrics there, and tried to understand how to make the team more efficient without working longer or harder. That led to a few talks, and also finding Code Climate and actually using it in production. 

Then earlier this year, I took a sabbatical. I was ideally going to start a consultancy on delivery engineering metrics, but Stripe came along with an offer of working to expand the product into Latin America. It’s a mission that I am extremely motivated by, and that I’ve really been thinking about a lot. I joined eight weeks ago, so I’m just getting started and trying to figure out how all my experience fits into the picture. 

So, welcome to my kitchen and thank you for having me.

Thank you for being here, and for having us in your kitchen. It’s a beautiful kitchen. You touched on this a little bit, but you just started a new role, and you’ve spoken a lot about how you used metrics in your last role. How are you thinking about metrics and similar strategies as you settle into your new job?

All right. Before I answer this question, I’m going to tell you, we’re going to attempt to make a mushroom risotto. There are some things that have been prepped. So you can think of this as work, engineering work that has already been scoped and outlined and is ready to be worked on. And then there’s some mushrooms and some onions that I need to chop and prep — this is work that is unscoped. 

Now, let’s go to the question. I think of delivery metrics as visibility. At Splice, it was very useful for me to see the bottlenecks that the organization had in our delivery pipeline, and how we could drive efforts to solve those bottlenecks.

Now I’m with Stripe, a much larger organization that has a ton of metrics all over the place. And I am leading a team that is more on the surface, right? We expand the existing product into the region, at least initially. I’m trying to figure out how to better measure or get insight into what the individuals of LatAM engineering are like, what their role is in the delivery pipeline, and how we can measure the bottlenecks they may have. So I’m trying to figure out how to see the delivery cycle, because I have no visibility yet. 

And in the past, is that how you’ve typically approached metrics, as a tool for visibility?

They’ve been a tool for visibility since we integrated the first version of our delivery framework at Splice. I remember my boss asked me, “What are the things that we’re even trying to solve?” And I couldn’t say, “I think we’re slow, it feels slow.” I needed to know why we were slow. So I read a bunch, and I found the book, Accelerate, by Gene Kim, Jez Humble, and Nicole Forsgren. I learned a ton about the capabilities that organizations should have, and how you can use metrics like Cycle Time or Deployment Frequency to evaluate your process.

Having these metrics gave me three things. It gave me the language to talk to the team about our delivery pipeline and things that we should consider. It gave me insight into what to measure and how to measure it, and where it matters. A ton of the research that is out there highlights why it’s important to have a high-performing engineering organization. With metrics, you can actually tie organizational performance to engineering and delivery. And that was really, really useful. Because I could talk to the board about how we think of staffing and whether a team was going fast or slow. 

And since you’re heading in this direction a little bit, what strategies do you use for talking about metrics with people who aren’t necessarily familiar with metrics, or who aren’t the ones using them? 

Metrics have helped me now illustrate portions of the engineering delivery process that aren’t intuitive. For stakeholders or people outside of engineering, Cycle Time as a concept and as a metric has been extremely useful in talking about staffing and throughput.

We are used to talking about velocity in points, and we’ve taken this metric as an actual metric of performance or delivery. But it is a representation of capacity, and it’s very relative. If we have two teams of four engineers, their velocity is going to be relative to the scale they’re using, as well as their skills, and even the work they’re taking on. So it isn’t a good metric for us to talk about the speed we’re actually delivering at. It’s just a measure of capacity, of how much we can take. It can tell us if we estimated the right amount of work and if we can consistently deliver on our estimates, but it doesn’t tell us how fast we are. That has become very useful in executive conversations with product or finance. We can set a realistic timeline when we have a better understanding of how fast we can go.

That’s how we talk about Cycle Time, as the speedometer for the team. And then how do you talk about metrics with your team, whether your team is new to metrics or familiar with them?

That is the most fun conversation to have, and I’ve found that this is one of the pieces engineering leaders are struggling with the most when they try to implement metrics. It’s been interesting for me to see that we talk a lot about being data-driven in our industry, but when we try to have a conversation about metrics and data, there’s a lot of concern, especially for ICs. And rightfully so, because there’s always the concern of, “How are these metrics going to be used against me?” And there is a general mistrust towards management. The hardest challenge for me in implementing metrics at Splice was getting the buy-in of the team.

I approached it in a couple of ways. First, I did have a ton of trust from them, and I had a lot of leeway to cash in on the trust currency that I had. But I still had to make an argument about how we were going to use metrics, and why. I wrote an entire RFC, and that’s how I convinced my team that treating this as a pilot was useful. The second thing I did was, I didn’t give access to those metrics to anyone other than myself and the Director of Production and Engineering. We could eventually give access to other people, but taking access away would be much harder.

There’s always the chance that metrics can be used against people, and since the purpose of these metrics is to measure the delivery pipeline and not individuals, I wanted to make sure that they knew we were measuring a process and not their individual performance. Some of the tools we were using had information about individuals, so I protected the access to that, and I think they trusted me with that.

Over time, I would share reports and I would share different bits of information that would give them more confidence in how we were using these metrics. It is on us as leaders to earn trust and demonstrate that we’re going to use these metrics for what they’re meant to be used for.

How did you choose where to start, and which metrics to measure?

We started by having an onsite with senior engineering leadership, where we tried to identify what we believed were the biggest problems at Splice at the time. Then we looked at what metrics we thought could drive some of those things. 

For example, feeling slow was one of these problems. So we started with Time to Merge, or how long does a Pull Request take to merge? We didn’t know what was good or bad, but we said, let’s say all PRs are going to be merged in 36 hours, and that will help us evaluate whether this happens. And if not, what obstacles do engineers have in getting their work merged?

And the second challenge we had was deployment. Our deployment wasn’t automated, so a merge didn’t necessarily mean that the code was accessible for other people to use. And so we created a deployment metric as well, and then started learning through that. It was a very iterative process.

And then how did you use those metrics to inform your strategy, and how did they impact what you might’ve been doing as a manager with your ICs?

Our strategy was to set some metrics that we believe may help us improve our delivery process, measure them, and get a baseline. From there we could try to understand whether they were really out of a normal range, and whether we really needed to improve. And then we would create projects that would help us drive down those metrics. And there was a production engineering team that had the purpose of doing this. 

The second part of being very specific about the strategy was that we had to crawl, walk, and then run into delivery. Primarily because we didn’t want to start running and go fast in the opposite direction, delivering a ton of terrible quality product.

So we had an angle on delivery, Cycle Time, and we had a Deployment Frequency metric as well. And then we had one that helped us monitor the quality of what we were delivering. It took us six months to properly deploy this entire plan. The first few months, while we figured out how to measure things, we deployed our own dashboards and we brought on a tool — I think you know about the tool, it’s called Velocity. This is not a sponsored talk. Once we had some of that visibility, it was easier for the team to know what they were driving towards. And they took ownership. The production engineering team at Splice took ownership of their own metrics and how to drive them. And they, in the end, made it successful.