Roundtable Discussion: Perspectives on Cycle Time

Your engineering speed is your organization’s competitive advantage. A fast Cycle Time will help you out-innovate your competitors while keeping your team nimble and motivated. It’ll help you keep feedback loops short and achieve the agility necessary to respond to issues quickly. Most importantly, it’ll align everyone – from the CTO to the individual engineer – around what success means.

To discuss Cycle Time and its implications, we invited a panel of engineering leaders and industry experts for a virtual round table.

Code Climate Founder and CEO, Bryan Helmkamp, was joined by:

• Bala Pitchandi, VP of Engineering at VTS
• Katie Womersley, VP of Engineering at Buffer
• Bonnie Aumann, Remote Agile Coach at CircleCI

Over the course of the hour, these panelists covered topics such as using metrics to design processes that work for your team, building trust around metrics, and tying delivery speed to business success.

Here are some of the key takeaways:

Prioritize innovation over semantics.

Bryan Helmkamp: When I think about Cycle Time, I think about the time period during which code is in-flight. And so I would start counting that from the point that the code is initially written, so ideally, the time that that code is being typed into an editor and saved on a developer’s laptop or committed initially, shortly thereafter. Then I would measure that through to the point where the code has reached our production server then deployed to production.

Bala Pitchandi: I define Cycle Time as when the first line of code is written for a feature — or chore, or ticket, or whatever that may be — to when it gets merged.

Bryan Helmkamp: I think it’s also important to point out that we shouldn’t let the metrics become the goal in and of themselves. We’re talking about Cycle Time but really, what we’re talking about is probably more accurately described as innovation. And you can have cases where you’ll have a lower Cycle Time, but you’ll have less innovation. It can be a useful shorthand to talk about (e.g., we’re going to improve this metric), but really that comes from an objective that is not the metric. It comes from some shared goal. That understanding matters much more than the exact specific definition of Cycle Time that you use because really you’re going to be using it to understand directionality: are things getting better? Are they things getting faster or slower? Where are the opportunities to improve? And for that purpose, it doesn’t really matter all that much whether it’s from the first commit or the last commit or whether the deploy is included; you can get a lot of the value regardless of how you define the sort of thing.

The faster you fail, the faster you innovate.

Bala Pitchandi: When I was pitching this to our executive team and our leadership team, I used the analogy of Amazon and Walmart. Most people think that Amazon and Walmart are competing against each other on their prices. But keeping aside the technology parts of Amazon, they’re basically retailers. They both sell the same kind of goods, more or less, but in reality, they’re competing against each other on their supply chain. In other words, you could sell the same goods, but if you have a better supply chain, you can out-execute and be a better business than the other company, which may be doing the exact same thing as you are. And I think that I was able to translate that to the software world by pointing out that we could be building the same exact software as another company, but if we have a better software delivery vehicle or software delivery engine, that will result in better business goals, business revenue, and business outcomes. That resonated well, and that way, I was able to get buy-in from the leadership team.

Katie Womersley: The most interesting business discovery we’ve had from using Cycle Time has been where it’s been slow. It’s been a misunderstanding on the team about what is agile, where we’ve thought, “Oh, we’re very agile.” Because we’ve had situations where we’re actually changing what we’re trying to ship in PRs — we’re reworking lines of code because we think that we’re evolving by collaborating between product managers and engineers with the PM changing their mind every day about what exactly the feature is. And the engineer says, “I’ll start again, I’ll start again, I’ll start again.” And the team in question felt this was just highly collaborative and extremely agile, so that was fascinating because, of course, that’s just not effective at all. You’re supposed to actually get something into production and then learn from it, not just iterate on your PRs in progress. We actually need some clarity before we start coding, and I do believe that that had business impact on knowing what we were shipping.

Bala Pitchandi: We believe in this concept of failing fast. We want to be able to have a small enough Cycle Time to ship customers something quickly. Maybe it’s a bet on the product side, and wanting to ship it fast, sooner and get it into the customer’s hands, our beta users hands, and then see if it actually sticks and if we’ve been able to find the product market fit. That failing fast is really valuable, especially if you are in the early stage of a product. You really want to get quick iteration and get early feedback and then come back and refine your product thinking and product ideas.

Embrace resistance as a learning opportunity.

Katie Womersley: I got quite a bit of pushback on wanting to decrease Cycle Time. Engineers were really enjoying having work-in-progress PRs. They liked being able to say: well, here’s the shape of something, and how it could look as a technical exploration, and some of that was actually valid… But the real concern here is, we are using metrics as a way to have data-informed discussions. It’s a jumping-off point for discussion. It is not a performance tracking tool. And that seems to be the number one thing, and that’s important. We’re not trying to stack rank who has the most productive impact and then fire the bottom 30%. We’re trying to understand at a systems level where there are blockages in our pipes; we’re trying to look at engineering as a system and see how well it’s working. I think that’s very, very important to communicate. I think for software engineering in general, we have a great culture for the most part of blameless postmortems, understanding human factors in engineering, and not attributing outages solely to human error. Basically, you want to take that mindset that you already have from incident response and incident management, and you want to apply that now into team process management as well.

Bonnie Aumann: Esther Derby has a podcast called Tea And The Law Of Raspberry Jam, which actually has an entire episode on coaching past resistance, and the most important part is actually to try to take that word out of your vocabulary. Resistance almost always comes from somebody who really cares and sees value in what is happening. And if you can understand where they’re coming from, you can learn better about how you might introduce something and get a better idea of what will benefit the team and the company. And it’s not to say that you need to be held hostage by one cranky person, but that one cranky person might just have liked a little bit more information for you than you knew before.

Bryan Helmkamp: In speaking with so many engineering organizations and getting started with data and insights, I think it can feel a little bit difficult to get started. It can feel a little bit like, “We have to get this exactly right.” And it’s common that people, as with anything, will make some mistakes along the way. So my recommendations for that are, give yourself the space to start small, maybe just looking at even one tactical thing. “How long is it taking for us to get code reviews turned around?” is a great example. Really focus on building up trust through the organization because trust is a muscle. And as you exercise it, it’s going to grow. Incrementally paint the picture to the team of what you’re trying to achieve and give them the why behind it, not just the what.

Design a Code Review process that works for your team.

Katie Womersley: Something that we do – and this is really controversial – but the vast amount of PRs are merged without review. This will probably shock many people, and the reason we do that is in every asynchronous team: when you’re waiting for review, you lose context…We have a rating system for our PRs. In fact, most of them are small PRs. They’re uncontroversial changes. And we actually don’t need that developer to wait a whole day for somebody to come online to say: ‘Arms up, let’s march.’ It’s just not that risky. Also, we’re in social media management. We don’t make pacemakers. Just from a business perspective: how badly wrong can things go? Most of our PRs are absolutely safe to merge without review. And then you will get PR comments, which are aimed at individual growth and performance, saying this could have been done in a cleaner way, for example. And then on the next PR, hopefully, the engineer would incorporate that advice.

For high profile PRs (e.g., you’re changing Login, you’re changing Billing), yes, that needs to get reviewed before going out … Most people will say it’s really important to have code reviewed before you merge things. And is that nice to do? Yes, it is. But in our very asynchronous team, what is the effect of doing that? Actually, it is a negative effect. So a lot of our code does not get reviewed before merge.

Bala Pitchandi: I couldn’t agree more on the over-relying on reviews. At VTS, we do pair programming a lot. Even when two developers were paired on a PR, we used to still require a third person to review. Later, we just figured that two people working on a PR is enough of a review, so that if you’re pairing on a PR, you just merge it.

As coding days go down, productive impact scores go up.

Bala Pitchandi: I guess, of all the bad things that happened with the global pandemic, one benefit was that we all went distributed. Our Cycle Time actually went down by 20% without having to do anything. Something that you often hear about engineers is that they’re always complaining about distraction. There’s too much noise, people are talking, they want a noise-canceling headset. We were like, you know what? There is actually truth to that because when everyone went remote, guess what? There was less distraction, they had more heads-down time, and they were actually able to contribute to getting the code through the pipeline lot faster. And then we were able to build on that. Focusing on the team-level metrics and encouraging the teams to really focus on their own things that they care about was really helpful.

Bonnie Aumann: We’ve been remote from the beginning. The pandemic made everything more intense. I think is what we really learned. And the thing that became harder was making sure that people took the time and space to actually turn off and recharge. One of the anti-pattern practices is that on a team of very senior people, each person will take something to do and then push it forward. But then, of course, your overall Cycle Time on each of those goes down because you haven’t focused on pushing it all the way through. I think there’s been a lot of cultural outreach from managers to employees to be like, have you hydrated today? Have you done your basic care? When was the last time you saw the sky? It makes it more of a complete package.

Katie Womersley: I have a great mini-anecdote on that. We switched to a four-day workweek to get ahead of burnout with the pandemic. And what we saw in our data was, while the number of coding days went down; obviously, it’s a four-day workweek. We actually saw a productive impact score go up across every single team…We’re getting more done in four days than we were in five days, which is completely wild, but we can show that’s working. We’re in company planning discussions now, and we’re asking, “Should we move back to a five-day workweek to get our goals achieved?” But the data actually shows that we’ll get less done because people will be more burned out.

Agile provides teams with valuable vernacular, but be wary of doing agile by the book.

Bonnie Aumann: I think it’s complicated, but I’ll try and narrow it down. An absolute unit of agile is feedback and feedback cycles. When you’re looking at a thing like Cycle Time, it is a tool to let you know how fast you’re going, but it’s just information. What you do with that information is up to you. If you see people are working seven days a week and you think that’s a good thing, then you’re going to use it to make sure that it’s a good thing.

I will say that one of the things that is beneficial of doing agile by the book, particularly if you’re doing it by The Art of Agile Development or some of the books that really get down into the nitty-gritty, is that you have everybody speaking the same language, you have people using the same words. Because one of the hardest things about getting good metrics on a multi-team situation is making sure that people are using words the same way. And when something opens and when something closes, is when something starts and when something finishes. If you’re using Trello in one place and JIRA in another place, and GitHub issues in a third place, getting that overall view of what your data is, is challenging.

The more you rely on concrete data, the more flexible you can be.

Katie Womersley: It’s so important to look at your metrics and figure out what actually works with the team you have and the situation you’re in, and not just listen to a bunch of people on a panel and be like, “We must do that.”

Bryan Helmkamp: Your point reminded me of an interesting quote about agile and just this idea of process adherence becoming the goal in and of itself. I think the quote was something like, if you’re doing agile exactly like the book, you’re not actually doing agile. It’s inherent. To bring this around to Cycle Time and data, the answers are different for every team. But what I think is really powerful is that by having a more clear objective understanding of what’s going on, that opens the door to more experimentation. And so it can free up a team to maybe try a different process than the rest of the org for a while. Without it just being like: Well, we feel this or that, and being able to combine both of those things into a data-informed discussion afterward about how it went and being able to be more flexible despite the fact that data may make everything feel more concrete.