Coaching

In this free, 45-minute webinar for CTOs, VPs, and managers of managers, we explain how an engineering analytics tool like Velocity can help every member of your team excel — whether they’re a new hire or an emerging leader.

• Code Climate Engineering Data Specialist, Sherianne Bolling, walks through the Velocity metrics and reports you need to scale your team while empowering ICs and managers to level up.

• CTO of Greenlight, James Gaythwaite, discusses how Velocity metrics are helping him lead his team through a period of hyper growth. With Velocity, he is able to:
  • Support new hires struggling to get up to speed
  • Identify high performers and give them opportunities to grow into leaders
  • Empower managers to improve team processes and become more effective coaches

To find out how Velocity can help you level up members of your team, reach out and talk to a product specialist.

The more engineers on your team, the harder it is to gain visibility. If you’re managing four people or more, you’re probably asking questions like:

• How do I supervise the work of all of these engineers?
• How do I make sure I’m adding value where needed?

You can leverage Velocity to address both of these concerns.

Get Immediate Visibility into Your Engineers’ Work

A quick scan of the Activity tab in the Team360 report will give you a sense of both what your team is working on right now and their general work habits.

Hover over a purple circle to see your team’s commits and understand what they’re working on without having to tap them on the shoulder.

Look for:

• Developers who haven’t committed code for a few days. They might be bottlenecked, especially if they’re new to the team.
• Developers whose commits don’t match your expectations for what the team is working on.
• Big commits and PRs (visualized by the largest shapes on the graph).
• Work that is distributed in a way that doesn’t match your expectations.

Use this concrete data as a jumping-off point for in-depth conversations with your team.

Identify High-Risk Pull Requests

Use the Pull Requests report to see PRs that may warrant your attention.

Most managers start by looking for open, high activity PRs. Activity is defined as PRs that are getting either a lot of commits or a lot of comments. To surface the oldest PRs, sort by age by clicking on the "Age" column within Pull Requests. Pay close attention to anything that’s been open for over 72 hours.

You might want to talk to someone like Harry Potter to see if there’s anything you can do to unblock him.

Understand Which Developers Might Need Help

There are a number of metrics that can help you find stuck engineers:

1. Weekly Coding Days represents engineering capacity, or how much time per week developers are able to spend actually coding. When this metric dips, it’s an indicator that a team member might be (1) working in large chunks or (2) stuck on a feature or bug.
2. PR Throughput & PRs Reviewed are output metrics. If these look unexpectedly low, it might be worth a quick conversation to understand what’s going on.
3. Review Cycles & Rework are both proxies for churn. Review Cycles represent the number of times a Pull Request has gone back and forth between the author and reviewer. Rework is the percentage of lines of code that have been rewritten by the same developer within 3 weeks and is helpful for identifying churn earlier on in the development cycle.

Compare these metrics by team or by person in the Compare report to identify who is stuck.

Dive Deeper into Churn Metrics

High Review Cycles and high Rework represent waste in the development process and are therefore a great place to start optimizing.

A developer with high Review Cycles has been asked to go back and fix things in their PRs multiple times. If you see someone with higher than expected Review Cycles, you can investigate further by going into the Analytics tab.

Run a query for Review Cycles, grouped by contributor over the last week or two. Scroll to the bottom until you see the following bar graph visualization:

To understand what’s blocking specific engineers, like Hecate, you can click the arrow next to her Review Cycle count to drill down into specific PRs.

Sort the number of Review Cycles from high to low and look through the ones that have been open the longest. Anything above 1.5 Review Cycles may indicate a problem. In this case, the top two open PRs with 4 Review Cycles is a good place to start.

You can run this same process for Rework, which is another indicator of churn, but at the coding level. Note that the top quartile of engineering organizations keep their Rework to less than 4%.

Use Metrics to Enhance Visibility

Though metrics can’t tell managers everything they need to know, they can be a valuable tool for gaining insight into the work that’s happening on your team. With data like that found in Velocity, you can spot stuck work before it becomes a bottleneck, and make sure your team’s processes are effective.

If you’re not already using Velocity, speak to one of our specialists and find out how you can get a more complete picture of the work being done on your team.

‍

Effective retrospectives are an important opportunity to provide guidance and invest in your team’s future. Though the discussions you have during retros will surface areas of friction or frustration, you may find that anecdotal evidence and gut feel don’t provide enough information. It can be hard to know how to help your team without a concrete means of gauging why developers are struggling (or succeeding).

Leverage data from a Software Engineering Intelligence platform likeVelocity, and you’ll be able to transform your retros into valuable opportunities for learning about team members and how they function within your engineering process. With this information, you’ll be able to run more effective retrospectives by offering relevant support, setting more specific goals, and ensuring your retro action items drive improvement.

Uncover Data-backed Insights

To get a sense for how your team is doing, use Velocity to determine if your engineers are trending up or down over time for the following PR-based metrics:

1. Cycle Time, which indicates the time between when the first commit is authored and when a PR is merged. This metric can be used as your team’s speedometer. Aim for a Cycle Time of under 2 days.
2. Time to Open, which is defined as the time between the earliest commit in a Pull Request and when the Pull Request is opened. Top-performing teams take less than half a day on average to open a PR.
3. Review Cycles, or the number of times a Pull Request has gone back and forth between the author and reviewer. Note that keeping your average Review Cycles below 1.3 keeps your team in the top 50% of engineering organizations.
4. Time to Merge, or the duration between when a Pull Request is opened and when it is merged.

Set Quantifiable Goals

To make your learning actionable, head to the Targets page, select your metric, and define a goal. In this case, we’ll aim to keep 90% of Code Reviews to one Cycle. When Review Cycles are high, it can mean one of three things:

• There are differing opinions about what “done” means.
• There’s misalignment around what kind of changes are expected to come out of the review process.
• There are conflicting ideas about how a solution should be implemented.

You can hover over the interactive column chart to see how often your team hit their goal. Selecting any week that didn’t meet an SLA will bring up a drill-down of all the underlying data points (commits, PRs, or reviews), so you can investigate the source of the issue.

In the above example, there are 12 PRs that you can dig into. Leverage this data to investigate places where your goals weren’t achieved and to offer guidance where necessary.

Data is Key to Driving Improvement with Effective Retrospectives

With the right data, you’ll be able to objectively assess your team’s progress and find concrete opportunities for advancement. Combine that with qualitative feedback from your team, and it’s possible to hold retros that effectively surface issues and point towards possible solutions.

Speak to one of our specialists about Velocity, and find out how data can help you drive progress on your engineering team.

‍

One of the first and most essential uses of Velocity is to cut through the noise and help managers identify the signals of stuck engineers. This enables management to eliminate unnecessary check-ins, while still having the ability to unblock engineers and boost engineering productivity by stepping in to help an engineer who might hesitate to raise their hand.

Velocity provides visibility into:

• Who on your team is blocked
• Whose work has been churning

Look out for four main behavioral patterns in Velocity to help address these concerns.

Engineers Who Haven’t Committed for a Long Time

A quick scan of the Activity tab will help you identify developers who aren’t checking in code.

Head into the Team360 report, select the Activity tab, and look for team members with few or no commits, represented as purple circles.

In the example above, Hecate hasn’t committed in a couple of days. This could indicate that she is:

• Working on a large chunk of work locally
• Stuck for whatever reason
• Tied up in non-engineering related projects

If you see a similar work pattern in your team’s Activity Log, you might want to check in to identify the bottleneck and help your developer get back on track.

Engineers Who Are Committing but Churning

Any developers who are committing, but not opening PRs, might be churning. Once again, go to the Activity tab in the Team360 report to see which engineers’ work appears to be blocked.

As noted in the key, Commits and Merged Commits are indicated by dark and light purple circles respectively and open PRs by light blue diamonds. You’ll want to look out for clusters with a high count of circles and a missing count of diamonds.

As you can see in the top row, Donalbain has been consistently committing code, but not opening any PRs.

This could be because he is:

• Committing a lot then planning to open one big PR (which isn’t ideal)
• Committing something, then redoing the work he just did for some reason
• Committing something, then heading off in a different direction and starting a new track of work

Take this opportunity to dive in and identify the issue.

Engineers Who Have Long-running PRs

Long-running PRs may indicate that an engineer is stuck on that particular unit of work, or that they’re multi-tasking, causing delays for multiple PRs.

Investigate all open and active PRs in the Pull Requests report. (Note that if you look at this report in the morning, it might look bare, since it automatically shows “today’s” activity. In this case, use the date picker to extend to yesterday, or the past two days to see what’s in progress).

To surface the oldest PRs, sort by age by clicking on the “AGE” header. Pay close attention to anything that’s been open for over 72 hours.

A PR might be long-running because:

• An engineer is having trouble with this PR and keeps adding on commits.
• It’s unclear whether this PR is done.
• An engineer’s PR hasn’t been picked up for review, either because it was overlooked or because it’s perceived as complex.
• An engineer is blocked on another third party.

Engineers Whose Work is Stuck in the Review Process

Finally, the Analytics tab is a good place to go to identify late-stage churn. You’ll want to run a query for Review Cycles, or the number of times a Pull Request has gone back and forth between the author and reviewer.

To obtain this report, select Review Cycles as your metric, and group by contributor. Run a query for the last week or two, and scroll to the bottom until you see the following bar graph visualization:

When Review Cycles are high, it may indicate:

• There are differing opinions about what “done” means.
• There’s misalignment around what kind of changes are expected to come out of the review process.
• There are conflicting ideas about how a solution should be implemented.

Boost Your Own Engineering Productivity with Data

With the right data, you can identify which of your team members are stuck right now, so you can help remove the roadblock and get things moving again.

If you want to boost engineering productivity, but don’t have a way to track and analyze your engineering metrics, reach out to find out more about our Software Engineering Intelligence platform, Velocity.

‍

The authors of Accelerate surveyed over 23,000 individuals in 2,000 distinct software companies to uncover the methodologies that set top-performing organizations apart. Their research suggests that “speed and stability are outcomes that enable each other” and that any software organization can measure and improve these outcomes.

The Continuous Delivery (CD) best practices they recommend, such as keeping batch size small, automating repetitive tasks and investing in quick issue detection, all perpetuate speed and quality while instilling a culture of continuous improvement on the team.

While most of the industry is actively adopting CD, few have set up any way to measure their progress. Concrete metrics, such as those found within Velocity, are a prerequisite to ensuring success in this transition.

In this guide, we’ve outlined how you can use Velocity to answer:

• How effectively is my team adopting CD practices?
• What’s hindering faster adoption?

Measure Improvement to Shipping Speed and Throughput

There are two success metrics that can represent how “continuously” your organization is shipping: speed (Cycle Time), and throughput (Pull Requests Merged or Deploy Volume).

Start by looking at the Analytics report to see how well you’ve been improving on one of those metrics. We recommend looking back at least 90 days.

Cycle Time, or the time between an engineer’s first commit to merging to production, should be trending down. Good coding habits, such as working in small batches, should keep changesets moving through the process with little friction, while CI/CD tooling should automate a lot of late engineering or QA work that frequently blocks merging to production.

Here’s how you can read this trend:

• Decreasing means you’re moving in the right direction. You’ve already adopted some Continuous Delivery best practices that have unblocked engineers and enabled them to move a single work in progress through the pipeline as quickly as possible.
• Flat is what you’d expect when you’re not in a state of transition. Typically, teams hit a local maximum with process efficiency when they’ve optimized as much as they can. If you’re in the middle of transitioning to CD, however, a flat Cycle Time is a bad thing. It means that even if you’ve changed some of the tooling or the messaging around how to ship software, this has not had the intended effect.
• Spiky indicates inconsistencies, and that your process is not delivering predictable results. You’ll want to take a closer look at days or weeks with particularly high Cycle Times to diagnose why work is getting stuck.
• Increasing is not a state you want to be in for a prolonged period of time, but can be normal during change management, as your team learns new practices and transitions to new tooling.

Alternatively, you can use Pull Requests Merged or Deploys as your yardstick. In this case, you can invert how you interpret results. Increasing throughput is desired, while flat and decreasing trends are a red flag that your team’s new practices are not yet yielding better outcomes.

Dive Into Key Health Metrics Team by Team

After understanding your overall engineering speed, you’ll want to investigate health metrics to find specific areas for improvement.

Velocity metrics can serve as strong proxies for the following Continuous Delivery practices:

• Good Coding Hygiene, which means working in small batch sets (PR size) and opening Pull Requests early (Time to Open).
• High Review Effectiveness, which means balancing review thoroughness (Review Coverage) and speed (Review Speed), while ensuring that comments lead to action (Review Influence).
• High Engineering Capacity, which means developers have enough time for engineering work (Weekly Coding Days).

In Velocity’s Compare report, you can look at these metrics across teams or for individuals to identify coaching opportunities or process improvements.

Click on a team to drill down and see performance on an individual basis:

Finally, get more context on a team or individual by seeing how they’ve performed on a specific metric, historically. The Analytics report lets you pick how far back you look and then drill down into any units of work that are dragging the average up or down.

Find the Biggest Opportunities for Improvement in Your Software Delivery Process

Now that you have all the context for how your team is working, create a mental model of your software delivery pipeline to see where a unit of work is most likely to get stuck. This will help you prioritize where you should start making optimizations.

We recommend breaking your process into three portions:

• Time to Open, or how long does development take.
• Review Speed, or how long does work sit before getting picked up for review.
• Time to Merge, or how long does the entire code review process take.

You can look at these three metrics side by side, by selecting them in the Analytics report and viewing them as bar graph clusters by week or by month.

With this data, you’ll be able to determine which area of your process is worth a closer look. If multiple stages need attention, we recommend starting with the one that comes earliest in your development pipeline, as improvements at early stages can have an impact downstream.

To dig into your own data and start measuring your progress towards Continuous Delivery, sign up for a demo of our Software Engineering Intelligence platform, Velocity.

‍

Engineering is an expensive department. And given that global circumstances are cinching purse strings even tighter, it’s never been more important for a CTO to track and optimize where the money is going.

In this 45-minute webinar, Code Climate Engineering Data Specialist, Bob Brown, talks about how to use data-driven insights to build a leaner process at scale and get the intended ROI from your organization.

Bob covers how to dig into engineering data from Velocity to answer questions like:

• How does our Time to Market compare to industry standards?
• Why does engineering typically get blocked? What process changes can we make to eliminate bottlenecks?
• What are the distinguishing characteristics of our strongest teams, and how can we replicate them at scale?

To uncover your own data-driven insights, request a demo of Velocity.

To create a culture that both motivates engineers and improves their ability to drive innovation, managers need a comprehensive picture of where their team members are succeeding and where they can improve.

In this 45-minute webinar, Code Climate Engineering Data Specialist, Nico Snyder, talks about how to create measurable improvement (20% or more) in Cycle Time and team performance with holistic visibility into engineering work.

Nico covers how to dig into data from Velocity’s newest features to answer questions like:

• How are teams or developers performing? How do we improve?
• How do we drive continuous improvement across engineering processes?
• How do we communicate engineering success both within the department and to others in the organization?

If you’re not already using Velocity, you can request a demo here.

In this time of global and economic uncertainty, it’s never been more important to have a quick way of knowing which engineering processes are working and which are broken.

And while it can be tempting to focus on bottlenecks and struggling team members, it can be even more useful to look at the practices, behaviors, and culture of your strongest teams.

This post runs through a framework for using Velocity, our Software Engineering Intelligence (SEI)platform, to identify and scale the most successful habits of high performing teams.

Finding the Top 5% of Your Organization

While all engineering organizations might define success slightly differently, there are two metrics within Velocity that indicate an extremely proficient engineering team:

• Throughput, measured in deploys or PRs merged, which indicates how much your developers are getting done.
• Cycle Time, measured in hours from first commit to when a PR is merged to production, which indicates how fast your developers are getting that work done.

In Velocity’s Compare report, you can select both of these metrics and compare them across your organization to identify the teams that are merging the most the fastest.

Once you’ve identified your strongest teams using success metrics like Throughput and Cycle Time, you’ll want to dig into what has made them successful. For this, you’ll need different, diagnostic metrics.

Identifying the Best Practices of Your Strongest Engineering Team

You can think of your software delivery process in three phases:

• Time to Open, or how long does development take.
• Review Speed, or how long does work sit before getting picked up for review.
• Time to Merge, or how long does the entire code review process take.

Typically, a strong engineering team will move faster in one of these stages.

Velocity makes it easy to look at these three metrics side-by-side. You can view them as bar graph clusters by week or by month.

Or, you can view these metrics by team.

Here, we can see that the top team — Gryffindor — is most distinguished by their extremely fast Review Speed. Although they have a long Time to Open and Time to Merge, this isn’t remarkable when looking at the other teams. The other teams (especially the Hogwarts team) frequently had work stuck in the review process.

Pair your quantitative analysis with qualitative information, and speak to the members of the Gryffindor team. Find out what makes their review process different from the other teams’ processes, and think about ways the other teams can apply those learnings.

DORA metrics are also useful to identify high performing teams within the organization.

Creating a Blueprint For Your Entire Organization

Now that you’ve identified your top-performing teams and their defining characteristics, you can create a blueprint for better processes across your organization.

One of our customers, a health data analytics solution,used Velocity following a Series B funding round to level-up the way they coached engineers at scale.

Their VP of Engineering had been brought on to help build out the engineering department. But after getting to know his teams, he realized that there wasn’t any consistency in how and when teams shipped features to end-customers.

The VP of Engineering worked with his engineering leads and product managers to identify agile practices that worked for his team, then shared them org-wide. Together, they created documentation and set up onboarding and mentoring around encouraging healthy coding habits at scale.

With stronger processes in place, the team was able to increase PR throughput 64%. With objective data and learnings from your highest performing teams, you’ll be able to replicate successful practices across your organization, and help boost productivity at scale.

Find out how Code Climate Velocity can help your team improve Cycle Time and PR Throughput by booking a demo.

‍