You Can’t Set Effective Engineering Goals Without Data

The most popular goal-setting frameworks — FAST goals, SMART goals, OKRs — break when it comes to the engineering department.

The cornerstone of every framework is specificity, but most engineering departments run on benchmarks and estimations that are vague, subjective, or constantly in flux. Common project measurements, like story points or t-shirt sizes, lack objectivity. And while lists of completed features are definite cause for celebration, they’re too broad to provide insight into the actual work being done by the engineering department.

To get specific, you need data. With information from your VCS, it’s possible to gain visibility into your development workflow and spot opportunities for advancement.

Subjective measurements, like t-shirt sizes and story points, can be replaced with objective metrics that are comparable across projects and teams. For example:

• PR Size — PR Size is a count of the number of lines of codes changed, added, or removed in a given Pull Request, and it reveals how incrementally people are working. Smaller PRs are more likely to move quickly through the review process.
• Review Cycles — Review Cycles is a count of the number of times a PR is passed back and forth between author and reviewer. It can help you diagnose certain misalignments in the Code Review process, or identify developers struggling with the codebase.
• PR Cycle Time — Cycle Time, which measures the time from the first commit in a Pull Request to the time it’s deployed, is a powerful proxy for engineering speed.

Objective, consistent data points like these make it possible to assess your team’s baseline and measure progress across teams and projects. They also allow you to set actionable goals for software developers and drive high performance on your team — average teams that pair targets with relevant metrics are able to improve enough to reach the top quartile of performance.

Data and the FAST Framework

To illustrate the critical role data plays in goal-setting, let’s take a look at the MIT Sloan Management Review’s research-backed FAST framework. FAST goals are:

• Frequently Discussed — Goals should be part of reviews and other discussions to help the team prioritize and remained focused.
• Ambitious — Goals should not be simple to achieve, though they shouldn’t be impossible either.
• Specific — Metrics and milestones should be used to define success and measure progress.
• Transparent — Goals — and progress towards them — should be shared publicly.

With metrics based on data about your engineering workflow — data that already exists inside your VCS — engineering departments can set goals that meet all four criteria.

Data is the Foundation for Productive Discussions

It’s relatively easy to bring up your team’s goals frequently, but constant reminders don’t help your team understand how  to succeed. The most productive discussions will do more than just update your team on their progress — they’ll facilitate the sharing and refining of strategies. This will allow your team members to work together to get closer to their goal. Data can provide a concrete foundation for those discussions.

Let’s say a cohort of junior engineers is determined to build healthier coding habits, and set a goal to open small Pull Requests, with fewer than 150 lines of code added, changed, or removed. Tracking Pull Request Size will make it possible to visualize the team’s progress towards that goal while allowing you to isolate specific units of work that have fallen short.

You can bring some of those large Pull Requests to stand-ups or one-on-ones, and use them to discuss tactics for breaking up similar PRs in the future. With concrete data as a starting point, your team’s discussion will yield more productive, actionable takeaways.

Data Allows Precise Tracking, to Keep Engineering Goals Ambitious

If your targets are too easily attainable, you won’t be driving real progress. If they’re unrealistic, your team may be frustrated with their inability to reach them, and morale may suffer.

The way you track your engineering goals is just as important as the goals themselves — if you’re not careful, inaccurate tracking methods can obscure your team’s real progress, leading them to get discouraged or set less ambitious goals in the future.

Let’s say your team feels like Code Review is taking too long. You might aim to bring down your Review Cycles, or the number of times a PR is passed back and forth between author and reviewer. If you set a target of 1 Review Cycle, you could then track progress towards that goal by calculating the average number of Review Cycles for all PRs in a given time period. It’s a common approach, but if you have an outlier in your data — maybe one particularly complicated PR required a lot of back and forth in Code Review — that outlier will throw off your average. This can make it difficult to see the overall trend, and inaccurately suggest inefficiencies in the review process.

Instead of using averages, we recommend tracking progress as a percentage, OKR-style. So, rather than aiming to bring your average Review Cycles down to 1, you might aim to hit that number for 80% of Pull Requests.

Your team’s goal should not be perfection, and there are often good reasons for falling short of a target some of the time. Percentage-style goals will ensure that your team can set ambitious goals and still see progress.

Data Makes Transparency Possible

Engineering metrics allow leaders to compile detailed, accurate progress reports that they can feel comfortable sharing with the team, or with stakeholders.

Depending on the goal, it may make sense to use some discretion here. Though it’s often useful to share team goals with the rest of the department or organization, there are situations where it might be appropriate to keep that information within a smaller circle. For instance, if you’re working with an individual contributor on a specific goal, their progress might be best kept between the two of you, or within the smaller circle of your team.

Data Drives High Performance

Effective goal-setting will help you kick-off the Virtuous Circle of Software Delivery, in which progress motivates further improvements. That cycle is key to fostering a culture of Continuous Improvement — and becoming a high-performance engineering team.

As engineers start to make measurable progress towards their goals, they’ll reap the benefits of those improvements, which will encourage them to find new ways to do better. Involve your team in the goal-setting process whenever possible, and you’ll empower and motivate your team to be invested in their own improvement and professional growth.