Case Studies

OpenTable is a renowned leader in restaurant tech, with products that anticipate the ever-evolving needs of both restaurants and diners. Its website, opentable.com — which helps restaurants fill more than 1.6 billion seats a year — is supported by globally distributed teams of engineers.

In 2019, the site was rearchitected, and OpenTable was faced with the challenge of bringing together many teams as the commercial site was rebuilt as a monolith. During this project, Code Climate supported engineering leaders in gaining further visibility into their engineering processes, which enabled them to bring consistency and reliability to development — ultimately empowering their teams to collaborate more effectively.

Reliable Engineering Management with Code Climate

John Nolan, Senior Director of Engineering at OpenTable, said their approach to rearchitecting was essential for consolidating ideas and ensuring consistency across many globally distributed teams. Code Climate helped them to better measure effectiveness and more easily identify bottlenecks that might hold up progress.

After learning about the benefits of Software Engineering Intelligence platforms at a Code Climate event, Nolan knew Code Climate could provide the insight OpenTable needed to more formally measure engineering impact.

Code Climate delivered insight into engineering processes so that the business could better understand if their changes were effective. For instance, he was able to see the changes over time in Code Climate for metrics like Pull Request (PR) Throughput and Cycle Time.

The goal was to work with Code Climate and identify opportunities to help the teams work together more effectively. Ultimately, they would support improvements in efficiency and productivity as they improved communication and collaboration. In the first year, PR Throughput per Contributor increased by 40%. With more people working more productively, the team realized gains in output.

“Code Climate gave us valuable data to see that the things we were doing were solving the problems that we needed to,” Nolan said. “We saw metrics get better because we were building the right things in the right way.”

Critical Insight for Engineering Leadership

Nolan is in a new role and another step away from the code, and he finds the insight from Code Climate valuable in understanding the complexities of the engineering organization at OpenTable. Both standard and custom reports have proven helpful in identifying activity patterns and maintaining development speed across their widely distributed teams. He said that, while individual engineering managers often know about their own teams’ activities, the visibility Code Climate provides is useful in identifying trends and understanding team-by-team activity.

OpenTable has expanded Code Climate use and has now deployed the platform across all product engineering teams throughout its global engineering organization. Because Code Climate built its solution with enterprises in mind, with an unique ability to ingest and normalize data with reliability and speed at scale, supporting the engineering team at OpenTable to better visualize metrics and understand engineering impact and team health.

“It's invaluable to have a glance at teams' health and have the data necessary to understand where we need to address any issues,” he said.

After OpenTable again expanded their partnership with Code Climate, it has further increased visibility and supported leaders in identifying more opportunities to improve throughput and move faster as an organization. With that information, it was possible for the organization to make changes that kept PR Throughput high while decreasing Cycle Time by 57% and Time to Merge by 23%.

Code Climate supports OpenTable with the data they need. With Code Climate, they have the visibility and understanding necessary to maintain consistency across teams, ensure efficiency, and cultivate a culture of engineering excellence throughout the organization.

Key Highlights

• In the first year, PR Throughput per contributor increased by 40%
• In 2023, OpenTable saw a 57% decrease in Cycle Time
• In 2023, they saw a 23% decrease in Time to Merge

To find out how Code Climate can support you in identifying opportunities for improvement, speak to an expert.

In the midst of pandemic lockdowns, VTS, a leading provider of commercial real estate technology, was in a period of rapid growth. In addition to aggressive hiring, VTS grew through acquisitions, adding Rise Buildings and Lane to its portfolio. Soon after onboarding, they discovered the new teams had less effective SDLC processes, which caused Cycle Time to trend toward 80 hours — nearly double the average Cycle Time of the core VTS team.

Engineering leadership leaned heavily on Code Climate as they incorporated the new teams and integrated the new products into the VTS platform. They leveraged Code Climate's partnership to investigate bottlenecks, and discovered the teams were spending Cycle Time resolving issues and needed more efficient workflows.

Fostering an Elite Engineering Culture

Being customer-obsessed and striving for excellence are the core tenets are the foundation of VTS culture. And for engineering, these values drive an ambitious vision of producing elite engineering talent who innovate to serve customers, achieve business outcomes, and positively impact the broader tech industry.

With more than 20 teams and 200-plus engineers, VTS fosters a high-caliber engineering culture built on mutual trust. They have collectively embraced a vision of engineering excellence, and they leverage Code Climate to measure proficiency and success, surface bottlenecks, and actively explore ways to improve. Code Climate's solution delivers end-to-end visibility into the entire development pipeline, which is crucial for tracking engineering progress and achieving OKRs with a large, distributed team.

Prashanth Sanagavarapu, Head of Platform Engineering at VTS, said without these insights, every decision would be a shot in the dark. “As a manager, my worst nightmare is running blind. I need to make decisions based on data and facts, and Code Climate provides exactly what we need.”

Metrics That Matter

For VTS, Code Climate provides visibility into the metrics that matter, and it is more intuitive and robust than what is built into other engineering tools. For example, Jira reporting was inadequate because it lacked context, and engineering leaders couldn’t compare metrics to industry standards.

“An ops team may close 100 tickets, but what does that mean? Someone has to go into each ticket and read the description to understand what is happening, and that just isn’t sustainable,” said Sanagavarapu.

Code Climate allows them to analyze factors like Pull Request (PR) size, frequency, and time to close, enabling them to optimize workflows to consistently deliver incremental value and maintain engineering velocity. Sanagavarapu said he learns quite a lot through the platform: “It’s a fact tool for me. I can see the trends of what is working and what isn’t working for a particular squad and correlate it back to sprint retros.”

Cycle Time is the north star metric at VTS. Measuring Cycle Time every two weeks with Code Climate provides visibility into how fast they are shipping, both organization-wide and at the team level, and it enables them to quickly see when fluctuations occur. Then, within the platform, they can easily drill down to identify choke points and dependencies that may be impacting performance. Understanding if the Cycle Time went up due to outages, open RFCs, or a change in personnel helps leaders to understand trends and better allocate resources to ensure their teams have what they need to be successful.

Sanagavarapu said the ability to drill down to the individual contributor level is very impactful because it allows you to diagnose problems at any level and scale. Since partnering with Code Climate, they have improved Cycle Time by 30% and doubled their deployment frequency.

“Our average Cycle Time tends to be around 35 hours with 48 hours as our max threshold. When we exceed that, we know there is something going on. If it’s not a holiday or another known factor, we can dig to discover and understand the problem and who is being impacted — then, we can solve it.”

Maintaining Excellence During Rapid Growth

Enhanced visibility has been crucial for engineering leadership over the past two years, with company growth accelerating during challenging pandemic lockdowns. Sanagavarapu said more than 60% of the company’s 600-plus employees joined during this time, most of whom were engineers.

Infrastructure stability was a big challenge, so they worked to reduce the number of incidents so that dev teams could spend more time on value-add work. When they discovered a lag time in PRs due to time zone differences, they changed their workflows to reduce the time for feedback and better manage resources across teams. They also added in more test cycles so that rework happened less frequently. Now, the entire engineering organization maintains Cycle time under its 48-hour threshold.

“Code Climate provided insights that helped us accelerate integrating those teams into our culture and processes more quickly and effectively,” Sanagavarapu said.

Measuring Impact

VTS leverages Code Climate's solution to track and quantify impact at all levels. Engineering leadership can measure business impact by translating metrics into stories that show how engineering delivers value. They can understand how quickly teams deliver new features that are important for customers and compare the time spent on new feature work to rework time to ensure engineering time and resources are optimized.

Code Climate surfaces important trends that help engineering managers better understand the impact of process and workflow changes on developer experience. They can drill down to the developer level to diagnose issues and determine what might be consuming a squad’s time. Visibility into engineering capacity helps planning for major initiatives, allowing them to best leverage internal resources and balance workloads with external contractors.

As VTS works continuously to innovate, evolve, and achieve both engineering and business milestones, the insights derived from Code Climate are invaluable, Sanagavarapu explained. “Code Climate is not a reporting tool. It’s the heart of engineering excellence.”

Request a consultation to learn how to maximize engineering impact.  

We sat down with Jake Hiller, Head of Engineering at Codecademy, to find out how they use Code Climate’s  automated code review tool, Quality, to maintain their quality standards while rapidly growing their engineering team.

INDUSTRY Education

EMPLOYEES 50+

DEVELOPERS 20+

LOCATION Manhattan, NY

LANGUAGES Ruby, JavaScript, SCSS

CUSTOMER Since May 2013

"Code Climate keeps our process for creating PRs really low-effort so we can quickly test ideas and ship sooner."

Why Code Climate Quality

Like many rapidly developing teams, Codecademy was running into growing pains for both engineering onboarding and code review. They had tried using local analysis tools but found them cumbersome to integrate as development environments varied across the team.

With an engineering workflow centered around pull request reviews, and a desire to reduce friction in committing and testing code, they needed a solution that would optimize their pull request review process and enable new team members to quickly become productive.

Codecademy had been using Quality for their Ruby stack since 2013. When Head of Engineering, Jake Hiller, joined in early 2015, he saw an opportunity to alleviate their code review and onboarding issues by rolling it out to the whole team.

“We wanted to avoid anything that blocks engineers from committing and testing code. Other solutions that use pre-commit hooks are invasive to both experimentation and the creative process. Code Climate’s flexibility helps us maintain rules that are tailored to our team and codebase, while offering standard maintainability measurements. Plus it enables us to defer checks until code is ready to be reviewed, so we can quickly test ideas and ship sooner.”

“Code Climate helps us transfer knowledge to new engineers – like our coding standards, why we’ve made decisions over time, and why we’ve chosen certain structures and patterns."

Increased speed and quality

Since rolling out to the whole team, Hiller says Codecademy has seen an improvement in the quality of their code reviews and the ease with which new team members get up to speed.

“Code Climate helps us transfer knowledge to new engineers – like our coding standards, why we’ve made decisions over time, and why we’ve chosen certain structures and patterns. New engineers can look through the Code Climate issues in their PR, ask questions, and propose changes and suggestions to the team.

“It’s also increased the speed and quality of our pull request reviews. We’ve been able to spend more time discussing the important functional aspects of our code, and less time debating smaller issues. There are a lot of issues that can’t be fixed with an auto formatter, which is where Code Climate will always be really helpful for our team.”

About Codecademy

Codecademy was founded in 2011 as an immersive online platform for learning to code in a fun, interactive, and accessible way. They’ve helped 45 million people learn how to code, covering a wide variety of programming languages, frameworks, and larger topics like Data Analysis and Web Development. Their recently released Pro and Pro Intensive products provide users with more hands on support and practice material to help them learn the skills they need to find jobs.

If you’ve spent any amount of time working with software professionally, chances are you’ve experienced the vicious cycle of development (if you haven’t, take a moment to count your blessings then read on for a cautionary tale):

High pressure situations place strain on developers, which leads to sloppy work, resulting in software being delivered late and broken – when it’s delivered at all.

Instead of being pressure-driven, as in the cycle pictured above, we’re aiming for a “virtuous cycle” which focuses on quality:

A focus on quality leaves individual contributors feeling satisfied, which makes them more engaged. That engagement leads to a sense of responsibility over the product and the code. A sense of responsibility and ownership leads to higher quality work, and the cycle starts over again.

Here at Code Climate we have experience as software engineers, engineering managers, product owners, and startup founders – and in all those roles we’ve seen evidence that a commitment to code quality has amazing benefits. Over time your codebase will become healthier, making it easier to change, more fun to work with, and your features will be easier to implement. As a result your code will be more reliable when it runs, bringing benefits to your business. Sounds good, right?

Unfortunately, breaking out of a vicious cycle is challenging because the cycle feeds on itself and doesn’t allow much time for the work needed to establish good practices. But while there is work involved, it’s definitely not impossible, and good tools and advice can help you beat the clock and replace your vicious cycle with a culture of code quality.

We’ve identified steps you can take to move toward a sustainable culture of code quality, no matter what kind of code you’re working with:

• Codify your best practices
• Automate, automate, automate
• Get the right info to the right people
• Measure your progress
• Encourage participation and ownership

Update: See our CEO and Co-Founder Bryan Helmkamp introducing the Code Climate Quality Platform!

Today, we’re thrilled to launch the Code Climate Quality Platform − the first open, extensible platform for all types of static analysis.

We’ve come a long way since we started building a static analysis tool for Rubyists. Today, we’re fortunate enough to help 50,000 developers analyze about 700BN lines of code every weekday. As we’ve grown, we’ve seen that clear, actionable static analysis leads to healthier code and happier developers. Our new platform brings those benefits to every team, regardless of the technologies they use, so that they can focus on shipping quality software.

What does this mean exactly? First, we’re open sourcing our analysis tools, including the engines and algorithms we use to evaluate code. If you have a running Docker host, using Code Climate on your command line is as easy as:

boot2docker up && `boot2docker shellinit`  brew tap codeclimate/formulae  brew install codeclimate  

We’re also enabling anyone to write static analysis engines that run on our servers by following a simple specification. No longer will you have to wait for us to provide support for your programming languages, frameworks and libraries of choice. Finally, using our new Code Climate CLI, you can now run any Code Climate-compatible static analysis on your laptop – for free.

Let’s look at each of these in turn…

Open Source

We’re releasing the static analysis engines that power the new Code Climate Quality Platform, and going forward, all of our static analysis code will be published under Open Source licenses. Code Climate has always provided free analysis to Open Source projects, and this continues to deepen our commitment to, and participation in, the OSS community.

Code Climate has always stood on the shoulders of giants in depending on Open Source libraries for the bulk of our static analysis algorithms. We would not exist today if not for the great work of people like Ryan Davis (Flog, Flay) and Justin Collins (Brakeman) that demonstrated the value of static analysis and allowed us to quickly bring it to developers.

Open Source static analysis means you can dig in and understand exactly how your code is being evaluated. And customizing static analysis algorithms becomes as simple as clicking the “Fork” button on GitHub…

Extensible

With the release of our new Code Climate Engine Specification, anyone can write static analysis engines that run on our servers. Code Climate’s set of officially supported languages can be augmented with community supported static analysis engines so that you can get confidence in your code, regardless of your choice of technology. For years, our most common type of feature request has been, “Can you add support for X?”, where X is a programming language, version of a language, or sometimes a framework. We’ve always wanted Code Climate to work for all developers, but in the past we were limited by the effort required to add new languages.

We believe that you shouldn’t have to wait for a vendor to add support for the languages you love, so are finally removing ourselves as the bottleneck to new static analysis features. Anyone who is interested in using Code Climate with their favorite languages, frameworks and libraries is free to build an engine to do so. Results from engines retain all the benefits of the Code Climate product, including automatic analysis of every Pull Request, comparison views, ratings/GPAs, and notifications. Of course, there’s already a vibrant community of OSS static analysis tools, and we’re excited to see what people are able to build and integrate. It’s really astounding how simple it is to build a Code Climate engine, as we’ve watched developers build functioning analysis from scratch in a matter of a couple hours.

If you want to give it a try, join our new Developer Program and we’ll be there to guide you along the way.

Run Anywhere

In addition to the spec, we’re also releasing the Code Climate CLI, which makes it easy to get static analysis results (both from Code Climate’s official engines and community-supported engines) in one clear, unified report right on your laptop. When you’re ready, you can load your repository into codeclimate.com and we’ll automatically apply the exact same configuration you use locally to analyze every commit and Pull Request, making the results available to your entire team.

To make static analysis truly ubiquitous, we realized it was not enough to support a wide variety of tools, we need to make it trivial to run these tools anywhere the developer is working. You shouldn’t have to wait to push your source code to a remote server to get clear, actionable static analysis results. Now it’s possible to easily run the same static analysis we run on our servers on your command line.

New Static Analysis Engines

We’re fortunate to be partnering with creators of two prominent Open Source projects who understand the value of static analysis in ensuring healthy code, Tom Dale from Ember.js and Laurent Sansonetti from RubyMotion. Here’s what they have to say about the Code Climate Quality Platform:

“The Ember community takes good tooling seriously. I’m excited to partner with Code Climate and bring their service to our users because better static analysis means better Ember apps.”

– Tom Dale from Ember.js

“HipByte is excited to partner with CodeClimate to provide clear and reliable code reviews for RubyMotion projects. RubyMotion allows Ruby developers to write cross-platform apps for iOS and Android by leveraging the native platform APIs, and a tool for static analysis means fewer crashes and more reliable applications.”

– Laurent Sansonetti from RubyMotion

Accordingly, we’re releasing an Ember.js engine that brings the Ember Watson tool to all Code Climate users, and a set of custom checks for RubyMotion that will help ensure that your application code is reliable.

In addition, we’re proud to release eight new static analysis engines that you can start using with Code Climate today:

• Gofmt, Govet and Golint – The official style, bug and lint checkers from the Go team
• CoffeeLint – Style checking for the CoffeeScript dialect of JavaScript
• CSSLint – Style checking for all CSS stylesheets
• Rubocop – Style and quality checks for Ruby code (including support for RRuby 2.2+)
• ESLint – Linting and style checking for your modern EcmaScript/JavaScript code
• Bundler Audit – Find security vulnerabilities in your Ruby dependencies
• PHP CodeSniffer – Style checking for PHP

Healthy Code Ships Faster

In many ways, our new platform is a culmination of the experience we’ve gained over the past four years building and operating the most popular static analysis app. We’re excited to bring this to you, and look forward to continuing to bring you the best tools to ship better code, faster. If you want to try out the new Code Climate, download our CLI to get started.

GORUCO 2012, NYC’s premier Ruby conference, is around the corner and it’s going to be great. (I may be biased, as I help organize the conference, but it’s still true!) We have some fantastic speakers this year, and I’m especially looking forward to three talks that focus on different aspects of code quality:

• “Maintaining Balance while Reducing Duplication – Part 2” by David Chelimsky. This is a follow up to David’s excellent talk from RubyConf 2010 that explored the DRY Principle (Don’t Repeat Yourself) in depth and illustrated how it can be misapplied. The inherent tensions between different OO design principles is particularly interesting to me.
• “Sensible Testing” by Justin Leitgeb. Justin really impressed me with his lightning talk at a recent NYC.rb about Ruby mixins and their problems. With this talk, I think he’ll do a great job exploring testing strategies for Rails apps in a practical, nuanced way. I wouldn’t be surprised if the Testing Pyramid comes up, which more developers should take to heart. TATFT is great mnemonic, but applying it well over a large codebase and a long period of time requires a great deal of consideration.
• “Hexagonal Rails” by Matt Wynne. Alistair Cockburn has been talking about Hexagonal Architecture (which he now calls “Ports and Adapters”) for years as a way to structure applications for better maintainability. It’s always been enticing to me, but it’s never been clear how to apply it to a Rails application. Matt has started exploring just that on his blog and I’m thrilled he’ll be giving a full-length talk on it at GORUCO. Personally, I hope he covers the implications on the model layer at least as much as controllers/views, and perhaps how this architecture might relate to the Anemic Domain Model antipattern described by Martin Fowler. (If not, I’ll ask him during Q&A. Hah.)

And before you ask, yes, the good folks at Confreaks will be recording the talks and we hope to have them posted within a few weeks of the conference (if not sooner).

If you’re attending GORUCO this year, definitely stop me and say “Hello!” (Either during the day, or aboard the ::ahem:: yacht during the evening.)

Full disclosure: I had the honor of being a member of the GORUCO Program Committee this year that selected the speakers, so it may not be a complete coincidence that my interests are well represented. 🙂