Predicting quality: Best question to ask your software development vendor

When choosing a vendor to outsource software development, one of the top concerns is quality. The Systems Sciences Institute at IBM reports, “The cost to fix an error found after product release was four to five times as much as one uncovered during design, and up to 100 times more than one identified in the maintenance phase (source: here).”

There are many ways to evaluate whether a vendor will build to your quality standards. A leading indicator is the answer to this question:

“Do you perform continuous code quality inspection?”

If yes, how and with what tools? In our experience, although tools like SonarQube do not guarantee high quality, on-time delivery and overall reduced project costs, they do help us come through on our promises.

Why we use a code quality inspection platform

With a code analysis and continuous inspection platform you can expect quantifiable risk reduction for your project:

  • Real time, automatic code inspection (as opposed to ad-hoc code reviews) are one of the most effective means to identify and remove defects. Typically these tools will identify 60-70% of all defects and they achieve this at the earliest possible stage in the development process (source: here). Problems are fixed as they occur; the programmer cannot move beyond a bug until it’s fixed.
  • Reduces the technical debt that your project will accumulate, preventing the need of massive refactoring.
  • A good platform will include reporting that analyzes a project’s historical evolution from the quality point of view. Improvements on the caliber of the software will become something measurable, instead of an abstraction.

Why we like SonarQube

We evaluated several options and decided to implement SonarQube for continuous code inspection (among other functions not discussed here). It has robust features and we use them all. Here are a few:

  • The platform enables our teams to address many areas of code including duplicated code, coding standards, unit tests, complex code, potential bugs, comments, design, and architecture. It supports a holistic agenda of clean code.
  • Automatic feedback is provided to the development team in real time. This empowers them to find the best approach to fix the issues on quality. This is ideal, as they are the experts on the product that they are creating. External audits will lack the finesse that being involved with the code brings to the table. A good platform can even define a set of minimum requirements, creating a sort of “quality gated check-in” that will reject code not up to the standard.
  • Gives a moment-in-time snapshot of our code quality today:
    • Trends of lagging (what’s already gone wrong).
    • Leading trends (what’s likely to go wrong in the future) quality indicators.
    • Identifies what we’re doing well overall, and whether we’re getting better or worse.
  • Its reports and multiple views address source code from different perspectives. The reports are used by core developers and programmers as well as project managers and higher managerial levels.
    • Reports include density of bugs, not just raw numbers, e.g., how many bugs per x lines of code?
    • Provides metrics and statistics about our code and translates these values to real business values such as risk and technical debt.


What else we do to ensure excellent quality

The continuous inspection platform, in our experience, is just one feature of a comprehensive approach to building quality code. We also use other effective strategic initiatives and operational practices.

For instance, we have an internal component library for code and modules that can be re-applied across projects and teams. Our library is specific to components like email, login, and date/time handling modules. Components are built optimally to begin with, then they are tested and proven across multiple projects. When you can, why not start with clean, tested, reliable code?

There are many other development and testing strategies we tailor and apply on a case-by-case basis. Not all strategies are a good fit for all projects. Here are a few:

  • Continuous integration and delivery (CI/CD) is not just about speed; it also supports high quality.
  • Test automation like CI/CD does not just enable faster development cycles; it also creates a safety net.
  • Allotting time and space to build a sound testing strategy with the appropriate depth and breadth of testing.
  • Building a comprehensive test environment.
  • And many more…

Beyond tools and processes, we designate what we call “squad leaders” who are responsible for multiple project teams. They track how teams are performing in terms of quality, velocity, and innovation. Squad leaders quantify quality and apply improvements across teams and projects.

How you’ll know if you get the right answer

It’s the discussion started by your question, “Do you perform continuous code quality inspection,” that will shed light on your vendor’s ability to deliver on quality. There are many viable tools and methods out there. The most important point of validation is that your vendor has proven tools in place and in motion.

facebook twitter linkdin