Tracking Vulnerability Fixes to Production

by Michael Werneburg
on 2017.09.18

You are here:
Risk topics
» Risk topics blog
April, 2018

March, 2018
· the planning fallacy

February, 2018
· Valentine's day vm backup plan

November, 2017
· the unsafe workplace and the body's response

October, 2017
· ISACA article is live

September, 2017
· published
· the Equifax breach
· Tracking Vulnerability Fixes to Production

August, 2017
· evaluating third party cyber risk

July, 2017
· getting it wrong with R
· de-identifying health information
· that's a lot of tracking!

June, 2017
· gaming Google news
· privacy in this day and age
· another record breach
· writing an industry standard
· ISACA article accepted

May, 2017
· Covey time-management quadrants
· safe harbor de-identification of health data
· an ISACA article

April, 2017
· my guide on managing third party risk
· PMP for five years
· metrics that matter
· 720 reads in 48 hours
· I lost my job

March, 2017
· farewell, SIRA board
· the message and the medium
· an interesting take on consulting


As an IT auditor at a software company, I discovered that security vulnerabilities in our bespoke product had not been getting released to clients on a timely basis. We had been doing penetration tests for years, but obtaining the penetration test report had not translated to the fixes being released to the users. Our clients remained exposed to known vulnerabilities, a situation that meant my employer was assuming all potential liability for the situation.

There were, it turned out, many things that slowed delivery of the fixes. Some factors were organizational and some were technical. I address the organizational challenges of client resistance and lack of internal commitment in my recent Journal article. But I will offer an additional insight for readers of Practically Speaking on overcoming technical complexities in patching a bespoke software product.

The technical complexities in the environment were nearly endless. Some penetration test findings applied only to certain versions of the software. Some fixes were beyond the capabilities of the development platform and required extensive software workarounds. Other fixes required a minimum browser level that was beyond a client’s reach. Sometimes a peculiarity of the client environment prevented one fix or another from working, e.g., a home-brewed single sign on or bespoke antivirus configurations could hamper the roll-out of bug fixes. These complexities prevented the patch bundle from each year’s penetration test report from being deployed into the production environment.

The problem turned out to be both the vendor and the client believing each year’s findings could be resolved via a monolithic patch. By bundling the fixes, we greatly increased the likelihood that some complexity would render the patch unsuitable or undeliverable to a given client. Working with the developers, we devised a matrix that broke down each year’s penetration test results, with a row for each distinct finding in each report. We worked with the product owners to understand which finding applied to which version of the software. When a software fix was created, we recorded the version control branch number for the fix against the relevant finding. When a release was scheduled for a client, we worked with the project management office to ensure that the relevant fixes got scheduled.

It was messy and labor-intensive, but it worked. Supported by a strong version control system and a documented package release program, a reliable program for tracking penetration test fixes to production was put into effect. In time, we eradicated client exposure to known vulnerabilities, resolved our employer’s potential liability and were ready for each year’s fresh batch of findings.

This article was originally submitted to ISACA's "Practically Speaking" blog, here.

big list