Archive for Peter Lidstrom

Counting Vulnerabilities and SDL – No One Hears the Tree Falling

Posted in security with tags , , , , on April 20, 2008 by hellnbak

Those of you who have known me since early on in my career as an “infosec guy” know that I have always been very critical of Microsoft.  Between the days of Information Anarchy, my old school rants and the “AusCERT Incident” I have always been ready to pounce on the boys in Redmond when I felt that they were being shady or crossing a line.  That said, those that know me also know that I am a reasonable person and won’t jump on someone just for the sake of jumping on them.

Sadly, I am a sheep, and like most of you, I am on Facebook.  One of my Facebook (and professional/real life) friends is Ryan Naraine over at eWeek.  I noticed his status late this week was – “Dave Litchfield is also wrong –

So, wondering what the next Information Security drama was I headed over to Dave’s BLOG (which I didn’t know existed till now and I highly recommend it) and gave the post a read.  It seems that the hoopla is all about the Secure Development Lifecycle and counting vulnerabilities.  Dave’s post was simply a response to this post by Peter Lindstrom who is apparently some sort of analyst.


I have been on all ends of the security puzzle.  Two of those many ends are the vendor and, of course, the vulnerability researcher.  Based on my experiences I feel that I have a good insight in to not only the vulnerability reporting process but also the Security Development Lifecycle.  In fact, with all due respect to Dave and Peter, I think I have a much clearer insight than the both of them.

Let me say this right now. SDL works. 

You name the major vendor – Microsoft, Apple, IBM, HP, VMWare, etc… I have worked with them on getting vulnerabilities fixed and I have been doing so at different times during my 15 year career.  I was sitting at the lunch table in the 90s when a Sr. Microsoft executive looked at a group of researchers and said “If I had my way people like you would be in jail”.  I was there when Scott Culp during his MSRC days called every Security Researcher a Terrorist.  I have also watched the vendor everyone loves to hate — Microsoft — take a complete 180 on their security philosophy. 

So before you comment calling me a fan boy let me say this.  Is Microsoft perfect with how they deal with security vulnerabilities and bugs?  No, they are far from it.  But, they are better than the majority of the vendors I have had to deal with. 

Apparently what caused the drama amongst security bloggers was this comment:

Microsoft has systematically hired and/or contracted with every one of their most vocal critics (and most seasoned bugfinders) to do the work behind the scenes and they don’t count those vulns!

Dave Litchfield, who has a sucessful company in the UK, took offense to this remark.  To be honest, I don’t blame him.  Dave is best known for being the guy that completely destroyed Oracle Security as well as a few really good Microsoft bugs found by him and his team over at NGSS.  It is also common knowledge that NGSS does a bit of work for Microsoft on MS SQL that happens to fall into Microsoft’s SDL process.

Peter on the other hand, with all due respect and to the best of my knowledge, has never found, reported, or fixed a security vulnerability before.  In fact, even on his own blog he says the following:

“I have very little security background or knowledge”

Don’t get me wrong.  I have met Peter, once, and he seemed like a cool enough guy and someone that does in fact have more than a little security knowledge so I am not trying to pick on him but in this case he is blatantly wrong.

Ok lets be fair, he is not blatantly wrong.  He is three-quarters wrong.  Let’s look at his statement again:

Microsoft has systematically hired and/or contracted with every one of their most vocal critics (and most seasoned bugfinders) to do the work behind the scenes and they don’t count those vulns!

The first part is close to being right.  I wouldn’t say that Microsoft has hired every critic (if so where the hell is my check?) but they have hired/contracted most (perhaps only the good ones thus no check for me heh) SECURITY RESEARCHERS that have been successful in finding bugs in Microsoft software.  Note I did not say CRITICS as it is easy to be a critic and it is very hard to be a Security Researcher.

The part of the statement above that is laughable is the fact that it is an issue that Microsoft does not count any vulns found by their contract QA Engineers (hey security guys, we are really nothing more than QA).  All I can say here is no shit they don’t count them.  I know those of you who are are developers on commercial products are laughing at the thought of this right now.  But, could you imagine if every bug you found in your development lifecycle (which included the security component) had to be reported to Mitre, assigned a CVE, then counted publicly.  Not only would development time triple, but you would find yourself flooding those that count vulnerabilities with a whole lot of useless information.

(I know this post is getting long, for that I apologize, I am drinking my favorite drink (crown royal, 7-up and cranberry) while watching the UFC pay per view so I will try to end this quickly.  By the way, my money is on Serra not St. Pierre — yes I know I am going against the fellow Canadian.)

Perhaps we need to rethink our definition of a vulnerability to put this into the proper context but a bug found internally during the development process is not a vulnerability and therefore should not be counted.

The vulnerabilities that need to be counted are the ones that actually affect the end users and can compromise the security of those end users.  The “vulnerabilities” found during the SDL should not have to be public as they do not affect the public in the least.

It really is that simple folks and I am kind of surprised that after all this time we are still arguing about such a thing.  So yes, much like the tree falling in the forest… no one hears a vulnerability that is fixed during the SDL process nor should they.