Loading presentation...

Present Remotely

Send the link below via email or IM


Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.


OWASP - Cost-benefit for application security testing (2012)

Research on cost-benefit model for large-scale application security testing based on value of information concepts.

Paweł Krawczyk

on 23 October 2012

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of OWASP - Cost-benefit for application security testing (2012)

(cc) photo by Metro Centric on Flickr (cc) photo by Franco Folini on Flickr (cc) photo by jimmyharris on Flickr (cc) photo by Metro Centric on Flickr Building a Cost-benefit
Model for
Application Security Testing pawel.krawczyk@hush.com Can't we just do the pentesting? Plenty of applications
one, two ... five hundreds... thousands....
Tons of legacy applications
developed 10-20 years ago
Very different value
trivial desk booking for 100 users
huge financial/HR for 5m+ users What are our assets? In real life... How scientific is security? Our security industry is still immature
Attempts to rationalize and standardize:
Common Vulnerability Enumeration (CVE)
Common Weakness Enumeration (CWE)
Common Vulnerability Scoring System (CVSS)
OWASP Application Security Verification Standard (ASVS)
ISO 27034:2012
Targeted Level of Trust

These are all desperate attempts to manage
overwhelming amount of assets at risk Larry Suto, "Analyzing the Accuracy and Time Costs of Web Application Security Scanners", 2010 "Being Explicit About Weaknesses", R.A. Martin, S. Barnum, S. Christey, 2007 Why not "just do pentesting"? Just try
to apply
"pentest everything deeply" paradigm

to all these applications

Physically impossible
in large scale organisations
with live, changing applications
What happens if you try? Congestion "Due to our current
load you need to
wait 6 months to test
your application" Infinite funding "To test all our apps
we need zillion $$$
more" Infinite time "At current speed
we will cover all
our portfolio
in 8 years" Unloading the congestion Are some applications more important than the others? Lose $1-$140 per record
Potential loss $1m-140m Lose $1k-100k on investigation, repair What to take
into account? Classification
adequate level of assurance Number of sensitive records
Daily generated revenue
Remediation & investigation cost
Network exposure
Number of users
Access controls Risk criteria Public internet
~2 billion users can see the site
Limited number of users can see the site
Number of employees can estimate this Exposure None (public access)
Limited with self-registration (anyone can register)
Limited with managed access Access controls Economic view Based on
Decision tree analysis
Expected value of perfect information (EVPI)

What is likelihood that my application is vulnerable?
Calculate expected utility (EU) of various testing methods
Calculate EVPI
Compare EU to EVPI to see which testing method makes most sense Outline Expected Utility needs well defined probabilities
What is the probability of my app begin vulnerable?
Initially we have no idea
We can express that by likelihood of 50%

What if we could purchase that information?
With 100% certainty - "perfect information" Dealing with uncertainty Purchase really means
"do some kind of measurement" EVPI Expected Value of Perfect Information No prior information Perfect information $20k Upper limit of pentest cost Attack* scenarios Data breach
Theft of sensitive information
Impact proportional to number of records Automated, large scale malware planting and defacements
Impact - investigation, containment, remediation
Smaller - no revenue generating sites
Bigger - revenue generating sites (can be calculated) Opportunistic hacking Client facing attacks Examples: CSRF, XSS
incident response
service restoration
reputation Penetration test pricing Non-linear
Pentest scoping
Number of dynamic, static pages; forms; input fields per form
Number of man-days
Rules of thumb
Sometimes gets tricky
Man-days correlated with testing precision

In result
Pentester can value a trivial application at $20k
Pentester can value a complex application at $5k
...with obvious consequences for testing precision pawel.krawczyk@hush.com


http://ipsec.pl (slides etc) Questions & comments Desk booking HR App In application security since 1996
Making (embedded devices, including ITSEC, Common Criteria)
Breaking (penetration testing)
Since 2009 managing security of large business-critical application portfolio
at Aon - large insurance company App with hack impact $100k EVPI = PI ($0) - best ($-20) * "missing httpOnly" is not an attack

So what we treat as an attack? This is where EVPI models can really help Summary You can't manage what you can't measure Reality of security testing market Accuracy of security scanners What we do not want? This is what we do not want Hall of fame at datalossdb.org Loss if hacked Some factor "Economics is the social science that examines how people choose to use limited or scarce resources in attempting to satisfy their unlimited wants." Why economics at all? Does it make sense to... Spend less testing resources on Desk Booking app?
get less reliable results
but accept the risk
because the impact is much smaller

Spend more testing resources on HR app?
use resources saved on Desk Booking apps
get more reliable results
where the risk is larger

ISO 27034:2012 "Targeted Level of Trust"
Common Criteria - Evaluation Assurance Level
OWASP ASVS - Verification Levels Similar concepts Give budget planners rational arguments
Gain more knowledge for less money
Focus on high-risk applications
Reduce risk earliest where it's biggest
Full transcript