Send the link below via email or IMCopy
Present to your audienceStart remote presentation
- Invited audience members will follow you as you navigate and present
- People invited to a presentation do not need a Prezi account
- This link expires 10 minutes after you close the presentation
- A maximum of 30 users can follow your presentation
- Learn more about this feature in our knowledge base article
Do you really want to delete this prezi?
Neither you, nor the coeditors you shared it with will be able to recover it again.
Make your likes visible on Facebook?
Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.
OWASP - Cost-benefit for application security testing (2012)
Transcript of OWASP - Cost-benefit for application security testing (2012)
Application Security Testing firstname.lastname@example.org Can't we just do the pentesting? Plenty of applications
one, two ... five hundreds... thousands....
Tons of legacy applications
developed 10-20 years ago
Very different value
trivial desk booking for 100 users
huge financial/HR for 5m+ users What are our assets? In real life... How scientific is security? Our security industry is still immature
Attempts to rationalize and standardize:
Common Vulnerability Enumeration (CVE)
Common Weakness Enumeration (CWE)
Common Vulnerability Scoring System (CVSS)
OWASP Application Security Verification Standard (ASVS)
Targeted Level of Trust
These are all desperate attempts to manage
overwhelming amount of assets at risk Larry Suto, "Analyzing the Accuracy and Time Costs of Web Application Security Scanners", 2010 "Being Explicit About Weaknesses", R.A. Martin, S. Barnum, S. Christey, 2007 Why not "just do pentesting"? Just try
"pentest everything deeply" paradigm
(like WAHH, OSSTMM)
to all these applications
in large scale organisations
with live, changing applications
What happens if you try? Congestion "Due to our current
load you need to
wait 6 months to test
your application" Infinite funding "To test all our apps
we need zillion $$$
more" Infinite time "At current speed
we will cover all
in 8 years" Unloading the congestion Are some applications more important than the others? Lose $1-$140 per record
Potential loss $1m-140m Lose $1k-100k on investigation, repair What to take
into account? Classification
adequate level of assurance Number of sensitive records
Daily generated revenue
Remediation & investigation cost
Number of users
Access controls Risk criteria Public internet
~2 billion users can see the site
Limited number of users can see the site
Number of employees can estimate this Exposure None (public access)
Limited with self-registration (anyone can register)
Limited with managed access Access controls Economic view Based on
Decision tree analysis
Expected value of perfect information (EVPI)
What is likelihood that my application is vulnerable?
Calculate expected utility (EU) of various testing methods
Compare EU to EVPI to see which testing method makes most sense Outline Expected Utility needs well defined probabilities
What is the probability of my app begin vulnerable?
Initially we have no idea
We can express that by likelihood of 50%
What if we could purchase that information?
With 100% certainty - "perfect information" Dealing with uncertainty Purchase really means
"do some kind of measurement" EVPI Expected Value of Perfect Information No prior information Perfect information $20k Upper limit of pentest cost Attack* scenarios Data breach
Theft of sensitive information
Impact proportional to number of records Automated, large scale malware planting and defacements
Impact - investigation, containment, remediation
Smaller - no revenue generating sites
Bigger - revenue generating sites (can be calculated) Opportunistic hacking Client facing attacks Examples: CSRF, XSS
reputation Penetration test pricing Non-linear
Number of dynamic, static pages; forms; input fields per form
Number of man-days
Rules of thumb
Sometimes gets tricky
Man-days correlated with testing precision
Pentester can value a trivial application at $20k
Pentester can value a complex application at $5k
...with obvious consequences for testing precision email@example.com
http://ipsec.pl (slides etc) Questions & comments Desk booking HR App In application security since 1996
Making (embedded devices, including ITSEC, Common Criteria)
Breaking (penetration testing)
Since 2009 managing security of large business-critical application portfolio
at Aon - large insurance company App with hack impact $100k EVPI = PI ($0) - best ($-20) * "missing httpOnly" is not an attack
So what we treat as an attack? This is where EVPI models can really help Summary You can't manage what you can't measure Reality of security testing market Accuracy of security scanners What we do not want? This is what we do not want Hall of fame at datalossdb.org Loss if hacked Some factor "Economics is the social science that examines how people choose to use limited or scarce resources in attempting to satisfy their unlimited wants." Why economics at all? Does it make sense to... Spend less testing resources on Desk Booking app?
get less reliable results
but accept the risk
because the impact is much smaller
Spend more testing resources on HR app?
use resources saved on Desk Booking apps
get more reliable results
where the risk is larger
ISO 27034:2012 "Targeted Level of Trust"
Common Criteria - Evaluation Assurance Level
OWASP ASVS - Verification Levels Similar concepts Give budget planners rational arguments
Gain more knowledge for less money
Focus on high-risk applications
Reduce risk earliest where it's biggest