WWW: World Wide Wait - Devoxx Edition

Approach

Facts & Figures

Pitfalls

Results

Great to identify and eliminate bottleneck(s)

Validation against performance model is key

Nice to be the fastest, but is your web application framework a real bottleneck?

Conclusions

Art

Raw performance

Interpreting results

but there is more

• Programming model
• Ease of development
• ...

WWW: World Wide Wait

Not present in HAR

A Performance Comparison of Java Web Frameworks

Stijn Van den Enden

@stieno

Guy Veraghtert

guy.veraghtert

bout us

Ward Vijfeijken

@wardvijf

49.932 $

59.918 $

9.986 $

6.658 $

103.193 $

www.aca-it.be

myfaces 2.1.4-SNAPSHOT

522

days ... and counting

Longest time without a government in peacetime

Criteria

Based on work from Matt Raible

http://raibledesigns.com/

Developer Productivity

Developer Perception

Learning Curve

Project Health

Developer Availability

Job Trends

Templating

Components

Ajax

Plugins or Add-Ons

Scalability

Testing

i18n and l10n

Validation

Multi-language Support (Groovy / Scala)

Quality of Documentation/Tutorials

Books Published

REST Support (client and server)

Mobile / iPhone Support

Degree of Risk

Users

10000

5

Think Time

Cost of Scale

& Performance

Matters

Estimate Infrastructure Needs and Cost

QoS that can be guaranteed

ability to handle work with the same infrastructure

Client-side RIA

Server-side RIA

VS.

Testing

Scalability Testing: performance testing that is focused on understanding how an application scales as it is deployed on larger systems and/or more systems or as more load is applied to it

Response Time or Latency: the time a client has to wait from sending the request to the moment the last byte was received

Think time: The time taken by the user to think between submitting two concurrent requests to the system.

Throughput: number of requests / seconds

The app

Implementations

Results

Overview - detail screen

Statistics

0

1

>16

300,774,676

>700

GB of data

Samples

hours of test runs

2

Navigation

Setup Environment

Collect Results in steady state

Run multiple times

and validate results

Common patterns

Overview, 40ms sleep

Autocomplete

Ajax validation

3

4

Analyze Results/Metrics

Run with different User Load

What to expect

our performance model

First order indicator of optimal load

A little Theory

avg arrival rate

Little's Law

avg wait time in system

avg # of items in system

Queueing theory

# concurrent users = throughput * (response time + thinktime)

Partial DOM updates

What about browser rendering?

JMeter does not take browser rendering into account

measure

We need a way to

this

by Jan Odvarko

http://www.frogthinker.org/projects/harlib

http://www.janodvarko.cz/har/viewer

http://code.google.com/p/harviewer

Online HAR Viewer

Analyzing

HarLib

Problems

Distribute your tests on multiple machines so that you can run your tests in parallel

with real browser generated load

Is it really necessary to use

Would you recommend using Selenium Grid for performance/Load testing?

"Selenium Grid is not designed for performance and load testing, but very efficient web acceptance/functional testing. The main reason for this is that conducting performance/load testing with real browser is a pretty bad idea as it is hard/expensive to scale the load and the actual load is very inconsistent."

HAR: Http Archive

Selenium GRID faq

High cost of resources

so many browsers?

under load

Load is inconsistent

Specification

1 browser process consumes a lot of cpu and memory

Use 1 real browser to measure the rendering time

impact of the browser on the load machine(s) has high variability

Open format that can be used by HTTP monitoring tools to export collected data

simulated

browsers

Generate load with light-weight

Format is based on JSON

Possible solution

Tools support

Online load testing / website monitoring

Infrastructure

Firebug extension (Firefox)

NetExport Firebug extension

Uses real browsers

www.browsermob.com

Getting Browser results

JavaScript WebDriver Support

Google Chrome Developer tools

HttpWatch (IE and Firefox)

DebugBar (IE)

...

?

Automation

Selenium WebDriver

Automates browsers

Can be written in Java

Start with Firebug and NetExport extensions

www.seleniumhq.org

Cross-check JMeter - HAR

HAR has some problems

Manually

Two timing fields

identify and fix performance problems

+- same result

onContentLoad

less samples

SpeedTracer

Browser specific!

Theory: before onLoad

http://code.google.com/webtoolkit/speedtracer/

onLoad

Possible GET of Resource in onLoad

Theory: fired after all of the resources for a given page have been loaded

Difference request time - end-user load time

stable difference

hope

there is

W3C Navigation Timing API

window.performance.timing

PerformanceTiming exposes a bunch of events to JavaScript including redirects, DNS lookups, load times, etc

Event 'end' attributes (fe: loadEventEnd)

IE (9+)

Chrome (6+)

Firefox (7+)

Safari

Opera

Disclaimer

DOM updates for GWT not taken into account (after onLoad event)

Not present in HAR

W3C Resource Timing API

(early design stage!)

PerformanceResourceTiming exposes timing information related to HTML elements (fe images), XMLHttpRequest, ...