Performance Testing in
Continuous Integration Environments
Solmaz Abbaspoursani
@s_abbaspour
Performance Engineer at Move Inc
Builders Digital Experience
Solmaz Abbaspoursani
@s_abbaspour
solmaz.abbaspoursani@move.com
DevOps Practices
- Waterfall / Agile methodology
- One large team of QA and Devs
- Poor communication between QA, Devs and Ops
- 1 release every 30 ( + delays)
- Partial CI - Manual builds to QA/Prod environments
- Partial automated tests
- Long and tense integrations
- Painful deployments
- ...
- Late performance tests would delay release
- Agile
- Smaller teams
- Better communication among team
- Less than 1 week releases
- Reached continuous deployability
- Visibility into build process and status
- Automated tests
- Reliable deployments
- ...
- Early performance issue discovery
- Cultural shift
- Team structural changes
- Integrated tools
- Jira
- Github
- Octupus Deploy
- Teamcity
- New Relic
- Focused on automation
- Even office rearrangements!
- Interactive layout
- Whiteboards
- Monitors and green lights!
Performance Test Process
Integrating performance tests
2.
Server Side Performance Tests
Tools:
1. Neoload for load generation
2. Dynatrace for troubleshooting
Metrics
- Server Side performance
- Response times (Average and Median)
- Throughput
- Error rate
- Number of external service calls
- Monitoring
- CPU usage
- Memory
- Disk
- Network
Dynatrace + Neoload
Reports
1.
Performance test
on feature complete build
3.
Client Side Performance Test
Push "master" branch to performance environment
Tools:
1. Cucumber and Selenium: browser side testing
2. Dynatrace : browser metric gathering
Teamcity Nightly Tasks
Run :
- Unit Tests
- Automated integration tests
@performance
Feature: Measure Front-end Performance for Not For Sale User Scenario
Scenario: Measure Performance for Not for Sale Search Page
Given I am a new user
And I am in "Not For Sale" Landing page
And I want to measure performance for "RDC_NotForSale_Search_Page"
Given I am in "Not For Sale" SRP on "Las Vegas, NV" search
Then measure performance
Start and Stop Timers
+
Metrics
- Browser Metrics
- Onload time
- Timer duration
- Number of requests
- Number of domains
- JS activities
- Render time
- Number of exceptions
- Server time
- ...
Dynatrace + Cucumber
Reports
This would be too late!
4.
Reports and emails
Browser-Side performance analysis
Examples
Server-Side performance analysis