Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Automated Tests and Agile Practices supporting Continuous Delivery

Automated Tests and Agile Practices supporting Continuous Delivery: TDD, unit tests, DBunit, User Interface, Integration, Acceptance tests, continuous delivery

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Automated Tests and Agile Practices supporting Continuous Delivery

BDD (Behavior)

DSL for the Business, not for the tests
- DROOLS (JBoss Rules), iLOG...
- BRMS
CI >> CD:
Extreme automation!
(Script,Script,Script...)

Let's throw your server out the window? #devops
What kinds of tests we learned?
Unit
DB
Interface
Acceptance

But what are ...?
Functional Tests
Non-functional
Integration tests
Regression Tests
ATDD (Acceptance Test Driven Development)
What we learned?
What we learned?
Professionalism

"Happiness" is the best metric

- What is the cost of "sadness"?

Quality is a company asset
Pros:
DOM (document object model) level testing
- seek by ID, Class, Xpath
Integrate interface tests on CI
Therefore, can be used to build Acceptance Tests

Cons:
Refactoring in interface => rework
Black-box test -> Where's the error source?
Setup could be painful
Strange behavior (Fail-pass-fail)
Browser oriented defects
Firefox updated / webdriver updated

Ongoing projects
2 projects (src & test)
there are tests already
Inheritance -> TestBase
2 tables
2 entities
ORM tools
changes in objects are reflected on the tables
What we learned?
What we learned?
Visual Studio or Eclipse
"New House"
->File - New - New project
The Law of Action-Reaction
AAA - Arrange, Act , Assert
MSTest ou JUnit --> Assertions
What we learned?
What are Dependency Injections?
- Dependency Injection (DI)
- Inversion Of Control (IOC)

Do you remember Unit Tests?

When there are DI's, how to unit test?

Mocks!
Test >> Red >> Green >> Refactor >> Green
Pairs are changed every 4 minutes
Action / Pause
Computer + Projector
Audience can't interfere
Everyone must follow
When to ask:
- Pairs: whenever needed
- Audience: only on pauses or green
Pair needs to interact and agree!
Continuous Integration Script

1) Download the project
Compilation and Tests are OK locally

2) Code
Edit code + tests
Compilation and Tests are OKlocally

3) Is it your turn to integrate?

4) Update project

5) Compilation and Tests are OK locally

6) Commit/Pushproject

7) Compilation and Tests are OK on the CI robot
Automated Tests and Continuous Integration based on DOJOs
Unit Tests and TDD
Continuous Integration
Dependency Injection (Mock Objects)
Using Databases
User Interface
Acceptance Tests
Continuous Delivery
Tests using DB
One DB instance for each developer
Independent Tests
- Well-defined scenarios
- DB Setup - > Fixtures

The term "fixtures" is used to database versioning too
Problem:
As a Manager, I want to retrieve the total amount of sales for a given salesman and year
Unit
Tests

CI
Venda
ID_venda
ID_vendedor
data_venda
valor_total
ID_vendedor
nome
Vendedor
Principles:
Trust in your APIs (or tools) that have already been tested
Tests must be independent (like unit tests are)
CRUD:
When to use tests in DB
ORM tools
DBunit / NDBunit
SQL Queries and tests
Scenario setup (XML)
How it works?
Before Tests -> Load the XML in the DB
Execution
After Test -> Check values or restore backups
Scenarios:
XML
Objects
SQLs
Other tools
Training Area
As a Manager, I want to retrieve the total amount of sales given a salesmen and a year, but the sales can be shared between two or more salesmen
It's not in our objectives:
to make you an expert in any specific tools

Tools and frameworks chosen for practical matters
There's a lot of other options
Warm up
Don't be shy!
Volunteer to lead!
Ask anything you want!
PRACTICE
THEORY
Unit
Tests
CI

Database refactoring
Agile data modeling (without BDUF)
Database encapsulation
Developer sandboxes

http://www.agiledata.org/essays/bestPractices.html
DB good practices for agile devs
Mock
3.2.
3.1.
Mock
DB
Interface
Acceptance
DB
Interface
Acceptance
The 12 Commandments of Continuous Integration

1) Maintain a code repository
2) Automate the build
3) Make your build self-testing
4) Everyone commits early & often
5) Everyone expects their commits to succeed
6) Every commit (to mainline) should be built
7) Keep the build fast
8) Test in a clone of the production environment
9) Make it easy to get the latest deliverables
10) Everyone can see the results of the latest build
11) Keep the build “successful” – if you break the build, your first priority is fixing it
12) Automate Deployment

Fonte: http://martinfowler.com/articles/continuousIntegration.html
CI (Continuous Integration) Pros

Codebase kept in bug-free state (debug less)
Problems on codebase are monitored
Metrics and Coverage can be a good instrument!
cyclomatic complexity
Classes
Methods
depth etc....

Metrics need to be visible!
Bad Smells


Cons

Setup scripts
Tests can be slow, depending on your product type
10-minute build (Kent Beck)
Jenkins
Other CI Robots:
Hudson
TravisCI
Bamboo
CruiseControl
Team Foundation
TeamCity
Rational Team Concert
...
Obs.: Adapted f/ Vinícius Teles
Concepts of Continuous Integration
Automated Quality Control
Integration Robot - > Jenkins
End of "short blanket" effect
"Works on my machine"

Scripts for build automation
MSBuild
Ant
Maven
Gradle
Make, ...
Jenkins
Download
Run
Configure some plugins
Configure Project (Job)
Profit!
A lot of bad jokes :)
Dojo (in practice)
Unit Tests
Baby Steps
Test-Driven Development (TDD)
Fail Principle
Redundancy Principle
Pair Programming
Concepts of Dependecy Injection
Why not test what has DI too?
Mock Objects
Moq or Mockito
Refactoring on testing
[testInitialize] or @Before
Test-driven Development - TDD

Baby steps

TDD helps to focus on code-what-you-need
Kill the BDUF!

Test-Driven Design
Emerging design

As tests coverage increases
Devs become confident about their work
Safe Refactoring

Tests work as documentation
Unit Tests
Unit -> Pascal/Delphi -> same as Class in C#/Java
Unit Tests surrounding your code
But do you test the business rule or the code?
Frameworks:
JUnit, MBUnit, NUnit ... --> *Unit
MSTest, GoogleTest, TestNG
Bottom-Up x Top-down testing style
Keep It Simple (KISS principle)
Refactoring can be safe!
Fail Fast
But ...
They'll not catch all of your app errors!
The error can come from out of the unit you are testing
TDD
TDD Script
Fail-Safe principle of redundancy

Write a test
Run your tests, see the new one fail
Code the solution so your new test passes
Run your tests and see them pass
Refactor your code (Clean Code)
Run your tests and see them pass

Time for Ping-Pong Programming!
- Example: Even/Odd game.
R
C
D
U
Why do you need to test CRUD?

Sales System

Dojo
Rules
Environment
What is our problem
Course format
DOJO:
.NET
Visual Studio
MSTest
MBUnit & Gallio
ClientDetailsValidator needs an AgeCalculator instance!
Coding DOJO
When to use?
- Simulate a webservice
- Simulate a DB access
- Simulate system time
- Simulate sending emails
- Bypass captchas
- Simulate ...
When not to use?
- injected object that is important and there are no tests for it
- Integration Tests are needed!

Cons
- Mock can hide a defect
- Test code become coupled to business code
- Refactoring in the business rules can generate a lot of work! ( reworking/redoing on tests)
"A Coding Dojo is a meeting where a bunch of coders get together to work on a programming challenge. They are there to have fun and to improve their skills."
(http://codingdojo.org/)
Four questions:
Who depends on who?
Where are the mocks?
Syntax differences between the tools?
Why this example uses Mock?
Objectives
Practical Learning
Pulled Learning System
Everyone helps teach
Discussions are code-based
Continuous Learning
Safe environment
- Non-competitive
- Collaborative
- Inclusive
Baby Steps
Principles
Mock Objects
DI, IoC
Mock x Stubs
Class x Method mocks
Mocking Tools
- MS Fakes (.NET)
- EasyMock (.NET / Java)
- Rhino Mocks (.NET)
- Mockito (Java)
- GoogleMock (C++)
- ...
There're mocks for almost
every commercial language
Rules
010011100101011000000100110011011010011001010110
Interface Tests -> Everyone uses
Automate the dumb work!
Selenium 2
In Java, you can use FluentLenium extension


Be careful! There's an old version of Selenium
Not WebDriver-oriented (Selenium Core + RC)
There are tons of sites/tutorials that teach the old version
"Selenium automates browsers."
Supports top browsers (IE, Chrome, Firefox, Opera ...)
Can help QA's build scripts (Selenium IDE)
API Selenium WebDriver works in many languages (Java, C#, Ruby, Python, php, Perl)
CARLOS FELIPPE CARDOSO, CSM, CSPO, REP has a highly technical background with focus on Agile practices and automated testing, and he has been working with that since 2004. He plays the role of team leader and he is also the technical leader of a software team, which has over 500,000 users. Felippe speaks at the most important Agile events of Brazil, such as Agile Brazil and Agile Trends. His experience includes training in software development languages such Java, C#, Python etc., Agile development and XP practices, Facilitation Techniques for agile teams, mentoring, coaching, and in recent years he has helped many teams in Brazil to deliver software with quality and high performance.


Player A
Rush to have the problem solved
Promote "flamewar" discussions
Compete with others
Let people not understanding
We should not:
...
"It's better to be red in the face once than remain in doubt forever"
Luis Paulo de Toledo
(Scouting our training area)
Lessons learned
Carlos Felippe Cardoso
Training Area:
1.
As a Salesman/Director, I want to know my sales commission along the year
Problem:
1.1.
As a Salesman, I want to know my sales commission of a month
1.1.1.
As a Salesman, I want to know my sale commission for a specific sale
Acceptance Criteria
Commission values must be:
- <= US$ 10.000: 5%
- > US$ 10.000: 6%

Company loss must be zero
(Adversary)
1.
As a Salesman, I want to know my sales commission along the year
Problem:
2.
As a Director, I want to know the trademark royalties I'll have to pay 
3.
As a Manager, I want to monitor the sales of my Salesmen
4.
As a Director, I want to control the access and access attempts to the system.
Acceptance Criteria
Royalties are 20% of sales except for the commission
Problem:
As a Finance Director, I want to know the royalties I'll have to pay this month
2.1.
Problem:
As a programmer, I want to share and integrate my code with other programmers
5.
As a programmer, I want to have tests so that I can integrate my code without damaging other programers' code
5.2.
As a programmer, I want to have AUTOMATED tests so that they guarantee my code is protected when others commit
5.2.1.
As a programmer, I want a common repository where people can commit their code
5.1.
As a programmer, I want a repository for my team so we can commit our code before intating with other teams' code
5.1.1.
Configure Git using GitHub for Windows is easy
Steps:
- Create a remote repository on GitHub
- Initialize a local repository (git init)
- Add/Commit files to the local repository
- Push the files to the remote repository

Jenkins
- Run Jenkins with a JVM installed on your machine
- Plugins Configuration
- Environment variables and paths (.NET, JDK, etc)
- Jobs Configurations (Solutions/Projects)
Training Area:
Git + GitHub
Jenkins
Warm up:
Warm up:
Player B
Game over!
Gichin Funakoshi
Kent Beck
Kent Beck
Scott Ambler
Tools
DBUnit / NDBUnit
DBDeploy (versioning)
DBFit --> FitNesse family
Dev Magic Fake
AnyDbTest
Unitils
DataWings
TSQLMigrations (for versioning)
Liquibase
Martin Fowler

Confidence with tests
Scenario refactorings are painfull
Be careful of production data!
There are tools to manage DB evolution (DDL or DML)
pee before the Dojo, not during it!

Further Reading
There are four reasons why you should automate the regression test suite
1. It increases the amount of time testers have to test new functionality. (That’s manual exploratory testing. Which requires intelligence, creativity and adaptability. Which requires a human being.)
2. It increases the level of confidence in the regression test results (Let’s be honest, humans are poor regression testers. Checking the same thing over and over is tedious. Unlike computers, we get bored, we get distracted, we don’t notice regressions slipping past.)
3. Once automated you can run them any time you want (No more worries resourcing regression testing, just press “start”.)
4. It increases testers job satisfaction – less context switching, no repetitive testing, more time to find bugs, greater use of mental abilities.

http://www.seleniumwiki.com/testing-methodologies/agile-testing/automation-in-each-sprint-scrum/#more-696
http://www.seleniumwiki.com/category/software-testing/web-testing/
Shared salesvalues could be different for each saleman
Acceptance criteria
3.
As a Manager, I want to monitor the sales of my Salesmen
Problem:
4.
As a Manager, I want to monitor the sales of my Salesmen
Problem:
3.1
User must receive a message when his login was made with success

Later, we'll make some reports, so log the datetime for each event
Accept. criteria
3.
As a Director, I want to know what are the accesses made to the system
Cost of Test
X
Cost of error
Acceptance tests inherit:
User interface test
DB scenario

Great for:
- Legacy Code (no tests)
Training Area
Refactoring DB

Safe refactoring!
DB Versioning
Scenario copy/paste
From simple tasks:
create quick bug reproduction scripts
create scripts to automate exploratory testing

To complex ones:
create robust, browser-based regression automation
scale and distribute scripts across many environments
Training Area
- You don't need to be a dev to use Selenium IDE
- Record & play tools
- Export to many languages (IDE)
- WebDriver
Jason Huggins (selenium)
What we learned?
Acceptance Tests:
What are they?
How to do it?
Selenium WebDriver + DB
Not using DSL's
Don Wells
Simon Stewart (webdriver)
CONCLUSION
Tools
Free:
Selenium
Robotium (Android)
WatiN
Commercial:
Ranorex
Robotium Recorder (record & play Android)
TestComplete (Delphi)
FrogLogic
TOSCA Testsuite
Telerik WebUI TS Studio
Squish (C++ / Qt)
QuickTest (HP)
Legacy code without tests

Selenium2 (WebDriver)
interface
script tasks (QA?)
Run as an unit test
Black box tests
- Where there's smoke, there's fire!

But, the more layers...

BIG steps, BIG problems!
One Story could have more than one funcionality
Therefore Acceptance Tests tend to be more complex than Functional Tests
- Business alignment
- Explicit requirements
- Story is DONE!
Web-Testing types
1. Usability testing
2. Browser compatibility testing
3. Validation testing
4. Backend testing
5. Security testing
6. Functional testing
7. Volume testing
8. Load Testing
P.S.:

We can make acceptance without Interfaces !
(webservices, functional or what?)

They're great for validation with the Business Analysts
If your legacy code doesn't have tests...
Start with Acceptances Test
Cover the ugly parts of your system!
If you want to refactor: try to use some unit tests too!
Automate the build and CI
References:
http://www.infoq.com/articles/Dealing-with-legacy-code
Working Effectively with Legacy Code - Michael C. Feathers
Refactoring: Improving the Design of Existing Code - Kent Beck & Martin Fowler
But I don't know where to start!
Don't even think about unit testing your entire project
Baby Steps
Refactoring!
Write tests only for the business code you'll change
Start covering the critical parts of the system

Script
Identify the target
Analyse the dependencies
Decouple the code (Dependency Injection)
Tests
Refactor
Monitor test coverage (metrics)
(source: http://seleniumhq.org/)
Acceptance
Interface
Functional Tests with Cucumber
"Cucumber lets software development teams describe how software should behave in plain text. The text is written in a business-readable domain-specific language and serves as documentation, automated tests and development-aid - all rolled into one format."

"Cucumber works with Ruby, Java, .NET, Flex or web applications written in any language. It has been translated to over 30 spoken languages."
http://dannorth.net/introducing-bdd/
http://www.urubatan.com.br/introduzindo-bdd/ (traducao do texto original de Dan North)
http://pt.wikipedia.org/wiki/Behavior_Driven_Development
http://behaviour-driven.org/
http://cassiomarques.wordpress.com/2008/11/16/testes-funcionais-no-rails-usando-o-cucumber/
http://www.urubatan.com.br/cucumber-como-testar-aplicacoes-rails-nativo/
http://www.oreillynet.com/pub/a/ruby/2007/08/09/behavior-driven-development-using-ruby-part-1.html
http://www.oreillynet.com/pub/a/ruby/2007/08/30/behavior-driven-development-using-ruby-part-2.html
http://guides.rubyonrails.org/testing.html (TDD way)
References
As a Director, I want to know what are the accesses made to the system with sucess and with fail.
Are Automated Tests worth It?
Software Quality
Hands-on courses
There are ready-made components!

Class to access the database : VendaRepository

Careful! Do not try to solve this problem using query/stored procedure (there's a reason).

Actors involved: VendaRepository, CalculadoraComissao, CalculadoraRoyalties
What needs to be "mocked"?
Training Area:
Pros and Cons of Distributed Version Control Systems

Pros

Every copy is a clone of the entire repository (w/ historical data)
Better merges, Merge Workflow Concept (see the link below)
Offline operations (Commit, diff, history, revert etc.)
Performance
Commits can be synced between two repositories (push e pull)
Peer Commit / Group Commits
Commits Management (Code Reviewer?)
A manager to "approve" pushes

Cons

Taking care of N branches can be costful!
File locks are not an option
Change Management needs to be centralized
Huge repository size (specially with long-running projects)

Sources:
http://www.infoq.com/articles/dvcs-guide
http://en.wikipedia.org/wiki/Distributed_revision_control
http://stackoverflow.com/questions/2471606/how-and-or-why-is-merging-in-git-better-than-in-svn/2472251#2472251
"(...)Despite this, there is a historical difference that is evolving over time and that, ultimately, is becoming the focus of debate between the two sides. This difference was initially expressed by Dan North (responsible for the development of RBehave, which later became the basis of the Story Runner in RSpec) and later elaborated by David Chelimsky.

This point of view puts BDD not as a direct opposition to TDD in practical terms, but as a philosophical current which encompasses TDD and expands the speech to include elements that go beyond the methodologies and starts to fall within the procedural.

A clear example of this is the rise of frameworks oriented to stories (user acceptance testing) such as Cucumber. These efforts represent the graduation of BDD not only with a way to run tests but as a process of thinking and running tests throughout the range of needs of an application (unit and visual).

This conversation is the most important part of BDD and, in the case of Rails and Ruby, RSpec represents only one facet of what has been thought of by the community. Eventually, as Chelimsky puts, TDD will be a part of something bigger that will be called BDD.

Source: http://logbr.reflectivesurface.com/2008/10/27/algumas-consideracoes-sobre-tdd-e-bdd/
"Any fool can write code that a computer can understand. Good programmers write code that humans can understand."
Martin Fowler

"To be a good programmer you have to be a good tester."
Peter Norvig
http://martinfowler.com/bliki/TestCoverage.html
http://martinfowler.com/bliki/AssertionFreeTesting.html
(Randori Coding Dojo)
JAVA
Eclipse
JUnit
TestNG
OR
Unit
Tests
CI
Mock
DB
Interface
Acceptance
TDD
DB
CI
Mock
DB
DB
Continuous Integration
Continuous Delivery
Continuous Deployment
Duration:
40 min
Duration: 1h30
Duration:
1:30h
Duration:
1h
Visual Studio
MBUnit + Gallio
NDBUnit
Ferramenta ORM
NHibernate
Eclipse
JUnit
DBUnit
Ferramenta ORM
Hibernate
OU
Por que testar consulta a banco?

Arrange (AAA)
Popular DB (XML, Objetos, etc)

Ferramenta ORM

Independência de testes
Warmup:
ValorParticipacao
$55,59!
Result: $2,7795
Mock: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.
Stubs: provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it 'sent', or maybe only how many messages it 'sent'.
Martin Fowler
The Way of Testivus

If you write code, write tests.
Don’t get stuck on unit testing dogma.
Embrace unit testing karma.
Think of code and test as one.
The test is more important than the unit.
The best time to test is when the code is fresh.
Tests not run waste away.
An imperfect test today is better than a perfect test someday.
An ugly test is better than no test.
Sometimes, the test justifies the means.
Only fools use no tools.
Good tests fail.

http://www.artima.com/weblogs/viewpost.jsp?thread=203994
Dan North:
"BDD is TDD done right."
Minimize the different points of view between devs and business analysts
select sum(vv.valor) from venda v inner join vendavendedor vv on vv.IdVenda = v.Idvenda where vv.IdVendedor = :idVendedor and year(v.dataVenda)= :ano
Acceptance Testing
System Testing
Integration Testing
Unit Testing
Functional Testing
What needs to be "mocked"?
Jez Humble
http://en.wikipedia.org/wiki/Continuous_delivery
Server
1
2
3
4
Build
Test
Feedback
4.1
4.2
4.3
Developers
push
5
Manual Testing
k21.com.br
fb.com/k21Brasil
contato@k21.com.br
What about Mobile?
Compilation/Code Signing
Emulators
Tools (e.g.: TestFlightApp, vessel.io, Clutch.io, Testfairy)
Do-it-yourself
WebViews
Canary Releasing
Feature Flags
Do you need to access the prod WebService?

Why don't you use a mock?
- "Oh, but I don't trust in my WebService supplier! "

SoapUI
http://www.soapui.org

Apache JMeter
Popular statement:
"Clean it before using it!"
http://martinfowler.com/ieeeSoftware/failFast.pdf
Unit
Tests
TDD

http://martinfowler.com/articles/mocksArentStubs.html
(mat)
happens at
any moment

Full transcript