Loading presentation...

Present Remotely

Send the link below via email or IM


Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.


Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

PhD Presentation - Mandinga - v2

No description

Federico Toledo

on 3 March 2014

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of PhD Presentation - Mandinga - v2

Test Execution
Models and Metamodels
MANDINGA: Methodology for Automation Testing Integrating Functional and Non-Functional Aspects

Conclusion and Future Work
Systematic Literature Review for performance testing of Information Systems (with model driven approaches)
Goals achieved
Framework to test Information Systems
Functional and performance testing
Model driven approach
Goal 1

Automate the test case design of Information Systems
Future Work
Continue the development and experimentation with the Mandinga methodology
Goal 1

Automate the design of
functional test cases
for Information Systems
Goal 2

Reduce cost and increase flexibility of
performance testing

Goal 3

Integrated view: functional and non-functional aspects
PhD Student:
Federico Toledo Rodríguez

Macario Polo Usaola
Beatriz Pérez Lamancha

Thank you!!
Knowledge transferred to the industry
Implementation on a commercial tool
Used in real projects
Standard extension mechanism: UML Profiles
Tagged values
Extension of metamodels
Only UML-TP considers non-functional aspects

Current support of UML-TP for performance testing is not enough
Information System Models
UML model of the Information System Under Test
Data Model
Graphic User Interface Model
Model Driven Approaches
Modeling and Analysis of Real-Time and Embedded Systems
UML Profile
Standard supported by OMG

Properties metamodel
Proposed in CNR
Functional Testing
Specific for (it has its own metamodel)
Performance Testing
Non-Functional Properties to Performance Test Model
Non-functional requirements (PMM)
considered into the test model (UML-TP)
Generation of Executable
Performance Test Cases
From Selenium to OpenSTA
Integrated approach
One unified vision (model) of the system requirements and its verification, considering functional and non-functional (performance and availability) aspects.
Research methodology
List of publications
Publications per topic
PhD Thesis
March 7th, 2014
Ciudad Real, Spain

Industrially applicable results
Data Model
There is no standard

UDMP: UML Data Modeling Profile
Proposed by IBM
Entity-relationship in UML
Graphic User Interface Model
The standard is more complex than what we needed
We designed a simpler version named "GUIMP: Graphic User Interface Model Profile"
Business Rules
Integrated to UML
Standard by OMG
Functional specification of the requirements of the system under test
State of Art and Practice
Systematic Literature Review for information systems testing
To manage the models:
UML SDK in Eclipse
Information System Model construction
Test Model Derivation
Test Code Generation
Eclipse Plug-in
Final degree project (2012-2014)
Jesús Núñez
Automated testing tools
Functional testing services
Performance testing services
Information Systems with databases
Mobile and web
Standard supported by
Non-functional aspects
Focus on performance and availability
Standard by OMG

Defines the main concepts related with testing
Model driven approach?
Test Architecture
Test Behavior
Test Data
UML Testing Profile
Criteria to cover system code considering SQL sentences (white box approach)

Generation of
input test data
database states
from the definition of restrictions on test data
Taken from the database
Provided by the user
None of them
Verifies whether the system under test correctly deals with the database structure
"Negative" testing
Only a few provide a model-driven approach
Benchmark generation: comparing web servers, hardware, etc.

Performance tests generation from requirements

Models to predict performance and Software Performance Engineering (SPE)
Models to represent performance tests?
Functional Testing
Performance Testing
Relational Web
Problem to Solve
Case Study
Bancard (bancard.com.py)
Paraguayan financial company
Platform migration
High risk in the release of the new version
4 modules, more than 1500 entities
Eclipse Plug-in integrating M2M and M2T
Chapter 8

Under the expected
Average response time of "Add product" should be less 10 seconds
95% of the executions of "create invoice" should pass
State of the Art.
Information Systems Model
State of the Art.
Information Systems Model
State of the Art.
Functional Testing Model
State of the Art.
Functional Testing
State of the Art.
Performance Testing
State of the Art.
Performance Testing
State of the Art.
Performance Testing
What is interesting to test?
Some examples
Enter data through the user interface not respecting database restrictions
duplicated keys
null when not allowed
invalid data types
invalid foreign keys
Relationship boundaries
an invoice without products
an invoice with lots of products
Business rules
trying to enter a negative price
Test case design based on the database structure
Test case execution through the user interface

We are not testing the database!
CRUD: create, read, update, delete

Correspondence between entities and pages for their CRUD
Functional testing automation
Performance Testing
GeneXus team tests GeneXus with GXtest Generator
Analysis of Errors
Inconsistencies between database and logic layer
Non-existent resources
Non-editable fields in the update page
Derived attributes could be edited
Entities invoked in an unexpected way
Some test situations also helped to detect errors in the data model

ATL to generate GUI Model

Correspondence between entities and the pages offering CRUD operations
Reverse engineering with Relational Web
Export to RSA format
UML is not the same standard for everyone
It was necessary to adapt the XMI
Import from RSA
Test Patterns
ATL transformations
Looking for patterns to test
Export to RSA
Import from RSA
Acceleo scripts
Model to text
Test case behavior
Test data
Adaptation layer (test components)
MIM: Model Implementation Matching
GUI structure
GUI Navigation
UML-TP Extension
Test model considering workload and non-functional validations
Case Studies
New: extended for JMeter and for Mobile apps
One line of Selenium equivalent to 200 lines in OpenSTA
Test Model
Coverage criteria proposed by Andrews et al.
CRUD (create, read, update, delete) criterion
Testing the life cycle of each entity
C · R · [U · R]* · D · R
Valid and invalid test data (equivalence class partitioning criterion)
Data Types
Database restrictions
Business rules restrictions**
ATL transformations looking for
substructures in the data model
considering GUI elements
business rules
Interesting test situations for CRUD operations
Test criteria for class diagrams
AEM Criterion: Association-end multiplicity.
CA Criterion: Class attribute.
Useful to test relationships with foreign keys
Considering the columns of each table
Test Components
Data-driven testing
Same behavior for different platforms
Creation with invalid data
The test should verify that the instance was not created
The user should be notified
Creation with valid data
The test should verify that the instance was created with appropriated data
Test Data
Test Behavior
Even though it was designed for a specific purpose within this thesis, it can be used for any combination of Model-to-model and Model-to-Text transformation
Coverage criteria for data generation based on database structures
Coverage criteria for PMM models for performance testing
Goal 2

Reduce cost and increase flexibility of performance testing
Goal 3

Integrated view: functional and non-functional aspects
Applied in the industry with the implementation on GXtest
Smart Monkey Testing: Monkop http://monkop.com/
Integrating Mandinga approach
Model-driven approaches
The model is executable
Initial set of regression test cases at zero cost
UML and extensions
Standard transformation languages
Eclipse platform

Generated Functional Tests
Entities life cycle (CRUD) based on data model
System test, through the user interface
Well-known execution environment
Parametrized test cases (data-driven testing)
Extensible to different platforms

Generated Performance Tests
Easier to understand and maintain
Faster and more flexible methodology
Lessons learned
Annex I
Chapter 3, 4, 5, 7
Chapters 3, 6, 7
Chapters 3, 5, 6, 7
Coverage Criteria
Proof of concepts
Use in real projects
Analysis and refinement
Reduce testing costs
Improve time to market
If you develop with models you have to test with models
Model-to-model transformation languages
Model-to-text transformation languages
Standard: QVT (Query View Transformation)
De facto
MOFM2T standard
Pragmatic implementation:
We need a language to specify tests
Events and Operators
It is possible to relate events with operators
After, before, sequence, between, etc.

Create Invoice =
Payment AFTER
Register Client BEFORE
(SEQ (Add Product, 10))
State of the Art.
Information Systems Model
Chapter 4

Chapter 5

Chapter 5

Chapter 5

"Test adequacy criteria for UML design models."
Software Testing, Verification and Reliability 13.2 (2003): 95-127.
Chapter 5

Chapter 5

Chapter 7

Chapter 7

Chapter 6

Chapter 6

Chapter 6

The effort required with our framework was reduced from 2 to 5 times
Traditional approach: 6 to 10 hours per script
Our approach: 1 to 5 hours per script
Maintenance in traditional approach: rebuild the script from scratch
Our approach: adjust Selenium script, regenerate

Chapter 7

Chapter 8

Chapter 8

Information to derive test cases
Test Data
Automation of test execution
Improves test case execution time
It is easy to repeat it:
in different platforms
with different data
in different moments
Techniques to find errors under reduced cost
Typically associated to a Coverage Criteria

Looking for the inputs and situations that have more probability to find bugs
test cases
test data
Easy to repeat
Easy to control
More measurable
equivalent workload
PKV: Primary Key Violation
FKV: Foreign Key Violation
URV: Unique Restriction Violation
NNV: Not-Null Violation
DTV: Data Types Violation
Coverage criteria defined on the PMM operators
Full transcript