Loading presentation...

Present Remotely

Send the link below via email or IM


Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.



UCP/HCP Key Performance Indicators

David Oreper

on 15 May 2013

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of KPI

UCP/HCP Key Performance Indicators Release Phases Implementation Details Endgame Post-Release High level design
Detailed design and Architecture
Break features into tasks
Estimate duration for each task KPI: Task Completion Rate WHAT: Estimate completion based on dev days
WHY: Anticipate and resolve impediments to timely completion
HOW: Track rate at which tasks are completed
IMPACT: Schedule/Scope How Use Excel to enter tasks
Track owner, dev cost, % Complete
Developers update task completion weekly
Measure rate at which tasks are completed to predict completion date Task Details Quickly filter for assigned tasks
Add notes to follow-up
Use pivot charts to easily change representation of tasks Why not Microsoft Project? Excel calculations very powerful Dependencies are worked out peer to peer, not through gantt charts Each team/individual has complete freedom as to how to arrange and complete tasks Have tried to use project in the past
A lot of bugs
Calculations unreliable
Special license required MS Project Organization changes focus to qualification
Run new tests
Run regression tests
Track bug counts
Track test pass rate
Track test execution rate Endgame Details KPI: Bug metrics WHAT: Bugs: New, Resolved, Reopened, Verified
WHY: Observe bug rate trends to anticipate instability
HOW: Create reports on bug activity
IMPACT: Quality How Use bugzilla queries to gather results on a weekly basis.
Track on wiki
Create graphs Bug Activity Active By Feature Active By Priority NEW:
As we approach release date, RESOLVED bugs should surpass NEW bugs
VERIFIED should be equal to RESOLVED
REOPEN rate should be consistently low Determine quality by feature
Use information to determine load-balancing
Determine which team process was most effective Ensure P1/P2 bug counts stay low
Watch for spikes which indicate regressions KPI: Test Rates WHAT: Test execution and Test pass rate
WHY: Ensure all tests can be run. Ensure test results indicate high quality
HOW: Create reports on test activity/success
IMPACT: Quality How Use TTS (Test Tracking System) for manual execution
Use Automation Scoreboard for automated tests Automation Scoreboard Shows granular details for each test pass
Several test passes per release
Breakdown by owner, test plan, etc
Automatically creates charts
Automatically imports and aggregates automation results Use this information to focus automation resources
Maximize HW utilization by sharing resources at night TTS Use execution rate to determine number of test passes
Use pass percent to determine release quality
Focus on features with high failure rate for subsequent passes Show results for tests executed through automation
Dashboard for investigating failures and tracking bugs Results Filter Result
Summary Not yet
executed Associated
Bugs Broken down by feature by build Hardware utilized Results Task Summary Completion date based on completion rate and remaining work
Easy to see impact of each task on completion estimate
Milestones highlight near term deliverables Total Dev Days Dev days remaining Completion rate Completion date Milestones Assess how smoothly release process was executed
Discuss pros/cons with functional teams
Identify areas to improve
Measure release deliverables Post-Release KPI: Lines Of Code WHAT: Count total lines of product and test code
WHY: Just for fun
HOW: Use LOC utilities
IMPACT: None HCP LOC Total Test cases No correlation between LOC Delta and customer escalations
LOC Per month is trending up as team gets stronger
Tracking anomaly for Crossroads release - deviated from standard escalation tracking process Escalations
Customer base Customer esclations remain constant as customer base increases 30-40% increase in test coverage per release
Heavy investment in unit, integration, automation tests UCP LOC Relatively new product - a lot of code change
No data for customer escalations KPI: Customer Escalations WHAT: Total number of customer escalations compared to install base
WHY: Validate release quality
HOW: Measure total number of customer tickets
IMPACT: Quality HCP KPI: Test Cases WHAT: Total number of test cases
WHY: Ensure test coverage in increasing with features
HOW: Count total manual, automated, unit, integration tests
IMPACT: Quality HCP UCP 2.0: 36% Manual
2.0 Patch0: 36% Manual
2.1: 34% Manual
2.2: 33% Manual Large investment in automation
Percent manual trending down
Test coverage increasing Implementation Conclusion No single KPI is used as predictor of success
Each metric is used as loose indicator that things going in right direction
People are the key
Each KPI focuses on enhancing
Morale Retrospective KPI: Retrospective Feedback WHAT: Collect feedback on process pros/cons
WHY: Use feedback to refine process - keep things that worked well, improve things that didn't
HOW: Meet with each functional team individually to get each perspective on how things went
IMPACT: Morale KPI: Retrospective Not easily quantifiable but vital as a KPI
Gives people a chance to express frustration
Good opportunity to creatively solve pain points
Owners are assigned to highest value tasks How Are Tasks Sized? Tasks created after detailed design for each feature
Max size ~5 days
Each engineer creates estimates for their tasks
Estimates and tasks are reviewed by leadership team Weekly Report High level Snapshot Breakdown by Feature Team Progress by Milestone Overview across feature teams
Inspires healthy competition between teams
Very effective for communicating release progress 4.0: 33% Manual
5.0: 35% Manual
Full transcript