Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

DevOps 4 Devlopers - TCE Q3 2012

From Ops 2 Dev & Ops Again ...
by

Tikal Knowledge

on 9 August 2016

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of DevOps 4 Devlopers - TCE Q3 2012

I like to consider myself as a tidy kinda guy ...
My name is
Haggai
I started my journey
as a sysadmin
myself out of everything
Or the OPS side of DevOps ...
I was the guy you'd call
@ 10PM / 2AM
During normal hour everything works just fine :)
And scripts became unmanageable
Intro
Solution
#2 :: Product
I needed Configuration Management ...
Our infrastructure had evolved
Configuration, Build & Release Management
I was getting "System As A Service"
well that is what I
thought at the time ...
I started practicing
CI/D
SAAS project
As customer base grew so did our environment ...
We have two sites now ...
well not exactly ...
Only OPS have a clear view of whats going on in each environment
QA have a hard time to reproduce
R&D have a hard time supporting
WHAT DID WE DO ?
Q: At some point we wanted to add more servers to the business layer
A: It took us
two weeks
to wire the entire process and learned, the limits were some where else
Operations code was on a different SCM [or non at all]
R&D had no access or understanding of the environments
Still doing
Merge R&D's and OPS

SCM
Virtualize PROD-like
environments
Start treating our environment as
infrastructure
OPS, QA, PSR R&D are happy Product is thrilled !
Move everything into SCM [work in progress]
Environment Similarity
No more in-PROD hacking
*
Automate deployment process
* any ad-hock hacking is being automated / translated into configuration change, and included in the following release
#1 ::
SAAS
example

Setup a job for each group

R&D, QA, PSR and had Jenkins tie the knots ...
9.0: compile & unit testing - R&D feedback
Clover: code coveredge
Confgis: generate the entire configuration matrix
Jira / Sonar: code quality dashboard
test/test2 deploy: take latest configs artifact and deploy
CheckPoint !
In terms of CI we have immediate on submit feedback
Code quality metric is checked nightly
We still have a bunch of manual QA - so can't automate the entire process
In terms of CD - we deploy to Alpha and Test but cannot go "all the way"
& the Journey continues ...
In another project
Time
of build & test ...
Frequently
used platforms
& browsers
WHAT DID WE DO ?
Jenkins
multi-configuration
project
Maven "grabs" latest artifact from repositoy
Groovy expression
as the matrix filter
Apply !
It
doesn't matter if your dealing with *AAS / Product we need:
One Source tree [
SCM
]
Environment
descriptors & BoostStrapers
Automated deployment
A release mechanism [QA/GA]
I've been lucky !
I have seen quite a few projects
Which got me thinking ...
S
AAS Example
HOW
P
rod
uct Ex
ample
HOW
Apply
Maven compiles & unit-tests [ "Upstream" ]
Publish application & test-jars to artifact repository
DevOps for Developers
Manual trigger
The nightly will trigger
last successful build
*A Release build will require the
entire matrix
execution
aka hagzag:
http://www.tikalk.com/blogs/hagzag
I Set a of defining
a "One Size Fit All"
And many others ...
&
lets see some examples -
project names omitted upon client request
`
DIY & references
OpenStack
on
VirtualBox:
http://www.tikalk.com/alm/blog/expreimenting-openstack-essex-ubuntu-1204-lts-under-virtualbox
Jenkins
MultiJob
plugin
https://wiki.jenkins-ci.org/display/JENKINS/Multijob+Plugin
Chef
:: Getting started
http://wiki.opscode.com/display/chef/Fast+Start+Guide
One click release
Based
on both examples
In many projects I participated in
this i the full run [no ie on linux]
Commit "API"
Selected Environment
Select
version
Tests
Env
OS
App
os
Env
App
Test
Tel-Aviv, July 18
http://www.
tikalk
.com

Platinum
Sponsors
Silver
Sponsor
presentation will be avail online
Haggai Philip Zagury
Agent
Server
Solution by example
Environment
type
Server / Agent
nodes
perquisites
Configuration management
Job
/Node
relation
definition
Everything is code
1
2
3
A more structured way to look at this:
And has a version
Trigger Test phase - testing infrastructure changes !
Until we commit this functionality into the plugin
We use this
YAML
Store Artifacts
All Done :: Trigger Deploy phase
Trigger Test phase
Everything
Infrastructure
Environment &
Runtime
Everything
"regular"
deploy
[agent] build in
Multi-Job
context
"regular"
deploy
[server] build in
Multi-Job
context
Parallel
Deploy
execution ... [upon successful Build phase]
Jenkins Agent / Node
Bootstrapping
see: http://wiki.opscode.com/display/chef/Installing+Chef+Client+on+CentOS#InstallingChefClientonCentOS-Bootstrap
Back To yaml ...
Our application
depends
on server environment [
chef environment
]
Back To yaml ...
Got Context - Got Nodes :: Configure nodes
I always saw a challenge in tidying up
True story ...
I would SSH & Script
System had become someone else's problem
Our Plan
}
Ask OPS
Scaling out
[still work in progress]
A web application
build [ & deploy ]
Test all officially supported browsers
We have automatic testing [selenium]
The matrix was 24 different combinations & growing with each version
}
}
Orchestration
{
Two teams @ a Chocolate Factory
(Based on a true story)
Ops 2 Dev & Ops again
SAAS
PAAS
IAAS
MAAS
The next natural step for me was
I decided to take the
trip
from Ops to Dev
They do not scale, OS / system specific
In some cases a "fruit" of 2AM madness
Mostly in order to learn ...
Every project has a repository
R&D, QA, OPS
If something goes wrong ...
A Production Floor would be referred to as something like this:
Release / Deliver
app
OS
env
test
If all is well ...
...
Build
Deploy
Test
Polling
Store Artifacts

Get nodes
Configure
Bootstrap
Node(s)
Tie Nodes to Jenkins
Deploy latest artifacts
Trigger test phase
Test
Environment
Report
Deploy to PROD
1 Click - Release Cycle
Option:
trigger based on url change
vs.
Context
The
"conversation"
/
breakdown
of the
wall
between
Dev,Ops
&
QA
Development
Quality
Engineering
IT Operations
DevOps
app
OS
env
test
We need a Start / Stop switch to get this
conveyor
running
&
opencm@MW-Build:~$ knife bootstrap node1.milkeyway.com -r 'role[server]','role[agent]' -d centos5-gems
If we have no Chef server we can:
CookBooks have a recipe (1 .. n) - basic usage:
Add group / user
Install apache2, run as daemon
Multi-dristo providers built-in
Reuse nodes ? - Chef client will not run if already applied
Apply sudo configuration
Commit "API"
Trigger Deploy phase
Commit "API"
Hosted environment with ~16 servers [physical]
based build scripts generated environment description and domains [using weblogic]
RelEng deliver to OPS [SAAS-OPS]
OPS deploy => R&D support
The relay race ...
Whereami ?
Somewhere between Dev & Ops
Setting up dev/QA / Prod-like environments
Writing installation manuals for Ops
Development
Quality
Engineering
IT Operations
DevOps
Development
Quality
Engineering
IT Operations
DevOps
Any
team member can grab the latest and
install
on his machine
wget http://mwbuild.../artifactory/runtime-package.zip
runtime-package
Tests
Env
OS
App
os
Env
OS
env
Build + Deploy + Test
The cloud enables our conveyer to run 24/7
Release
app
OS
env
test
...
app
OS
env
test
Get
source
code
Apply
context
B
uild
D
eploy
T
est
R
eport
Polling ?
report
unit test
compile
prepare
runtime
=>
Quality Dashboard
% whoami
`
We have our Beans !
Let's Start Cooking ...
Chef server
Chef client
Chef client
A configuration management framework
Keeps
versions
of CookBooks (A repository for CookBooks and more)
Searchable
index - provide state of managed nodes
Attributes - variables you can utilize in your recipe
Recipes - Action Items to execute
Templates - template file(s)
and more
Taken from Tomcat recipe:
Commit "API"
Controlling the entire process from SCM commit message
JEOS
Sys Req
Application
Back To yaml ...
We need nodes / agents
Got Context - initialize instance(s)
Just Enough Operating system
How frequent does your OS change ?
Kernel / Driver developer ?
Application [OS-agnostic ...]
What's in your
infrastructure
?
Whats on our images ?
JEOS
Chef-Client
What is your infrastructure ? - Define it !
Remember This ?
Its still evolving ... and
Challenging
our
Deploy
ment process
methodologies
Manual
Trigger Multi-Job
Optional: trigger via
script / commit hook
will trigger HEAD /
${BRANCH}
variable
Parallel

build execution at each phase
"regular"
continuous
[server] build in
Multi-Job
context
"regular"
continuous
[agent] build in
Multi-Job
context
Deliverable:
Assemble Runtime Directory
Prepare runtime for developer
mvn clean install = runtime-package.zip
Prepare runtime for CI/CD
Profile
activation
Maven
Assmbly
- prepare-package
Utilize Parameters
Yaml
- Define where ... or
Let

define
Our application
depends
on environment
Get
Version
JEOS
Sys Req
Application
Build
phase "View"
`


Steps to success
uild
eploy
est
B
D
T
Inheritance [Unique Tag]
Little bit about Chef
Attributes
Tem
plates
: server.xml
crontab based polling
cookbook(s)
I
AAS
P
AAS
M
AAS
S
AAS
knife
search
node "chef_environment:
TEST
AND platform:ubuntu"
{
"name": "TEST",
"default_attributes": {
"tomcat": {
"listen_ports": [
"6080",
"6443"
]
}
},
"json_class": "Chef::Environment",
"description": "The test environment",
"cookbook_versions": {
"server": "1.0",
"agent": "1.2"
},
"chef_type": "environment"
}
Chef environment
description
Json / DSL
Switch
OS
no problem :)
maybe templates change ...
"Built-in"
providers
http://wiki.opscode.com/display/chef/Spiceweasel
yaml approach - which does much more ...
http://prezi.com/u4zb3h6s5vql/
devops-4-devlopers/
To be continued ...
Thank You To Our Sponsors
Junit & Selenium
And "knows" on which node to run
Frameworks ...
Getting
functional test
to
run automatically
Transform
functional QA

to
Automation
Challenge
DoD
Mavenize ...
haggai@tikalk.com
B
D
T
Dev
Ops
Qa
Team
Flow
Commitment
1
&
Declared
in
code
Declerative
mvn clean install [deploy]
knife cookbook upload -o /path/to/cookbook mycookbook
Tikal is a leading provider of open source solutions :
experts, software stacks, workshops and subscription for software development.

Founded in1999, based in Tel-Aviv Israel,
employing over 80 software experts.
A few words about where I come from ...
Build
Deploy
Test
Pull Code
Phase
Phase
Phase
Store
Store Cookbooks
Get Artifacts
Trigger
Configure
Provide
nodes
"One Size Fit All" Workflow
CI
QA have the same set ...
Leverage
Leverage Filtering
Now What ?
looking for commonalities / patterns !
Can I copy 'n' paste ?
Is my process well defined ?
checkpoint #n
Retroactive flow
"discovery"
{
https://wiki.jenkins-ci.org/display/JENKINS/Multijob+Plugin
Apply Context
Divide your flow to components / modules
Define paralleled phases for multi-module projects
Improved release pipeline
The began
Jenkins + ?
Do you stuff
{
Took it to the team
A Transparent & deterministic process
A Journey from
Tikal Community Event [TCE]
Apply
context
What's in
Phase X
"regular CI job"
"regular CI job"
parallel
Phase Y
"regular CD job"
"regular CD job"
parallel
Sequential
(put job in context)

Select
environment
+
If I have a "Common" => add another phase
Full transcript