CitconWiki - User contributions [en]

Reproducible CI

2013-08-24T21:43:44Z

Jonny:

'''Reproducible CI'''

* Can I do this with Windows servers? Yes.

* Versioning CI tools by having a job that commits the CI configuration to source control every hour.
* Using Jenkins config history plugin for Jenkins gives you history of config, but only by using the plugin.

* Has someone upgraded the compiler or patched the OS? Has that impacted the build?

* Some clients spin up a VM from an image and set the clock to a known time, leaving nothing to chance.

'''What is the end goal of reproducible CI?'''

* Being able to bring back the CI environment if it comes away.

* Some businesses need to be able to reproduce the application years in the past.
* If you're seeing weird behavior with your application, it may be caused by the CI server, and this way you can see what was happening with the build server for a specific artifact.
* Some clients will want to stay on the same version for years and years, so you may need to recreate a build that hasn't been run in years.
* With large and/or remote teams, being able to see who made what change and why is critical, especially if something goes wrong.
* With CI/versioning, you can promote changes to your pipeline.

* If you can have a reproducible build system, developers can prove to themselves that what they are going to add won't break the build. No "it works on my machine" excuses.

Tools:
* Vagrat
* Docker

* Is there a need for a "meta-pipeline" (pipeline for the pipeline) that builds and promotes the build system?
** Build systems are products, consumed by developers, and testing needs to happen before the change is pushed to prod server.

* We don't hold the pipeline to the same processes we hold for the source code we're building.

* Is the issue that the team is too big?
** Do we want our team of ten managing a build server, or do we want them delivering value to customers?
** Do I want 100 teams with their own build server, or do a want a couple guys managing a build server for 1,000 developers?

* Having the small team manage their build system empowers them about the build and deployment process.
* Having a single team manage it for a huge group of developers gets the developers out of touch with the deployment process.
* Developers care more about getting stuff done over reproducibility -- they'll install something to "get it to work" and then forget how it got there.
** Developers manage their server, but the slaves are rebuild nightly.
** All teams have their own build server, but agree on a specific deliverable type and publish it to an artifact repository. There's a master build server that watches the repo and does the integration of all the components.

* Does anyone care about traceability?
** Getting the code is easy.
** What about the tests that run?
** How was the build server set up that day?

* We can reproduce what we built, can we reproduce how we deployed it? If we change all the build scripts? If we change what tool we used?

* Don’t do * includes if you can avoid it.

* Tool chain from checkin -> deployment to production
** SCM
** git
** svn
** hg
** ClearCase
** Perforce
* Build Server
** “Runs the build”
** Transform source + dependencies into an artifact. (May also run tests.)
** Compile, link, package, test
** Static Analysis
** Jenkins
** BuildForge
** AHP
** CruiseControl
* Artifact repo
** Nexus
** Artifactory
** S3
** SCM
** Leave it in the build tool
** Push it to deployment tool
* Infrastructure
** Would follow the same chain to go from “source” to “image.”
** Long lived physical environments that get set up once
** Virtual machines that are treated like long-lived physical machines.
** Stand up virtual instances per build
** A person might build the machine, build the golden image, build a custom script.
* Deploy
** Chef/Puppet
** Capistrano
** uDeploy
** Orc Deploy
** Shell/Perl/Python/Ruby/etc scripts
** A person does everything manually following a document
* Database
** Liquibase
** Rails Migrations
** Dacpac
** Play Evolution
** A person that updates the database.
* Test
** Performance
** Integration
** Unit
** Deployment/Smoke
** Functional
** Load
** Security
** Installation/Manifest Check (make sure all expected files exist)
** A person does everything manually following a document
** Bees with Machine Guns
** HP LoadRunner/WinRunner/etc
** Cucumber
** Selenium
** Fitnesse
** Spock
* UAT / Business Signoff
** Blue / Green deployments
** Canary deployment
** Person clicks a button
** Manual signing of document
* Deploy to Production?
** Same as deploy to non-production
** A person does a partial deployment

* Should a build server deploy?
** Yes
*** Get it out there so people can use it
** No
*** Slow tests aren’t run
*** Deploy to one environment or many?

* Conclusion: script all the things. Version all the things. You can do it on Windows.
** Treat the systems the same way you are (should be?) treating the software going to the users.
** It’s worth it for any size team: one person, ten people, a thousand people.

Reproducible CI

2013-08-24T20:39:58Z

Jonny: Created page with "'''Reproducible CI''' Can I do this with Windows servers? Yes. Versioning CI tools by having a job that commits the CI configuration to source control every hour. Using conf..."

'''Reproducible CI'''

Can I do this with Windows servers? Yes.

Versioning CI tools by having a job that commits the CI configuration to source control every hour.
Using config history plugin for Jenkins gives you history of config, but only by using the plugin.

Has someone upgraded the compiler or patched the OS? Has that impacted the build?

Some clients spin up a VM from an image and set the clock to a known time, leaving nothing to chance.

What is the end goal of reproducible CI?

Being able to bring back the CI environment if it comes away.

Some businesses need to be able to reproduce the application years in the past.
If you're seeing weird behavior with your application, it may be caused by the CI server, and this way you can see what was happening with the build server for a specific artifact.
Some clients will want to stay on the same version for years and years, so you may need to recreate a build that hasn't been run in years.
With large and/or remote teams, being able to see who made what change and why is critical, especially if something goes wrong.
With CI/versioning, you can promote changes to your pipeline.

If you can have a reproducible build system, developers can prove to themselves that what they are going to add won't break the build. No "it works on my machine" excuses.

Tools:
* Vagrat
* Docker

Is there a need for a "meta-pipeline" (pipeline for the pipeline) that builds and promotes the build system? Build systems are products, consumed by developers, and testing needs to happen before the change is pushed to prod server.

We don't hold the pipeline to the same processes we hold for the source code we're building.

Is the issue that the team is too big? Do we want our team of ten managing a build server, or do we want them delivering value to customers? Do I want 100 teams with their own build server, or do a want a couple guys managing a build server for 1,000 developers?
Having the small team manage their build system empowers them about the build and deployment process.
Having a single team manage it for a huge group of developers gets the developers out of touch with the deployment process.
Developers care more about getting stuff done over reproducibility -- they'll install something to "get it to work" and then forget how it got there.
* Developers manage their server, but the slaves are rebuild nightly.
* All teams have their own build server, but agree on a specific deliverable type and publish it to an artifact repository. There's a master build server that watches the repo and does the integration of all the components.

Does anyone care about traceability?
Getting the code is easy.
What about the tests that run?
How was the build server set up that day?

We can reproduce what we built, can we reproduce how we deployed it? If we change all the build scripts? If we change what tool we used?

Don’t do * includes if you can avoid it.

Tool chain from checkin -> deployment to production

SCM
git
svn
hg
ClearCase
Perforce
Build Server
“Runs the build”
Transform source + dependencies into an artifact. (May also run tests.)
Compile, link, package, test
Static Analysis
Jenkins
BuildForge
AHP
CruiseControl
Artifact repo
Nexus
Artifactory
S3
SCM
Leave it in the build tool
Push it to deployment tool
Infrastructure
Would follow the same chain to go from “source” to “image.”
Long lived physical environments that get set up once
Virtual machines that are treated like long-lived physical machines.
Stand up virtual instances per build
A person might build the machine, build the golden image, build a custom script.
Deploy
Chef/Puppet
Capistrano
uDeploy
Orc Deploy
Shell/Perl/Python/Ruby/etc scripts
A person does everything manually following a document
Database
Liquibase
Rails Migrations
Dacpac
Play Evolution
A person that updates the database.
Test
Performance
Integration
Unit
Deployment/Smoke
Functional
Load
Security
Installation/Manifest Check (make sure all expected files exist)
A person does everything manually following a document
Bees with Machine Guns
HP LoadRunner/WinRunner/etc
Cucumber
Selenium
Fitnesse
Spock
UAT / Business Signoff
Blue / Green deployments
Canary deployment
Person clicks a button
Manual signing of document
Deploy to Production?
Same as deploy to non-production
A person does a partial deployment

Should a build server deploy?
Yes
Get it out there so people can use it
No
Slow tests aren’t run
Deploy to one environment or many?

Conclusion: script all the things. Version all the things. You can do it on Windows.
Treat the systems the same way you are (should be?) treating the software going to the users.
It’s worth it for any size team: one person, ten people, a thousand people.

CITCONNA2013Sessions

2013-08-24T20:36:52Z

Jonny:

CITCON NA 2013 Boston Sessions

Back to the [[Main Page]]

== 10:00 Topics ==

# [[Reproducible CI]]
# [[From 0 percent to greater than 0 percent Test Coverage]]
# [[To Branch or Not to Branch?]]
# [[Property Based Developer Testing]]
# [[Test Expressivity]]

== 11:15 Topics ==

# [[End to end Continuous Delivery tools]]
# [[Test Triage and Intermittent Test techniques]]
# ---
# ---
# [[Managing Test Data All The Way To Production]]

== Lunch Topics ==

# ---

== 2:00 Topics ==

# [[CI Feedback & Metrics]]
# ---
# [[Feature Toggling Strategies]]
# [[CI Challenges With Multiple Branches]]
# ---

== 3:15 Topics ==

# [[NextGen CI Toolkit Wishlist]]
# [[Dead Sea Effect]]
# ---
# ---
# ---

== 4:30 Topics ==

# ---
# ---
# ---
# ---
# ---

== Table View ==

{| class="wikitable"
|-
! Room name
! 10:00
! 11:15
! 2:00
! 3:15
! 4:30
|-
| Horace Mann
| ---
| [[End to end Continuous Delivery tools]]
| ---
| ---
| ---
|-
| Deborah Sampson
| ---
| [[Test Triage and Intermittent Test techniques]]
| ---
| ---
| ---
|-
| Thomas Paul
| [[From 0 percent to greater than 0 percent Test Coverage]]
| ---
| [[CI Challenges With Multiple Branches]]
| ---
| ---
|-
| Abigail Adams
| [[Property Based Developer Testing]]
| ---
| ---
| ---
| ---
|-
| Crispus Attucks
| [[Test Expressivity]]
| [[Managing Test Data All The Way To Production]]
| ---
| ---
| ---
|}