Continuous Delivery with Puppet - what is missing?

We met in Meeting Room One (with standing room only!).

Andrew_Parker kicked off with "Who is using Continuous Deployment"?

Andy wrote on the chalkboard

• Drift from management config
• Systems not designed for component management
• Simplify
• SE Practices
• Clear vision
• Case studies
• Dist systems too hard

Issues Discussed Related to Using Puppet for Continuous Deployment

• Just removing something from the manifest, doesn't mean that it will be removed from the server
• Drift from management configuration
• Keep all possible environments in one repository, don't use branches and cherry-pick.
• The number one thing that PuppetLabs could do would be to be clear about how these tools should be used...Active Evangelism. The conversation [in the industry] today is dominated by people who see it as an alternative to scripting
• Tools like Puppet are able to incorporate everything we have learned about answering difficult questions like "Has a service started?"
• One problem that caused swearing, according to Squirrel, knowing which Puppet Modules are being deployed
• Dry run is only partially good in Puppet, because the runtime behavior may change
• Don't do multiple systems in one PuppetMaster
• Do Puppet masterless, run Puppet apply
• Consider packing everything up into an archive so that the Puppet scripts live with the application. Put all the scripts
• 3 models with Puppet
  • Standalone
  • Multiple Puppet Masters - one for each environment
  • "Environments" in a Puppet Master - which DOESN'T work
• When going headless, use Hiera to hold environment specific variables
  • Supports GPG encryption for "secret" stuff
• How do you handle use cases like "Developers cannot have access to Production"
  • Sounds like a concern outside of puppet
  • Can use different repositories
  • Could use Hiera with different keys

General Notes Not Related Specifically to Using Puppet

• Don't let your development team create software that requires OS installs
• Some teams like to use another tool called Ansible
• JTF likes starting from a clean VM every time
• Companies like Netflix create a base AMI and then use that AMI across environments, like Staging, Development, Production
• Simplify...use tools like Puppet for what they do well, but as soon as your pipeline becomes too complex, simplify
• Infrastructure management comes from a different background than design/development. As soon as you start treating "infrastructure as code", design principles apply.
• The Difference Between Scripting vs. State Propagation
  • In scripting, order is important
  • Scripts must check for themselves if a service is running
  • Tools like Puppet allow one to imply that a service is running, and the tool decides how to handle it.
    • Similar to the way a version control system [GIT was used as the example] decides whether or not to pull anything down, or push anything up
  • State assertions make the system far more module
• Distributed logic is hard. Why would you create autonomous distributed logic if you don't have to.
• Look at [1] for a wealth of information
• Software is not convergent any more
• MCollective used for distributed system management

Final Statements

• Look for ways to remove complexity!
• Most big Puppet installs move to Masterless Puppet (using Ansible to control it because Ansible includes functionality like mCollective)