About the chef-repo

The chef-repo is the location in which the following data objects are stored:

• Cookbooks (including recipes, attributes, custom resources, libraries, and templates)
• Roles
• Data bags
• Environments

The chef-repo is located on a workstation and should be synchronized with a version control system, such as git. All of the data in the chef-repo should be treated like source code.

knife is used to upload data to the Chef server from the chef-repo. Once uploaded, that data is used by the chef-client to manage all of the nodes that are registered with the Chef server and to ensure that the correct cookbooks, environments, roles, and other settings are applied to nodes correctly.

Directory Structure

The chef-repo contains several directories, each with a README file that describes what it is for and how to use that directory when managing systems.

The sub-directories in the chef-repo are:

Directory	Description
.chef/	A hidden directory that is used to store validation key files and the knife.rb file.
cookbooks/	Contains cookbooks that have been downloaded from the https://supermarket.chef.io or created locally.
data_bags/	Stores data bags (and data bag items) in JSON (.json).
environments/	Stores environment in Ruby (.rb) or JSON (.json).
roles/	Stores roles in Ruby (.rb) or JSON (.json).

.chef/

The .chef directory is a hidden directory that is used to store validation key files and the knife.rb file. These files are required for interaction with a Chef server.

cookbooks/

The cookbooks/ directory is used to store the cookbooks that are used by the chef-client when configuring the various systems in the organization. This directory contains the cookbooks that are used to configure systems in the infrastructure. Each cookbook can be configured to contain cookbook-specific copyright, email, and license data.

data_bags/

The data_bags/ directory is used to store all of the data bags that exist for an organization. Each sub-directory corresponds to a single data bag on the Chef server and contains a JSON file for each data bag item. If a sub-directory does not exist, then create it using SSL commands. After a data bag item is created, it can then be uploaded to the Chef server.

environments/

The environments/ directory is used to store the files that define the environments that are available to the Chef server. The environments files can be Ruby DSL files (.rb) or they can be JSON files (.json). Use knife to install environment files to the Chef server.

roles/

The roles/ directory is used to store the files that define the roles that are available to the Chef server. The roles files can be Ruby DSL files (.rb) or they can be JSON files (.json). Use knife to install role files to the Chef server.

chefignore Files

The chefignore file is used to tell knife which cookbook files in the chef-repo should be ignored when uploading data to the Chef server. The type of data that should be ignored includes swap files, version control data, build output data, and so on. The chefignore file uses the File.fnmatch Ruby syntax to define the ignore patterns using *, **, and ? wildcards.

• A pattern is relative to the cookbook root
• A pattern may contain relative directory names
• A pattern may match all files in a directory

The chefignore file is located at the root of the /cookbooks subdirectory in the chef-repo. It should contain sections similar to the following:

# section
*ignore_pattern

# section
ignore_pattern*

# section
**ignore_pattern

# section
ignore_pattern**

# section
?ignore_pattern

# section
ignore_pattern?

Examples

The following examples show how to add entries to the chefignore file.

Ignore editor swap files

Many text editors leave files behind. To prevent these files from being uploaded to the Chef server, add an entry to the chefignore file. For Emacs, do something like:

*~

and for vim, do something like:

*.sw[a-z]

Ignore top-level Subversion data

If Subversion is being used as the version source control application, it is important not to upload certain files that Subversion uses to maintain the version history of each file. This is because the chef-client will never use it while configuring nodes, plus the amount of data in an upload that includes top-level Subversion data could be significant.

To prevent the upload of top-level Subversion data, add something like the following to the chefignore file:

*/.svn/*

To verify that the top-level Subversion data is not being uploaded to the Chef server, use knife and run a command similar to:

$ knife cookbook show name_of_cookbook cookbook_version | grep .svn

Ignore all files in a directory

The chefignore file can be used to ignore all of the files in a directory. For example:

files/default/subdirectory/*

or:

files/default/subdirectory/**

Many Users, Same Repo

It is possible for multiple users to access the Chef server using the same knife.rb file. (A user can even access multiple organizations if, for example, each instance of the chef-repo contained the same copy of the knife.rb file.) This can be done by adding the knife.rb file to the chef-repo, and then using environment variables to handle the user-specific credential details and/or sensitive values. For example:

current_dir = File.dirname(__FILE__)
  user = ENV['OPSCODE_USER'] || ENV['USER']
  node_name                user
  client_key               "#{ENV['HOME']}/chef-repo/.chef/#{user}.pem"
  validation_client_name   "#{ENV['ORGNAME']}-validator"
  validation_key           "#{ENV['HOME']}/chef-repo/.chef/#{ENV['ORGNAME']}-validator.pem"
  chef_server_url          "https://api.opscode.com/organizations/#{ENV['ORGNAME']}"
  syntax_check_cache_path  "#{ENV['HOME']}/chef-repo/.chef/syntax_check_cache"
  cookbook_path            ["#{current_dir}/../cookbooks"]
  cookbook_copyright       "Your Company, Inc."
  cookbook_license         "apachev2"
  cookbook_email           "[email protected]"

  # Amazon AWS
  knife[:aws_access_key_id] = ENV['AWS_ACCESS_KEY_ID']
  knife[:aws_secret_access_key] = ENV['AWS_SECRET_ACCESS_KEY']

  # Rackspace Cloud
  knife[:rackspace_api_username] = ENV['RACKSPACE_USERNAME']
  knife[:rackspace_api_key] = ENV['RACKSPACE_API_KEY']

Create the chef-repo

There are two ways to create a chef-repo when using the Chef boilerplate repository as a base:

• Clone the chef-repo from GitHub
• Download the chef-repo as a tar.gz file and place it into local version source control.

Generate

To create a chef-repo, run the following command:

$ chef generate repo REPO_NAME

This command uses the chef command-line tool that is packaged as part of the Chef development kit to create a chef-repo.

Use git

Chef is maintained on GitHub. To contribute to Chef, such as submitting a pull request, requires using GitHub and git. The sections below describe how to use git to set up the Chef repository, keep it current and synchronized, and how to use branches to submit pull requests.

Set Up Repo

Use the following steps to set up a development repository for Chef:

1. Set up a GitHub account.

2. Fork the https://github.com/chef/chef repository to your GitHub account.

3. Clone the https://github.com/chef/chef repository:

   $ git clone [email protected]:yourgithubusername/chef.git

4. From the command line, browse to the chef/ directory:

   $ cd chef/

5. From the chef/ directory, add a remote named chef:

   $ git remote add chef git://github.com/chef/chef.git

6. Verify:

   $ git config --get-regexp "^remote\.chef"

   which should return something like:

   remote.chef.url git://github.com/chef/chef.git
   remote.chef.fetch +refs/heads/*:refs/remotes/chef/*

7. Adjust your branch to track the chef/master remote branch:

   $ git config --get-regexp "^branch\.master"

   which should return something like:

   branch.master.remote origin
   branch.master.merge refs/heads/master

   and then change it:

   $ git config branch.master.remote chef

Keep Master Current

Use the following steps to keep the master branch up to date.

1. Run:

   $ git checkout master

2. And then run:

   $ git pull --rebase

The following rakefile can be used to update Chef, Ohai, and cookbooks. Edit as necessary:

projects = %w[chef cookbooks ohai]
chef = "#{ENV['HOME']}/projects/chef"

desc 'Update local repositories from upstream'
task :update do
  projects.each do |p|
    Dir.chdir('#{chef}/#{p}') do
      sh 'git fetch chef'
      sh 'git rebase chef/master master'
    end
  end
end

Sync Master

Use the following steps to synchronize the master branch.

1. Run:

   $ git fetch chef

2. And then run:

   $ git rebase chef/master master

   Note

   Use rebase instead of merge to ensure that a linear history is maintained that does not include unnecessary merge commits. rebase will also rewind, apply, and then reapply commits to the master branch.

Use Branch

Commits to the Chef repositories should never be made against the master branch. Use a topic branch instead. A topic branch solves a single and unique problem and often maps closely to an issue being tracked in the repository. For example, a topic branch to add support for a new init system or a topic branch to resolve a bug that occurs in a specific version of CentOS. Ideally, a topic branch is named in a way that associates it closely with the issue it is attempting to resolve. This helps ensure that others may easily find it.

Use the following steps to create a topic branch:

1. For a brand new clone of the Chef repository (that was created using the steps listed earlier), fetch the opscode remote:

   $ git fetch chef

2. Create an appropriately named tracking branch:

   $ git checkout --track -b CHEF-XX chef/master

   Set up a topic branch to track chef/master. This allows commits to be easily rebased prior to merging.

3. Make your changes, and then commit them:

   $ git status

4. And then run:

   $ git commit <filespec>

5. Rebase the commits against chef/master. After work in the topic branch is finished, rebase these commits against the upstream master. Do this manually with git fetch followed by a git rebase or use git pull --rebase.

   git will let you know if there are any problems. In the event of problems, fix them as directed, and then mark as fixed with a git add, and then continue the rebase process using git rebase --continue.

   For example:

   $ git fetch chef

   followed by:

   $ git rebase chef/master CHEF-XX

   Or:

   $ git pull --rebase

6. Push the local topic branch to GitHub:

   $ git push origin CHEF-XX

7. Send a GitHub pull request for the changes, and then update the Chef ticket with the appropriate information.

Delete Branch

After work has been merged by the branch maintainer, the topic branch is no longer necessary and should be removed.

1. Synchronize the local master:

   $ git checkout master

   followed by:

   $ git pull --rebase

2. Remove the local branch using -d to ensure that it has been merged by upstream. This option will not delete a branch that is not an ancestor of the current HEAD. From the git man page:

   -d
     Delete a branch. The branch must be fully merged in HEAD.
   -D
     Delete a branch irrespective of its merged status.

3. Remove the local branch:

   $ git branch -d CHEF-XX

   Or remove the remote branch by using the full syntax to push and by omitting a source branch:

   $ git push origin :CHEF-XX