GitForEmacsDevs

Introduction

Some features described on EmacsWiki are available only in the latest EmacsDevelopmentVersion. To obtain those features, you will need to build Emacs using the development source code. This page is about that source code and how to use it. First and foremost, every contributor should be acquainted with and follow the guidelines in the source code repository CONTRIBUTE file.

This is a quick-start guide to using Git for Emacs development. It presents a simple Git workflow oriented toward the “regular contributor,” whether a “core” committer, a maintainer of a well-defined package (e.g., a major mode), or simply someone who enjoys occasionally hacking on Emacs and then contributing those improvements to the community. The emphasis is on necessary and simple, not on “general” nor on maximal efficiency. Emacs developers will surely need various other workflows at times, and there are in most cases more highly performant ways of accomplishing most tasks. However, the ways presented here are intended to be straightforward, effective, reasonably efficient, and relatively simple.

“VCS means never having to say you’re sorry.” It’s true, but it’s still worth being careful, even with git. In the workflows described below, a quick “git status” is inserted before each commit. This is optional, of course, but it’s generally a good idea if (a) you haven’t committed in that branch within your short-term memory, or (b) you’ve done a merge.

Centralized workflow

The workflow documented on this page takes advantage of the fact that Git is a decentralized version control system. It is especially useful when you are working on large changes to Emacs.

If you just need a quick and simple workflow, see GitQuickStartForEmacsDevs. That page describes a simple CVS-like centralized workflow, which allows you to contribute changes to Emacs as quickly as you can, with a minimum amount of extra effort. One advantage of that workflow is that you can use VC-dir without complications.

Later, when you have a chance, come back here and learn the workflow described on this page. This workflow hews more closely to what experienced Git users are accustomed to, which will make it easier for you to get support online. One disadvantage of this workflow is that VC-dir support is still somewhat lacking.

Terminology

If you were last involved in Emacs development before Bazaar, and find the terminology in this guide confusing, read Git for CVS Users and come back.

If you are used to Bazaar, bear in mind that the data you will clone when using Git is not for a branch but for the entire repository with all its branches. You can switch between these branches as you need. You’ll still be pushing commits to a branch, either the master branch, or the release branch (currently called emacs-24, but that will change as future Emacs versions are released).

The command examples are generally given in Unix command-line style, but they are equally applicable to MS-Windows, if you use the Bash prompt installed and configured by Git for Windows.

Getting Started

Getting help on Git

Git provides both Unix-style man pages (start with man git; Git for Windows because doesn’t install man pages, use git help git instead) and embedded help (start with git help). See OtherResources for more documentation.

The Git Work Routine

Overview of the Git work routine:

  1. Tell Git your name and email address; these will be used as defaults to identify your commits. (Usage: git config --global user.name “Frank Chu”; git config --global user.email fchu@example.com”)
  2. Clone the repository: git clone <membername>@git.sv.gnu.org:/srv/git/emacs.git
  3. Do trivial, single-commit work on master and commit to upstream from there; do your non-trivial work in local feature branches, committing locally as you go (more about that below).
  4. From time to time, update to get upstream newness with git pull (this is like cvs update or bzr up).
  5. When a task is finished (a local branch’s changes are complete), then either
    1. merge them into the master branch and push (if you’re a maintainer and have the necessary access rights — this is analogous to committing in CVS), or
    2. package up your changes with git format-patch and send them as a patch series to the project mailing list (analogous to posting a patch).
  6. Receive feedback.
  7. Lather, rinse, repeat until the change is accepted.
  8. If you did all this on a local feature branch, you can remove the branch now. The change history will live upstream.

Now let’s look at the specific git commands.

Personalizing Git

Unlike centralized VCSes where you are identified by an account in the repository, in Git you can clone repositories in many places, and you may be sending changes to many places (including, but not limited to, the upstream Emacs repo). So Git stores your identity locally, and includes it as part of every revision you commit. This information is your name and email address. To configure Git to use them, do

      git config --global user.name "J. Random Hacker"
      git config --global user.email "jrandom@gnu.org.invalid"

When you are integrating somebody else’s work, you will want to be identified as the committer (so you can handle technical issues with the commit), but not as the author. In that case you can use the --author option to git commit to specify the author.

You should also do

      git config --global transfer.fsckObjects true

(See the thread "Recommend these .gitconfig settings for git integrity." for a detailed explanation of why, but basically this tells Git to check the integrity of data it receives. Also, if you already set either of the fetch.fsckObjects or receive.fsckObjects config options to “false” in your ~/.gitconfig, you should change them to “true”, or just remove them so that the “true” value from ‘transfer.fsckObjects’ is used instead.)

Merge Driver for ChangeLog files

The default merge driver is not aware of the special structure of ChangeLog files and thus will frequently create merge conflicts in those files when you merge your local changes with upstream, or merge 2 branches that diverged a little. The Gnulib project contains a merge driver for ChangeLog files that avoids these problems. Please consider installing it; for the instructions, see UsefulAddOns near the end of this page.

Initializing Your Local Git Repository

This section describes how to set things up if you plan on making contributions to Emacs (whether or not you’re one of the Emacs maintainers). You can also use this way just to track the Emacs sources, even if you never contribute changes.

For the general public, Emacs’s repo lives at git://git.sv.gnu.org/emacs. Committers should prefix it with their Savannah user-id: <membername>@git.sv.gnu.org:/srv/git/emacs.git. Apply git clone to these; it will clone the repo into a directory named ‘emacs’:

      git clone <membername>@git.sv.gnu.org:/srv/git/emacs.git  # For those with write access
      git clone git://git.sv.gnu.org/emacs.git                  # For the rest of us

Workflow for Regular Development

Doing Quick Fixes

You can do quick, self-contained fixes (one-line patches and the like) directly to the master branch if you want:

      cd $DEVHOME/emacs/
      git pull                    # get in sync with the upstream master
      <<<make your edit>>>
      git status                  # (optional) sanity check -- lists changed and unknown files
      git commit -a -m "Fix silly typo in README"
      git show                    # double-check what you will be pushing
      git push

The last git push command might fail if someone else pushed their changes while you were working on yours. In that case, you will need to re-sync with upstream:

      git pull --rebase

If this fails and announces merge conflicts, resolve the conflicts by editing the files mentioned in the error message, and tell git to continue with rebasing:

      git rebase --continue

Finally, type git push again to push to upstream.

git will automatically identify you as the committer and author of these changes (see PersonalizingGit). If you are integrating another developer’s work, you can use --author 'J. Other Hacker <jhacker@gnu.org.invalid>' to identify JO Hacker as the author of the change.

Note: For larger tasks, such as a complicated bug fix or new feature that might require multiple commits and rounds of feedback, we always recommend creating a “task branch” dedicated to that fix or feature – see FeatureBranch.

Creating a Task Branch

When you are working on a task that touches many files, or requires a lot of thought and several stages, or for any reason may take a fair amount of time, it’s a good idea to work on a separate branch to keep the changes from interfering with concurrent work (including your own!), and to keep the history separate from other work. This makes review easier, too. So first create a new branch:

       cd $DEVHOME/emacs
       git checkout -b TASKNAME

…where “TASKNAME” could be, e.g., the bug tracker ticket number of the bug you’re trying to fix, or just a short, descriptive name for the task.

Working in a Task Branch

At this point, you’re ready to do work. You edit the source files, build and test as needed. When you’ve accomplished a subtask, it’s a good idea to stop and commit your work to the branch. Here, you often have a larger commit log message than cannot comfortably be entered on the command line with -m. In this case, you put the log message in a file we’ll call log-message.txt. Then

      git status                  # (optional) sanity check -- lists changed and unknown files
      git add FILE1 ...           # to tell Git to track any new files created for this task
      git commit -a -F log-message.txt

You can also do git commit -m "blah blah blah", if your log message is short enough. Remember, these commits are local: they affect only your task branch, not the upstream repo. No one else will see the commits until you send them upstream.

(Note: the Emacs project will continue to keep versioned ChangeLog files. See this mail for more.)

If the task isn’t done yet, keep working. Edit yet more source files, until you reach a good stopping point:

      git status                   # (optional) sanity check -- lists changed and unknown files
      git add more-new-files ...
      git commit -a -F another-log-message.txt

etc. You can commit any time you reach a good checkpoint. We recommend committing much more frequently than you might with CVS or Subversion. This makes it easier to find the commit that introduced a bug.

From time to time you may wish to refresh the local repository by updating to get changes from upstream:

      git checkout master
      git pull

And after refreshing, you’ll want to get those changes into your task branch, by merging them:

      git checkout TASKNAME
      git merge master
      git status                  # (recommended) sanity check -- lists changed and unknown files

The merge may fail due to an upstream change that conflicts with your branch. You’ll need to fix the problem (looking for conflict markers and editing, just as in CVS and Bazaar). Then use git add on the resolved file to mark that it should be added to the next commit.

      git add file-you-changed ...      # Mark these conflicts as resolved.
                                        # Git keeps track of unresolved conflicts,
                                        # and won't let you commit until all are resolved.

If you fix conflicts by editing the files in Emacs, you should see the smerge-mode turned on automatically for files with conflict markers. Saving files when smerge-mode is active automatically invokes git add for that file, so manual git add commands are no longer necessary.

Once you have achieved a clean merge, commit it:

      git status                  # (recommended) sanity check -- lists changed and unknown files
      git commit -m "Merge from mainline"

Note that you will need this commit command only if conflicts were detected during the merge.

Once you have completed the task, you’ll want to send it for review or maybe even put it into upstream. You can send for review either by publishing your branch (see PublishBranch) or sending a bundle (see SendABundle), and if you are a committer you can merge to upstream yourself (see MergeToUpstream).

Good Commenting Practice

See the CONTRIBUTE file in the Emacs source code repository for more up-to-date commit-log and other instructions.

Commit log messages should have a standalone first line not over 72 characters long. Do this so the summary listings displayed by git log --oneline and similar tools are intelligible.

Avoid using git hashes as references to previous commits. These will break if we ever have to change version-control systems again; given that Emacs has been through no fewer than four VCSes in its lifetime, it is not a good idea to depend on this never happening again. Instead, quote a distinguishing part of the comment for the commit, or write something like “last commit” when appropriate.

Sending a Patch Set for Review

Another way to post changes for review is to create a “patch set”, which is a sequence of files containing your change, each in a special augmented patch format that Git understands:

      git format-patch <BEGIN>..<END>

where <BEGIN> and <END> identifies the range of commits in your change. This will produce a sequence of files, one per commit in the sequence, each with a name generated from the summary line of the commit.

Send mail to either the Emacs Devel list or the list for the bug. Describe the change and attach the patch set to that mail (please use MIME type text/plain). If the reviewers ask for further tweaks, repeat the cycle. Edit the files, build and test as necessary, and

      git status                  # (optional) sanity check -- lists changed and unknown files
      git commit -m "Address reviewer comments"

Merging Into the Upstream Master

If you are a committer, you can merge to the upstream master directly.

First, update your repository:

      cd $DEVHOME/emacs
      git checkout master
      git pull
      git merge TASKNAME

Run the tests:

      make check

and then commit

      git status                  # (recommended) sanity check -- lists changed and unknown files
      git commit -m "fixes debbugs:12345"

which merges all your new commits to the upstream master. Now last-minute check of what you are going to deliver:

      git show

Finally, push upstream

      git push

Alternatively, if you use VC in Emacs, Emacs will populate the commit message from your ChangeLog updates. To use VC:

  vc-dir the root directory of Emacs sources
  vc-dir-mark to mark files to commit
  vc-next-action to edit a commit message
  log-edit-insert-changelog to populate from ChangeLogs, possibly using prefix-arg
  vc-next-action to commit

Note that VC does not yet support git push, so you will need to type that command manually.

In this process, updating your master with git pull is not really optional. There is a race condition here, just as in CVS and Bazaar: if your mirror is not up-to-date because another developer has pushed since you last updated, your push will fail. In this case you should abort the merge:

      git reset

Then do another git pull, merge from the TASKNAME branch again, and push.

When you’re done with your work and it’s all been sent upstream, you can delete your task branch…

      git branch -d TASKNAME

When you decide to work on the next task, create a new task branch:

      cd $DEVHOME/emacs
      git checkout -b NEW_TASKNAME

…and you know what to do from here.

Working with the Release Branch

Switching Between Master and the Release Branch

If you are a frequent contributor or a core developer, you will need to work both on the master branch and the release branch (the latter is currently called “emacs-24”, but that changes with every major release). In Emacs development, changes that fix bugs that are present on the release branch are pushed there and later merged to the master branch. In addition, some changes pushed to master are back-ported to the release branch.

This means you will have to switch frequently between the master and the release branch. Switching branches in git is easy:

      git checkout emacs-24  # switch to the release branch
      git checkout master    # switch to the master branch

In general, you can use the same workflow with the master and the release branches as outlined above for a feature branch. However, due to the high rate of changes on master, it diverges from the release branch very quickly: there are many changes in infrastructure, many new files appear, old files are deleted, etc. The result of this is that switching between master and the release branch almost always needs a long build (to byte-compile many Lisp files), and frequently will require a full bootstrap. If the time this takes annoys you, you can use one of the following alternatives.

The first alternative is simply to clone the upstream Savannah repository twice:

      git clone <membername>@git.sv.gnu.org:/srv/git/emacs.git emacs-trunk
      git clone <membername>@git.sv.gnu.org:/srv/git/emacs.git emacs-branch

This will clone the repository into 2 sibling directories, emacs-trunk and emacs-branch (you can use any other names, of course). By default, cloning always checks out the master branch, so you need to switch to the release branch in the emacs-branch clone:

      cd emacs-branch && git checkout emacs-24

After this one-time setup, you do all the master-related work in emacs-master and all the release branch-related work in emacs-branch; switching branches then boils down to switching directories. Don’t forget to issue the git pull command in each branch, before you begin making changes!

This workflow has the advantage of being very simple, but it wastes disk storage, since the repository meta-data is stored twice. If you want to avoid waste of storage, you can use 2 variations:

  1. Make the second clone local. With this method, you make the second clone from the first one, not from the Savannah URL. Then the common meta-data files will be hard-linked, and only one copy will actually be present on the disk. To update the second clone, issue the git pull command in the 1st clone, then one more time in the second one.
  2. Use the git-new-workdir script to create a second working tree from the same repository meta-data. Please read the documentation of that script for the details. As of git version 2.5, the builtin git worktree command is preferred.

Merging Changes from the Release Branch to Master

From time to time, bugfixes pushed to the release branch should be merged onto master. We recommend to use the admin/gitmerge.el file for that purpose. It will take care of many things which would otherwise have to be done manually, like ignoring commits that should not land in master, fixing up ChangeLog entries, and automatically dealing with certain types of conflicts.

Before merging, make sure to git pull to get your repository in synch with upstream.

Then load gitmerge.el and invoke M-x gitmerge RET in a buffer whose default-directory is the top-level repository directory. It will prompt for the branch from which to merge, defaulting to origin/emacs-24, which you should accept.

You will then be presented with a list of commits not yet merged to master, with some of them already marked for being skipped (because whoever pushed them decided they shouldn’t be merged to master for some reason). Review the commits, using ‘s’ to toggle their “skip” mark as needed. You can use the ‘l’ command to show the log of the commit at point, ‘f’ to show the files it modified, and ‘d’ to show its diffs. When you’re done, hit ‘m’ to start the merge.

If there are merge conflicts that gitmerge.el cannot fix automatically, it will stop and present the list of files with conflicts. Resolve these conflicts, save the modified files, and invoke M-x gitmerge again in a buffer whose default-directory is the top-level repository directory.

When you are done with resolving the conflicts, the results of the merge will be committed locally to the master branch of your repository. Review the merge-commit one last time:

      git diff origin/master

If satisfied, push:

      git push

If the push fails because someone pushed since your last git pull, rebase on the latest master and push again:

      git pull --rebase
      git push

Back-porting Changes from Master to the Release Branch

Sometimes, a change pushed to master will be deemed to need to be back-ported to the release branch. To do this, use the cherry-pick method. First use the git log command to find the commit you are looking for. Then switch to the release branch, update it, and do the cherry-pick:

      cd ../emacs-branch     # only if you use a separate directory for emacs-24
      git pull
      git checkout emacs-24  # make sure you have the release branch checked-out
      git cherry-pick -xe COMMIT

where COMMIT will typically be the SHA1 checksum of the commit you want to back-port, copy-pasted from the git log display. You can use only the 7 leading hex digits of the SHA1 value.

The -e option you pass to Git allows you to edit the commit message; use that to add an indication of back-porting, e.g. add “Backport:” to the commit log. This is so gitmerge.el will skip this commit during the next merge onto master.

Finally, review your back-port and push:

      git show
      git push

Resources

If you’ve read this far, you should have a basic understanding of Git, and how to develop Emacs using Git. Below are some more resources that you or people you work with might find useful.

Resources for Casual Developers

Ideally, one-time contributors, beta testers, and anyone else needing to follow bleeding-edge Emacs would use the same workflow as described above for regular contributors. However, in case that workflow feels like too much trouble, then GitQuickStartForEmacsDevs offers a couple of slightly simpler setups.

Even if you use one of those simpler setups at first, we still encourage you to migrate to the RegularContributors workflow as soon as you can – not because you necessarily plan to become a regular contributor, but because then your setup will be the same as what most other people are using, so they can better help you if you have questions.

Useful Add-On Configuration

We have mentioned the git-merge-changelog tool above. Here are some instructions on installing it and on configuring your Git to use it.

It can be installed using

    git clone git://git.savannah.gnu.org/gnulib.git
    cd gnulib
    ./gnulib-tool --create-testdir --dir=/tmp/testdir123 git-merge-changelog
    cd /tmp/testdir123
    ./configure
    make
    make install

Some GNU/Linux distributions also provide packages for it.

MS-Windows users can download a pre-compiled binary from the ezwinports site.

To enable git-merge-changelog, add to .git/config of the checkout (or to your $HOME/.gitconfig) the lines

    [merge "merge-changelog"]
      name = GNU-style ChangeLog merge driver
      driver = /usr/local/bin/git-merge-changelog %O %A %B

(change the “/usr/local/bin/” part to name the directory where you installed the program). To the ~/.gitattributes file add the line

    ChangeLog    merge=merge-changelog

See gitattributes(5) for more info. Additionally it can be useful to define a custom hunk-header (the text shown after the @@ in the textual diff output) for Emacs lisp related files. Add the following lines to the .gitconfig file

    [diff "lisp"]
      xfuncname = "^(\\(.*)$"
    [diff "texinfo"]
      xfuncname = "^@node[ \t][ \t]*\\([^,][^,]*\\)"

and to the .gitattributes file

    *.c     diff=cpp
    *.h     diff=cpp
    *.el    diff=lisp
    *.texi* diff=texinfo

Other Resources


CategoryContributingToEmacs