This document describes how to upgrade the Mercurial version deployed to hg.mozilla.org.
Managing Mercurial Packages¶
We generally don’t use Mercurial packages provided by upstream or from distros because they aren’t suitable or aren’t new enough. So, we often need to build them ourselves.
Mercurial RPMs are created by invoking
make targets in Mercurial’s
build system. From a Mercurial source checkout:
$ make -j2 docker-centos6 docker-centos7
This will build Mercurial RPMs in isolated Docker containers and store
the results in the
.rpm files can be found
CentOS 6 RPMs are configured to use Python 2.6, as that is the Python used by CentOS 6 by default. Various Mercurial extensions at Mozilla require Python 2.7. So the system Mercurial RPMs produced via this method aren’t guaranteed to work with everything. Furthermore, Python 2.7 is faster than Python 2.6. For these reasons, it is recommended to avoid running Mercurial from the system Python on CentOS 6. Instead, run CentOS 7 or install Python 2.7 and run Mercurial from a virtualenv.
Building .deb Packages¶
The process for producing Debian .deb packages is similar: run Mercurial’s make targets for building packages inside Docker:
$ make docker-ubuntu-xenial
.deb files will be available in the
Uploading Files to S3¶
Built packages are uploaded to the
moz-packages S3 bucket.
CentOS 6 packages go in the
CentOS6 folder. CentOS 7 packages in the
When uploading files, they should be marked as world readable, since we have random systems downloading from this bucket.
Upgrading And Modifying Templates¶
The repository contains a vendored copy of Mercurial’s templates plus
modifications in the
hgtemplates/ directory. These templates are
used by the hgweb server.
We have made several modifications to the templates. The most significant modification is the addition of the gitweb_mozilla theme, which is a fork of the gitweb theme. We have also made a number of changes to the json theme to facilitate rendering additional data not exposed by Mercurial itself.
When modifying a template, it isn’t enough to simply change a template
hgtemplates/: you must also track that change by recording
it somewhere in
Most modifications are tracked by patch files. Essentially, there exists a standalone patch file describing the change. To modify a template via patch file, do the following:
- Create a new Mercurial changeset like you normally would. i.e. make file
hg committhe result.
- Export the just-created changeset to a standalone patch file. e.g.
hg export . > hgtemplates/.patches/my-change.patch.
- Track the new patch file via
hg add hgtemplates/.patches/<name>.patch.
- Modify the
hgtemplates/.patches/seriesfile and add the new patch file to the list.
run-tests /hgserver/tests/test-template-sync.tand verify the test passes.
test-template-sync.t verifies that the current state of the checkout
matches what would be obtained if all modifications were performed on a fresh
copy of the templates. In other words, it verifies we can reproduce the
current state of the templates.
If the test passes,
hg commit --amend or
hg histedit your
changesets so the modifications to
hgtemplates/.patches are part of
the changeset that modifies files in
hgtemplates/. Then submit that
for review and land when acceptable.
For other modifications (such as adding or removing a file), see the
file lists at the top of
to influence behavior.
When Mercurial is upgraded, we need to synchronize our vendored templates with the new templates from upstream.
To do that, we run the
This script will:
- Wipe away
- Copy the canonical templates from upstream into
- Perform special modifications to templates (notably adding and removing certain files and performing hard-coded template transforms).
- Attempt to apply and commit each patch listed in
Ensure your working directory is clean and
hgtemplates/ is free of
untracked files before continuing. Run
hg revert -C hgtemplates/
hg purge hgtemplates/ to do this.
To perform an upgrade:
$ hgtemplates/.patches/mozify-templates.py /path/to/mercurial/templates \ hgtemplates hgtemplates
$ hgtemplates/.patches/mozify-templates.py venv/mercurials/4.6.2/lib/python2.7/site-packages/mercurial/templates \ hgtemplates hgtemplates
This tells the script to copy templates from the 1st argument, to grab patches and files from the 2nd argument, and to write the result into the 3rd argument.
If everything is successful, several commits would have been made. You can
hg show stack (assuming the
show extension is enabled) to
see them. These changesets can be removed via
hg prune or
without causing harm.
If the script fails, chances are it failed to run
hg import to apply
a patch. Your working directory may or may not be in a good state. Check
hg status and resolve via
hg revert etc as appropriate.
To recover from a non-working patch file, you’ll need to update the
failed patch file until it applies cleanly. To do that, look at the
process output for the name of the patch file that failed to apply. Next,
you’ll attempt to apply it manually. e.g. if the
foo.patch file fails:
$ hg import --partial hgtemplates/.patch/foo.patch
You will then need to resolve any conflicts, fix the files until they are
in the state you want, etc. Then
hg commit --amend the result. This
will produce a new changeset with a working version of the patch.
Next, you will update the failing
.patch file with the new version and
commit the result. e.g.
$ hg export > hgtemplates/.patch/foo.patch $ hg commit -m ‘hgtemplates: update foo.patch for Mercurial 4.7 upgrade’
Then you need to start the template upgrade process over from the beginning
with the modified
.patch file in place. e.g.:
$ hg up @ $ hg rebase -s tip -d . $ hgtemplates/.patches/mozify-templates.py venv/mercurials/4.6.2/lib/python2.7/site-packages/mercurial/templates \ hgtemplates hgtemplates
You can also safely
hg prune or
hg strip the changesets produced by
Once you’ve repeated this process and
hgtemplates/ now contains the upstream templates plus
hgserver/tests/test-template-sync.t so it picks up
the Mercurial templates from the appropriate Mercurial version in its
mozify-templates.py invocation. And run this test and verify all is
happy. Then commit that change.
At this point, the repository has several commits. There could be
hgtemplates/.patches/. There will be changesets
tracking the upstream changes to
hgtemplates/ and changes made by
each patch. And there should be a changeset for the change to
At this point, it is recommended to run
hg histedit and roll all the
changesets together. This will produce a unified changeset containing every
change. It should effectively be a diff of the upstream changes plus whatever
changes to patches were needed to accommodate upstream changes. This
changeset should be suitable for review.