Committer how-to

Page Contents

This page applies to Committers only, not to usual contributors. A Committer is a person with extra rights who receives his or her status via invitation. You don't need to be a Committer to contribute, anyone can fork and send pull requests on Github; see more on the page for contributors.

Git commit policies

Committers can commit directly into the Apache Git repository of FreeMarker, and so preferably don't use GitHub pull requests. (See the repositories and branches here.)

Committers have the technical permissions to commit directly into the "main" branch (or branches), and they are expected to use that wisely. Depending on the complexity of the feature, the number of active Committers in the time period, and whether the feature is experimental, they may should work in a feature branch instead. If the feature is finished and discussions were resolved (on some way described by Apache policies), it can be merged back into the "main" branch.

All commits should have helpful comments; don't be afraid of writing long descriptions. If the commit is related to a Jira issue, the comment should start with the Jira issue identifier, like "FREEMARKER-16: ..."

Merging in pull requests from GitHub

Pull requests from GitHub are merged in by Committers. Before accepting a pull request, ensure that:

  • You have checked all the changes. In particular, the files (with a few exceptions like the README.md) must have the standard Apache copyright headers, and must not state that they have a different license.

  • The contributor has an ICLA or CCLA at Apache (see on http://people.apache.org/). If not, the pull request can be merged only if the commit only did trivial changes (like fixing typos) that would be easy to replicate in case legal issues are raised later. In particular, new files added is a sign that ICLA/CCLA will be needed.

  • The contributor has used the proper branch of the project (it's sometimes overlooked)

The commit comment should automatically describe that it's a merge and where it was merged from. After that you may also want to add a summary of what the merged branch does. In general, pull request merge commits should adhere to the same policies that Committers use to commit directly into the Apache git repository.

To do the actual merging, if your Apache and Github accounts are linked (see how on the ASF Writable Git Services homepage), the Merge (and Close) button on Github should appear and work.

If you can't/don't want to use Github, use this command on your clone of the Apache repo (not the Github repo) (need to be tested if it still works after the migration to Gitbox):

git pull --no-ff https://github.com/apache/<PROJECT> refs/pull/<PR_NUMBER>/head

Where:

  • <PROJECT> is usually freemarker

  • <PR_NUMBER> is the pull request number that GitHub shows prominently after a "#"

Pushing such a merge commit to the ASF repo will automatically close the pull request on Github.

Closing GitHub pull requests without merging

You should state the reason of closing in the closing comment.

Yet again, if your Apache and Github accounts are linked, the Close button on Github should appear and work.

Without linked accounts, we used this command on the clone of the Apache repo (but apparently it doesn't work on non-master branch anymore):

git commit --allow-empty -m "closes apache/<PROJECT>#<PR_NUMBER>: <WHY>"

Where:

  • <PROJECT> is usually freemarker

  • <PR_NUMBER> is the pull request number that GitHub shows prominently after a "#"

  • <WHY> is the reason of the closing. For clarity, you may want to end it with "Closed PR without merging." or something similar.

Making releases

For each release, one of the Committers takes on the role of Release Manager. The Release Manager should follow the procedure here, and should update this page where they has to deviate from it.

For general info about making releases, please read https://www.apache.org/legal/release-policy. The FreeMarker-specific additions to those are:

  • The official release is provided as source code to build the project (src tar.gz). For the convenience of users the release is also accompanied by compiled binaries (bin tar.gz, which also contains the documentation). Both artifacts are downloadable from the Apache Software Foundation, as usual.

  • Releases should have a usual jar artifact, an src artifact and javadoc artifact either in the Apache Maven staging repository (for RC-s), or in the Maven Central Repository (for finals).

  • In the 2.3.x branch, we release two products, the normal and the GAE (Google App Engine) version. Hence when executing the release steps, you do many of them twice. Note that source code changes should be made in the 2.3-gae branch, and then merged into the 2.3 branch (unless a change only applies to 2.3).

  • Releases with deeper changes sometimes should have one or more Release Candidate releases (RC01, RC02, etc.), which is legally a normal release (not just a internal/developer one), and is promoted on our web page. The differences to a final release are these:

    • RC-s aren't released to the Maven Central Repository, only to the Apache Maven repository

    • RC documentation (Manual + JavaDoc) is only published under https://freemarker.apache.org/builds/X.X.X-rcX/

    • RC-s promise no backward compatibility for the new features they add; they may be redesigned or dropped till the final release comes out.

The steps of making a release:

  1. Check that version.properties is up to date.

  2. Ensure that the version number in the title and version history of the Manual is up to date. Also the release date Version history section of the Manual has to be updated. That's tricky as you can't know it exactly, yet you can't change it when the actual release is about to happen, as that would change the release artifacts that were voted upon. So write the current date and "+ release process" after it; that's how it will be in the released artifacts.

  3. Ensure that all changes in the 2.3-gae branch were merged into the 2.3 branch. (Also that both branches where git push-ed.)

  4. Do a clean checkout of the branch to release, just to ensure that you won't include any files temporary files that you put there during development. Work in that checkout from now on. Don't forget to create the build.properties in it.

  5. ant dist

  6. ant rat, check output. There can't be any binaries, archives, and especially not unknown or unapproved licenses in the reports. Skip to the next point if there were none! Otherwise, check if the problematic files are fine legally. If they weren't produced in this project, add them to the related LICENSE files (there are several of them, depending on if it's for the source, the binary distribution, the Maven jar artifact, or the Maven javadoc artifact). You also have to add them to the related rat-excludes file (there's one in the project root for the source release, and another in src/dist/bin for the binary release) so that the files won't be reported next time. Note that rat-exludes should contain a comment that clarifies the origin of the files. Finally restart the process from rebuilding the distribution.

  7. Compare the jar to the earlier stable release for API compatibility issues (both source and binary level). So far we have used "Java API Compliance Checker" (JAPICC) with these arguments: -keep-internal -skip-internal-types \._ -l FreeMarker old.jar new.jar. Note that it's prone to generate false alarms, so simply review each problems and decide which is real.

  8. Ensure that this is up to date: https://dist.apache.org/repos/dist/{dev,release}/freemarker/KEYS

  9. SVN commit the release files into the proper subdirectory of https://dist.apache.org/repos/dist/dev/freemarker. Note that this is "dev/", not "release/"! You will likely find the "dev" directory empty, so see the proper layout under the "release" directory as a sample, but don't commit to there accidentally.

  10. Upload to ASF Nexus staging

    1. If your Maven settings-security.xml is relocated to an encrypted storage (recommended), mount it, otherwise you will get 401 error when uploading. (How that's done: create a file called settings-security.xml next to ~/.m2/settings.xml, with content <settingsSecurity><relocation>your-decrypted-storage-mount/settings-security.xml</relocation></settingsSecurity>)

    2. ant maven-dist

    3. Go to https://repository.apache.org/, log in, chose "Staging Repositories", find and click the repository just created. Check the upload files manually (on the "Content" tab). Then "Close" the repository. Press "Refresh" until it's successfully "closed". (Be careful, do NOT "Release"!)

  11. Upload the content inside documentation/_html from the binary release to https://freemarker.apache.org/builds/X.X.X-voting/ (by committing it into the "asf-site" branch of https://git-wip-us.apache.org/repos/asf/freemarker-site.git). The change log will be a link pointing to a page inside this.

  12. Voting on the FreeMarker developer list

    1. Start a thread on the FreeMarker developer mailing list with subject "[VOTE] Release Apache FreeMarker X.X.X". You should use an earlier release vote mail as a template. Some notes:

      • Use your apache.org e-mail address as the sender

      • Link artifacts. The src artifact is the release that's voted upon. The bin is for convenience only, though of course still need to be checked.

      • Include and link via the Git commit hash (not a tag)

      • Link the change log (page under https://freemarker.apache.org/builds/X.X.X-voting/)

      • Specify Maven staging repository URL and artifact coordinates, also the URL of the repository directory that contains this release for easier reviewing

    2. Needs at least 3 "+1" votes from FreeMarker PMC members, majority win, and minimum 72 hours (but see Apache policies). Generally, we wait more, so that more people can test there release.

    3. Post "[RESULT][VOTE]" mail to the FreeMarker developer list. Again, you should use an earlier release vote result mail as a template.

    4. If it was a failed vote, after fixing the issues, the release process restarts with a different dist.apache.org URL-s (add an attempt number to the directory for example)

  13. Copy the release files over from https://dist.apache.org/repos/dist/dev/freemarker into the proper subdirectory of https://dist.apache.org/repos/dist/release/freemarker and commit them. (Of course get rid of any voting attempt numbers in the directory names.) After a few minutes of delay, they should appear under https://www.apache.org/dist/freemarker/.

  14. Go to https://repository.apache.org/ and find the staging repository that belongs to the voted version. Select it, press "Release" at the top. Now the repository should appear under https://repository.apache.org/content/repositories/releases/org/freemarker/ Later (like hours later) it will appear in the Maven Central Repository too.

  15. Tag the release on Git (tags are like v2.3.45-gae and v2.3.45). Be careful not to tag a commit made since the voting has succeeded.

  16. Add the release on: https://reporter.apache.org/addrelease.html?freemarker

  17. Update doap.rdf in the freemarker-site repository, by adding a new release element. Don't forget to commit it into the asf-site branch as well, not just into master, or else it's not published.

  18. Wait 24 hours so that the mirrors get synced. Check https://www.apache.org/dist/freemarker/ and http://archive.apache.org/dist/freemarker/. Check Maven Central repository too (in the case of non-RC releases).

  19. Delete the non-latest release of each maintained branch from https://www.apache.org/dist/freemarker/ (Old ones are automatically available on http://archive.apache.org/dist/)

  20. Update https://freemarker.apache.org:

    • Upload latest JavaDoc

    • Update release date in the Manual in the Version history (now that we know it), then generate the Manual with ant manualOnline.

    • Upload the new Manual (the online version you have generated in the last step, not the offline one)

    • Update to latest release in the download section

  21. Update other web pages:

  22. Announce:

  23. Close issues on Jira that were resolved by this release (the Version history in the Manual should contain links to them)

  24. Update try.freemarker.apache.org to use the latest FreeMarker version. About the deployment, see more here...

  25. If the release has introduced new syntax or built-ins or directives, the FreeMarker dependency of JBoss Tools FreeMarker IDE must be updated. As JBoss has stopped maintaining that plugin, currently we are maintaining a https://github.com/jbosstools/jbosstools-freemarker fork for this.

Deploying Maven SNAPSHOT versions

After you have merged/committed your changes to the respectable branch, you may want to publish that version to https://repository.apache.org/content/repositories/snapshots. Users or fellow developers can use this Maven repository to try snapshot versions.

To publish the snapshot version, fist check that version.properties contains the proper version, most importantly, that mavenVersion ends with -SNAPSHOT. (If not, it will deploy to the release staging repository.) Then issue ant dist maven-dist. (Note that just as with a real release, the artifacts will be signed with PGP.)

Updating the FreeMarker home page

The content of the home page can be changed by committing it into the asf-site branch of the freemarker-site repository. However, you shouldn't directly edit the content of most HTML files there, as pretty much all of them are generated, and then is just manually copied over there. The generated pieces are:

  • The contents of the docs/api is generated from the freemarker project, by issuing ant javadoc there (not in the site project). So to change the content, you should edit the documentation comments in the java files.

  • The contents of docs/ is generated from the freemarker project, by issuing ant manualOnline. So to change the content, you should edit <freemarker-project>/src/manual/en_US/book.xml.

  • Most other content is generated from the site project, by issuing mvn -U package. So to change the content, you should edit <site-project>/src/main/docgen/book.xml (in the master branch, not in the asf-site branch).

After you commit into the asf-site branch of the freemarker-site repository, it will automatically appear on the web site, with some small delay (a few seconds usually). If it doesn't, that's probably because of a known infrastructural glitch, where for too big commits this synchronization doesn't happen. In such case, commit some small change to trigger synchronization again.

Updating Docgen

Docgen (freemarker-docgen Git repository) generates HTML from a DocBook XML. It's a dependency of FreeMarker binary release building (as that contains the Manual in HTML format) and of freemarker-site.

Docgen is not published to the Maven Central Repository (yet?), only into the Apache Maven Snapshot Repository. That works for us because it's an internal build dependency, and the dependent projects are set up to find it in the Apache Maven Snapshot Repository. If you modify Docgen, just issue mvn deploy to make the new version available for the later dependent builds.

Regarding editing the Manual and the Site DocBook

Both the site and the Manual is generated from the XML files (DocBook format) by the custom Ant task defined in the docgen project (freemarker-docgen repository). That project also provides an XMLMind XML Editor (XXE for short) addon for editing these files. For more guide lines see src/manual/en_US/docgen-help/editors-readme.txt in the freemarker repository. About the same guide lines apply to the site DocBook as well.

Dealing with security vulnerabilities

If someone reports a security vulnerability, normally they shouldn't do it on a public forum (see how to report it here), and similarly we shouldn't discuss it on a public forum (such as on the developer mailing list), but on the private mailing list of the project. Thus the vulnerability can be fixed and released before it's openly discussed. As a developer, you must not forget that commits are also publicly visible. How to commit, release, and communicate a concrete vulnerability should be discussed on the private mailing lists of the project before doing publicly visible moves. See this page for further guidelines.

PMC member resources

This section is for PMC (Project Management Committee) members only, and meant to contain information that is specific to the FreeMarker project.

To share files among PMC members (such as Board report drafts), use this SVN repository: https://svn.apache.org/repos/private/pmc/freemarker/. It can be read by PMC members only.