release scripts: support reproducible builds via SOURCE_DATE_EPOCH

[PtJade-Ceramic]

Add reproducible-build support: when SOURCE_DATE_EPOCH is set, the release scripts pin file/directory mtimes and the 7z/ZIP paths use -mta- -mtc- so the installer, portable, MinGit and tar archives are byte-identical across repeated builds on the same SDK snapshot. This is a no-op when SOURCE_DATE_EPOCH is unset.

[dscho]

I fear that this is a lot of code (which would need to be simplified before it could be merged, anyway) for an incomplete solution.

The real problem is of course that it depends on much more than just the build-extra repository which bits and parts are combined into a full Git from Windows release. Specifically a full Git for Windows release is usually done starting with a specific commit in the git-sdk-* repositories, and then building Git and then installing it into that checkout. So there's really no single commit from which this is built. And therefore it would really be a challenge to reproduce any release, and setting the epoch here is just maybe one percent of the entire effort that would be required.

I don't think reproducible builds are feasible in Git from Windows.

[PtJade-Ceramic]

I fear that this is a lot of code (which would need to be simplified before it could be merged, anyway) for an incomplete solution.

The real problem is of course that it depends on much more than just the build-extra repository which bits and parts are combined into a full Git from Windows release. Specifically a full Git for Windows release is usually done starting with a specific commit in the git-sdk-* repositories, and then building Git and then installing it into that checkout. So there's really no single commit from which this is built. And therefore it would really be a challenge to reproduce any release, and setting the epoch here is just maybe one percent of the entire effort that would be required.

I don't think reproducible builds are feasible in Git from Windows.

Thanks for the review -- both points are fair.

I agree that this cannot, on its own, make a full Git for Windows release reproducible: a release is assembled from a specific git-sdk-* snapshot plus a freshly built Git, and there is no single commit that pins all of that down. Setting SOURCE_DATE_EPOCH in the packaging scripts is indeed only a small slice of the effort.

The scope I am aiming for is deliberately narrower: given one and the same SDK snapshot, the packaging step itself should be deterministic. Building the installer, portable, MinGit or tar archive twice on that snapshot should yield byte-identical output instead of silently embedding the build time. That property is verifiable on its own and useful for debugging -- when artifact bytes differ, it is no longer an unactionable mystery -- but it makes no claim about reproducing a release from scratch.

On the code-size concern: the per-script ~37-line blocks are now a single shared helper (pin-mtimes.sh, ~70 lines including comments), the duplicated hardcoded source list in installer/release.sh is gone, and the parent-directory walk is deduplicated. While reworking this I also found that the installer path had never actually pinned anything: the Perl one-liner's print (x), "\n" form drops the trailing newline, so the whole file list collapsed into one line and every lookup failed. With that fixed, the installer reproducibility claim is now actually exercised, not just the archive paths.

[dscho]

I am afraid that I still do not follow how this would be useful for the Git for Windows project. In the official release process, the proposed premise "building from the same snapshot" simply does not exist.

I understand that you want this. Yet I do not understand why the Git for Windows project should accept a complex (and, frankly a bit fragile, given that building the Git executables and code-signing them with a current timestamp is a crucial part of the release process) and substantial amount of code (the added pin-mtimes.sh is almost half the size of mingit/release.sh, which itself is not exactly light on -- necessary! -- complexity), when Git for Windows itself won't benefit from it. It sounds like a net negative on the project to me: carry more, and complex, and fragile code, for reasons that lie outside the project and its direct users.

In other words, from where I sit the most plausible course of action would be to keep this code in your fork, and leave it out of this here repository.

Also, I wonder whether you have considered extracting the mtimes from an existing installer/archive you wish to reproduce and forcing the respective files' mtimes in your git-sdk-* checkout to those values? That should achieve the same reproducibility goal without burdening build-extra with code that does not benefit the Git for Windows project.

[PtJade-Ceramic]

I am afraid that I still do not follow how this would be useful for the Git for Windows project. In the official release process, the proposed premise "building from the same snapshot" simply does not exist.

I understand that you want this. Yet I do not understand why the Git for Windows project should accept a complex (and, frankly a bit fragile, given that building the Git executables and code-signing them with a current timestamp is a crucial part of the release process) and substantial amount of code (the added pin-mtimes.sh is almost half the size of mingit/release.sh, which itself is not exactly light on -- necessary! -- complexity), when Git for Windows itself won't benefit from it. It sounds like a net negative on the project to me: carry more, and complex, and fragile code, for reasons that lie outside the project and its direct users.

In other words, from where I sit the most plausible course of action would be to keep this code in your fork, and leave it out of this here repository.

Also, I wonder whether you have considered extracting the mtimes from an existing installer/archive you wish to reproduce and forcing the respective files' mtimes in your git-sdk-* checkout to those values? That should achieve the same reproducibility goal without burdening build-extra with code that does not benefit the Git for Windows project.

Thanks for the detailed review and for laying out the reasoning so clearly. You are right on both counts: the official release flow does not start from a fixed snapshot, and pinning mtimes in the packaging scripts is only a small slice of what full reproducibility would require. Since this would add complexity and maintenance burden to build-extra without benefiting the Git for Windows project, I will not pursue merging it here.

Your suggestion of extracting the mtimes from an existing installer/archive and forcing the corresponding files' mtimes in the git-sdk-* checkout is a good one — it can reach the same goal without touching build-extra at all.

Thanks again for the feedback and for the alternative direction.

---

I tried the approach you suggested — extracting the mtimes from an existing artifact and forcing the matching files' mtimes in a git-sdk-* checkout -- and it does not reach the same result, so I went back to the SOURCE_DATE_EPOCH version.

The packaging scripts do not merely repack existing files: on every run, mingit/release.sh and archive/release.sh regenerate some members at build time (LICENSE.txt, the system gitconfig with the [include] sections, etc/package-versions.txt, the bin/git.exe/bash.exe/sh.exe redirectors and the dev/* symlinks). Those members are written after any external backfill could run and they do not live under the SDK root (they sit in the build-extra overlay), so an external tool cannot touch them; two plain builds therefore still differ in the MinGit .zip and in the tar.bz2 archive.

Only pinning mtimes in-script, at pack time after those members are generated, covers them -- which is what the SOURCE_DATE_EPOCH code in this PR does. With it, building the installer, portable, MinGit and the tar archive twice on one and the same SDK snapshot is byte-identical (verified green in CI on my fork); backfilling the mtimes of the merely-repacked files makes MinGit and the archive match too, so those regenerated members are exactly the residual difference.

Your broader points stand and I am not pursuing a merge here: the official release flow does not start from one fixed snapshot, code signing embeds a current timestamp, and this only pins the packaging slice. I am keeping it in my fork as a debugging aid for the narrow "same snapshot, build twice" property.