Skip to content

Reuse the existing signed package when the unsigned content is unchanged - #829

Open
favoyang wants to merge 1 commit into
bdovaz:masterfrom
favoyang:reuse-unchanged-signatures
Open

favoyang wants to merge 1 commit into
bdovaz:masterfrom
favoyang:reuse-unchanged-signatures

Conversation

@favoyang

Copy link
Copy Markdown
Contributor

Problem

Every build of a package runs upm pack, and signing is not deterministic: the attestation embeds an RFC 3161 timestamp, so the same content signed twice produces different bytes. The signed tarball, its SHA-1 and the dist.shasum in the registry metadata therefore change on every rebuild, even when the package content is identical.

This happens on every image update, because a changed registry version clears the cache and regenerates all packages. Downstream mirrors and clients then see a checksum change for packages that did not change, and anything caching by checksum has to re-download everything.

Change

  • The unsigned tarball is written to <file>.tgz.unsigned, hashed with SHA-256, and moved into place only when it has to be signed.
  • The SHA-256 of the unsigned tarball is recorded in <file>.tgz.unsigned.sha256 after signing.
  • On a rebuild, when the recorded digest matches and the signed tarball and .sha1 are present, the existing signed package is kept and the signer is not called.
  • forceUpdate no longer deletes signed packages before the conversion, so the reuse path can see them; the conversion still runs for every version.
  • The recorded digest is deleted together with the package.

Packages without a recorded digest (all existing ones) are signed once, then start reusing.

Validation

  • dotnet build clean, no warnings.
  • dotnet test src/UnityNuGet.Tests — 504 passed, 22 skipped, 1 failure: CanParse_PackageWithRuntimes, which also fails on the unmodified branch here (it downloads a package from nuget.org), so it is unrelated.
  • End-to-end with a stub upm executable that behaves like the real signer (new bytes on every run): two consecutive forced rebuilds produced byte-identical artifacts with this change, and different artifacts without it.

Notes

  • Only active when signing is configured.
  • If a future version changes the conversion output, the digest differs and the package is signed as today.

Signing is not deterministic: the Unity attestation carries an RFC 3161
timestamp, so signing identical content produces a different tarball, a
different SHA-1 and a different published checksum. A full rebuild (image
update) therefore rewrites the checksum of every package, which invalidates
downstream mirrors and caches for no reason.

Record the SHA-256 of the unsigned tarball next to each signed package and
reuse the signed artifact when a rebuild produces the same unsigned bytes.
The conversion now writes the unsigned tarball to a sibling file, hashes it,
and only calls the signer when the digest differs from the recorded one.

forceUpdate no longer deletes signed packages up front, so the reuse path can
see the previous artifact; the conversion still runs for every version. The
recorded digest is removed together with the package, and a failed conversion
keeps the previous signed artifact when it was reused.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant