feat(s3): support SigV4 header signing for v4 requests#150
Open
mgajek-cern wants to merge 1 commit into
Open
Conversation
mgajek-cern
force-pushed
the
feat/149-s3-sigv4-header-signing
branch
2 times, most recently
from
79678dc to
c513c92
Compare
Implement the v4 branch of signRequest() to produce header-signed S3 requests (Authorization, x-amz-date, x-amz-content-sha256) as an alternative to the existing presigned-URL output of signURIv4(). Add a RequestParams setting setAwsSigV4HeaderMode() to opt requests into header signing. Default is false, preserving existing behavior: v4 requests continue to use presigned URLs unless explicitly opted in. v2 requests are unaffected and continue to use header signing as before. Route BackendRequest::configureS3params() through the new path when the setting is enabled. The TPC code path in src/modules/copy/copy.cpp is intentionally left unchanged — destinations expect URL-bearer credentials. Canonical headers are de-duplicated by key, and the signer respects a caller-supplied x-amz-date so signing can be verified deterministically against AWS's published SigV4 test vector (GET Object example). Streaming/multipart payload signing (STREAMING-AWS4-HMAC-SHA256-PAYLOAD) is out of scope; PUT/POST use UNSIGNED-PAYLOAD. getAwsSignaturev4() now takes the signing date as a parameter instead of reading the current date internally, so the signing key is scoped to the request's x-amz-date rather than wall-clock time. signURIv4() passes its existing date and is behaviourally unchanged; this also removes a latent key/scope mismatch when signing would straddle midnight. This enables interop with S3-compatible endpoints that reject URL-embedded signatures (e.g. Copernicus Data Space, behind a gateway that only accepts the Authorization header form that boto3 uses by default). Closes: cern-fts#149 Refs: - https://docs.aws.amazon.com/AmazonS3/latest/developerguide/sigv4-auth-using-authorization-header.html - https://docs.aws.amazon.com/AmazonS3/latest/developerguide/sig-v4-header-based-auth.html - http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
mgajek-cern
force-pushed
the
feat/149-s3-sigv4-header-signing
branch
from
c513c92 to
50ad3c4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes: #149
Reviewer/s: @joaopblopes, @andrea-manzi
Following the discussion with Andrea and Joao regarding the addition of AWS SigV4 header signing for RI-SCALE (Copernicus Data Space interop), either would be suitable to review.