Skip to content

Sending an image hangs forever if reading it back from storage fails #42514

Open

Description

@ndo84bw

Description:

When the server can't read an uploaded image back from storage while creating the thumbnail, rooms.mediaConfirm never responds. The composer stays greyed out until our reverse proxy gives up after 120 s, and the message is lost.

The culprit is image.pipe(transformer) in resizeImagePreview and createImageThumbnail (FileUpload.ts). pipe() doesn't pass source errors on, so toBuffer() never settles.

We hit this with Garage as S3 storage. Since 8.3.0, every image over 5 MB fails the AWS SDK checksum check there (Garage #1228). But any read error will cause the hang.

Steps to reproduce:

  1. Use Garage (dxflrs/garage:v2.1.0) as S3 storage
  2. Send an image over 5 MB

Expected behavior:

The image is sent without a thumbnail. sendFileMessage already does that in its catch, the error just never gets there.

Actual behavior:

The request hangs until the proxy times out, and the message is lost.

Image

Server Setup Information:

  • Version of Rocket.Chat Server: 8.6.1 (same code on develop)
  • License Type:
  • Number of Users:
  • Operating System:
  • Deployment Method: docker
  • Number of Running Instances:
  • DB Replicaset Oplog:
  • NodeJS Version: 22.22.1
  • MongoDB Version:

Client Setup Information

  • Desktop App or Browser Version: any
  • Operating System: any

Additional context

This fix worked for me on 8.6.1. The image is sent without a thumbnail, and the error is logged normally:

const failed = new Promise<never>((_, reject) => stream.pipeline(image, transformer, (err) => err && reject(err)));
return Promise.race([result, failed]);

pipeline() alone isn't enough, toBuffer() still never settles.

Workaround: AWS_RESPONSE_CHECKSUM_VALIDATION=when_required.

Relevant logs:

=== UnCaughtException ===
Error: Checksum mismatch: expected "guNN8A==" but received "vt0U6w==" in response header "x-amz-checksum-crc32".
    at ChecksumStream._final (@smithy/util-stream/dist-cjs/checksum/ChecksumStream.js:53:33)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions