Page Comparison

...

• When the multipart upload to S3 encounters an error (i.e. incorrect checksum of a chunk), the record will be Rejectedwith an appropriate error reason

Storage

The S3 storage is treated as one of /wiki/spaces/CS/pages/20643843 in the system to re-use many existing features

• A new cluster group “resumable_uploads” with the role TRANSIENT is created, highly similar to the existing cluster group “ingest”, which contains 1 shared storage pool for the S3 object store

• The uploading record will be linked with this shared storage pool and the session ID will also be stored as the distribution ID

• The file is stored on the S3 object store using the standardized naming convention <Record ID>/<Record ID>.<Original Extension>

• The garbage collection will clean up this object store for this record when

  • The When the ingest workflow has transferred the record to the definitive storage

  • The When the record is permanently deleted (manually or automatically after being inactive for 2 weeks or longer)

  The used capacity will tracked by the standard “Storage-free space” workflow

  • When the upload is still in progress it can use the session ID to delete it