SUMMARY
A potential data loss issue has been discovered when using incorrectly configured policies against S3 destinations.
SEVERITY
This issue has been classified as critical. Even though many users will not be affected at this time, product misconfiguration results in data loss.
VERSIONS AFFECTED
12.12 to 12.12u3 inclusive
DETAIL
This issue pertains only to:
- Change Destination Tier and Retarget Destination Policies
- Using S3 as the Policies’ New Destination
- Where the New Destination bucket is configured to use the Migrate with original filenames option
Normally, the configuration described above is not permitted - Change Destination Tier and Retarget Destination Policies are immediately terminated with an appropriate error when attempted against such buckets. This is advised in the GUI when switching the bucket layout configuration to the (non-default) Migrate with original filenames option: such buckets “may not be used as the new destination for Change Destination Tier or Retarget Destination Policies.”
If a Change Destination Tier or Retarget Destination Policy is incorrectly configured to move data to a Migrate with original filenames bucket, in some cases the logic to terminate the policy will fail. Instead, the policy will process these files incorrectly, leading to undesirable behaviour including data loss.
DETECTION
Check in agent.log on all S3 Gateway Servers for a message similar to the following:
2021-11-23 14:15:41.237 (GMT+10:00) <Q04> ERROR: uncombined error:
[0] ERR_DMSTREAMAMAZONS3_CREATEALWAYSOVERWRITE_FAILED [ERR_ADD_OPERATION_NOT_PERMITTED_FOR_DEST_CONFIG] [Bucket 'my-bucket' is configured to 'Migrate with original filenames']
The presence of this error (including the ‘uncombined error’ line) indicates the incorrect processing of a file. Contact Moonwalk immediately should you encounter this issue.
FIX
This issue will no longer occur once the software is updated to 12.12u4 or later.