A potential data loss issue has been discovered when using incorrectly configured policies against S3 destinations.
This issue has been classified as critical. Even though many users will not be affected at this time, product misconfiguration results in data loss.
12.12 to 12.12u3 inclusive
This issue pertains only to:
- Change Destination Tier and Retarget Destination Policies
- Using S3 as the Policies’ New Destination
- Where the New Destination bucket is configured to use the Migrate with original filenames option
Normally, the configuration described above is not permitted - Change Destination Tier and Retarget Destination Policies are immediately terminated with an appropriate error when attempted against such buckets. This is advised in the GUI when switching the bucket layout configuration to the (non-default) Migrate with original filenames option: such buckets “may not be used as the new destination for Change Destination Tier or Retarget Destination Policies.”
If a Change Destination Tier or Retarget Destination Policy is incorrectly configured to move data to a Migrate with original filenames bucket, in some cases the logic to terminate the policy will fail. Instead, the policy will process these files incorrectly, leading to undesirable behaviour including data loss.
Check in agent.log on all S3 Gateway Servers for a message similar to the following:
2021-11-23 14:15:41.237 (GMT+10:00) <Q04> ERROR: uncombined error:  ERR_DMSTREAMAMAZONS3_CREATEALWAYSOVERWRITE_FAILED [ERR_ADD_OPERATION_NOT_PERMITTED_FOR_DEST_CONFIG] [Bucket 'my-bucket' is configured to 'Migrate with original filenames']
The presence of this error (including the ‘uncombined error’ line) indicates the incorrect processing of a file. Contact Moonwalk immediately should you encounter this issue.
This issue will no longer occur once the software is updated to 12.12u4 or later.