Known Limitations

This page presents the technical and functional limitations of DataChain Core as well as the best practices to implement to benefit from the best performance.

Limitation on using multiple S3 buckets with the same name

Each DataChain Core instance can only register one S3 bucket for a given name, regardless of the configured S3 endpoint.

If you attempt to configure multiple connectors to distinct S3 buckets but with the same name, only the first registered bucket will be taken into account. Subsequent exports to a bucket with the same name will systematically be redirected to this first bucket, even if the endpoint or region are different.

Example

If you configure these two connectors:

  1. Bucket "my-bucket" with the endpoint s3.eu-west-1.amazonaws.com

  2. Bucket "my-bucket" with the endpoint s3.us-east-1.amazonaws.com

Then all exports specified to "my-bucket" will be sent to the S3 bucket in the eu-west-1 region, because it was registered first.

Recommendation

To avoid any ambiguity, make sure to use unique bucket names for each S3 connector configured in your DataChain Core instance. This will ensure that exports are sent to the expected S3 bucket each time.

This limitation applies per instance. If you have multiple DataChain Core instances, you can reuse the same bucket names, but each instance will only consider the first registered bucket for each name.