Release Notes

This page provides technical information about DataChain versions, migration procedures, and specific steps required for certain version upgrades. For functional evolutions, please refer to the functional release notes.

Version 9.1.1 → 9.1.7

No additional technical information.

Version 9.1.0

Major changes to docker images

Keycloak decoupling

The keycloak decoupling, already started in 9.0.0, continues. It should be definitively completed with the migration of DcCode and Maestro into the console, thus offering a more integrated functional environment and a technical environment allowing the Datachain suite to be integrated with a compatible OAuth SSO.

As is already the case for most of our customers using a swarm deployment, we recommend putting keycloak in a dedicated docker stack.

Future keycloak versions will be aligned with the current major keycloak version, and we will continue to keep it up to date on our side to ensure the initial configuration (in particular the creation of the various clients used by Datachain). For this 9.1.0, please keep your keycloak version, the latest version available being 9.0.18.

Major PostgreSQL update

For this new version of Datachain core, we have updated the PostgreSQL version, which is now PostgreSQL 18.3. As with keycloak, the postgres version will now be aligned with the major postgres version, for better visibility of the version in use.

The new PostgreSQL version is 18-trixie.

Migrating to this new version is recommended, but not mandatory to update Datachain. If you do not wish to migrate the data when deploying this version, the update can be done at a later date. In this case, please keep the PostgreSQL version you are currently using, the last version before 18-trixie being 9.0.18.

The migration procedure consists of:

Stopping the stacks/services using the database
Dumping the databases
Backing up the database data directory (as a second backup) or creating a new volume for the new database version
Starting the new database
Restoring the data from the old database into the new database.

Note: For DC-Maestro users, it is also possible to migrate the maestro and airflow databases to this new database, in order to keep only a single deployed version of PostgreSQL.

For the postgres image change, take this new name into account

    image: b9kd47jn.gra7.container-registry.ovh.net/dc/pg_all:9.0.xxx

    image: b9kd47jn.gra7.container-registry.ovh.net/dc/dc_pg:18-trixie

Replacing the Redis service with a Valkey service

Redis is a cache and queue management solution used for many purposes in the Datachain suite. The latest version used is version 7. We have now chosen to use the Valkey solution, which is the same type of solution, supported by the Linux Foundation. We recommend migrating to this new product for Datachain core. We are currently validating it for Dc-Maestro, which currently uses a separate Redis service. Migrating to this new version is very simple. Edit the docker-compose file associated with Redis and change the lines:

    image: redis:7.0-alpine
    command: redis-server #... with potentially some parameters in your deployment

    image: valkey/valkey:9.0-alpine
    command: valkey-server #... with potentially the same parameters in your deployment

Traefik version update

We have updated the Traefik version, the version used in most of our customer deployments in Swarm (excluding K8s) being 2.x. We have validated the use of version 3.6. This version requires changes to the deployment descriptors of the services linked to Traefik, as induced by the product.

To migrate to this new version, the traefik configuration file must be updated: Change the lines

      - "--providers.docker=true"
      - "--providers.docker.swarmMode=true"
      - "--providers.docker.network=traefik_network"
      - "--providers.docker.exposedbydefault=false"
      - "--providers.docker.endpoint=unix:///var/run/docker.sock"

      - "--providers.swarm.watch=true"
      - "--providers.swarm.exposedbydefault=false"
      - "--providers.swarm.network=traefik_network"
      - "--providers.swarm.endpoint=unix:///var/run/docker.sock"

In all component deployment files using traefik, change the following line or remove it (see below)

      - "traefik.docker.network=traefik_network"

    - "traefik.swarm.network=traefik_network"

Note: this line can be removed, because in that case traefik uses the default network, through the "providers.swarm.network" configuration parameter present in the traefik service declaration, to communicate with the other services.

Database driver updates

The PostgreSQL driver goes from version 42.7.7 to version 42.7.8. The other drivers are unchanged.

Database type	Version	Driver compatibility matrix	Comments
PostgreSQL	42.7.8	https://jdbc.postgresql.org/download/
SQLServer	12.6.4	https://learn.microsoft.com/en-us/sql/connect/jdbc/microsoft-jdbc-driver-for-sql-server-support-matrix?view=sql-server-ver17#sql-version-compatibility
Oracle	19.22.0.0	https://www.oracle.com/uk/database/technologies/faq-jdbc.html (question: What is the JDBC and RDBMS interoperability matrix or the certification matrix?)	With internationalization support (oraI18n)
Mysql	9.4.0	https://dev.mysql.com/doc/connector-j/en/connector-j-versions.html
Mariadb	3.5.5	https://mariadb.com/docs/connectors/mariadb-connector-j/about-mariadb-connector-j

Database type

Version

Driver compatibility matrix

Comments

PostgreSQL

42.7.8

https://jdbc.postgresql.org/download/

SQLServer

12.6.4

https://learn.microsoft.com/en-us/sql/connect/jdbc/microsoft-jdbc-driver-for-sql-server-support-matrix?view=sql-server-ver17#sql-version-compatibility

Oracle

19.22.0.0

https://www.oracle.com/uk/database/technologies/faq-jdbc.html (question: What is the JDBC and RDBMS interoperability matrix or the certification matrix?)

With internationalization support (oraI18n)

Mysql

9.4.0

https://dev.mysql.com/doc/connector-j/en/connector-j-versions.html

Mariadb

3.5.5

https://mariadb.com/docs/connectors/mariadb-connector-j/about-mariadb-connector-j

Version 9.0.3 → 9.0.18

Some technical database migrations are performed when installing the release compared to the previous version. Please make sure the patches ran correctly after deploying the new version by checking the migration service logs.

Versions 9.0.1/9.0.2

Configuration of the console module

Added new settings for the console for LDAP synchronization:

dc.app.console.provisioning.ldap.first.ldap-type and dc.app.console.provisioning.ldap.second.ldap-type allowing to indicate the directory type:
- AD: Active directory
- STANDARD : Other LDAP directories (default)
dc.app.console.provisioning.ldap.first.count and dc.app.console.provisioning.ldap.second.count to enable pagination and specify the number of entries to be escalated (ldap pagination). Specify a value other than the default (-1) to enable paging.
dc.app.console.provisioning.ldap.first.referral and dc.app.console.provisioning.ldap.second.referral of possible values follow, ignore, throw for the explicit processing of LDAP referrals (notably in case of a synchronization error with active directory. See here.)

The value of the parameters dc.app.console.provisioning.ldap.first.user.relative-dn, dc.app.console.provisioning.ldap.second.user.relative-dn, dc.app.console.provisioning.ldap.first.group.relative-dn, dc.app.console.provisioning.ldap.second.group.relative-dn can now be set to an empty value.

Version 9.0.1

No additional technical information.

Version 9.0.0

The V9 version of datachain introduces a large number of features as well as many technical changes.

Console module

The console module is now deployed with datachain Core. This module manages users/groups and grants them access to the various Datachain Core and Datachain Marketplace modules. It will be used to assign the user roles of dc-maestro and dc-marketplace in a future version. As a result, on platforms where maestro is deployed, the keycloak module remains deployed, to allow communication between the maestro, keycloak and datachain-core modules.

The console module notably limits the tight coupling between Datachain-Core and the keycloak SSO and should make integration easier in information systems already equipped with an OAuth-type SSO. To do so, the console module can be provisioned in particular from an LDAP directory (generally used to integrate users into the SSO), thus avoiding unnecessary re-entry of user information.

Migration steps

1. Checking/Creating the keycloak clients

In the dc-realm security realm, check that the clients dc_core, dc_marketplace and dc_console exist. If the clients do not exist, create them by enabling "client authentication" and "standard flow", and retrieve the access_tokens, to be reported in the following configurations. The three creation steps are:

step 1

step 2

step 3

Retrieving the access_tokens

2. Adapting the Postgres image used for dc-core

In the database deployment file (docker-compose.yml): check that the pg_all image is used instead of the pg image. Replace it if necessary.

3. Checking that the databases exist (dc_console and dc_marketplace)

Depending on the docker image and the deployment history of the solution, it may be necessary to create the databases.

→ To check, open a shell on the Datachain database:

psql -U postgres
\l

If the dc_console and dc_marketplace databases do not appear, create them.

CREATE DATABASE dc_console WITH OWNER postgres ENCODING='UTF8' TEMPLATE=template0 TABLESPACE  pg_default CONNECTION LIMIT  -1;
CREATE DATABASE dc_marketplace WITH OWNER postgres ENCODING='UTF8' TEMPLATE=template0 TABLESPACE  pg_default CONNECTION LIMIT  -1;

Quit the database.

\q

4. Reviewing the dc-core deployment descriptors

4.1 Preparing the console deployment

A file describing the platform elements is provided to the console, in order to transfer the information to the various modules and allow inter-module communication. In this version, only the Console, marketplace and datachain core modules are concerned. The other Datachain modules will be integrated in a later version.

Prepare the topology description file by adapting the public and internal URLs of the elements to be deployed, as well as the codes, labels and descriptions. The associated codes must be reported in the configuration of the modules concerned.

If the marketplace is used:

{
  "products": [
    {
      "code": "DATACHAIN_CONSOLE",
      "label": "Datachain Console",
      "description": "Datachain Console",
      "internalUrl": "http://dc-console:8093",
      "publicUrl": "_____PUBLIC_URL_TO_CONFIGURE______",
      "type": "DC_CONSOLE"
    },
    {
      "code": "DC_MP",
      "label": "Datachain Marketplace for Efficient Sharing",
      "description": "A nice description of the datachain core platform",
      "internalUrl": "http://dc-marketplace:8092",
      "publicUrl": "_____PUBLIC_URL_TO_CONFIGURE______",
      "type": "DC_MARKETPLACE"
    },
    {
      "code": "DATACHAIN_CORE",
      "label": "Datachain Core Platform",
      "description": "Datachain Generics Data & Handle Data",
      "internalUrl": "http://dc-backend:8089",
      "publicUrl": "_____PUBLIC_URL_TO_CONFIGURE______",
      "linkedProducts": [
        "DC_MP"
      ],
      "type": "DC_CORE"
    }
  ]
}

If the marketplace is not used:

{
  "products": [
    {
      "code": "DATACHAIN_CONSOLE",
      "label": "Datachain Console",
      "description": "Datachain Console",
      "internalUrl": "http://dc-console:8093",
      "publicUrl": "_____PUBLIC_URL_TO_CONFIGURE______",
      "type": "DC_CONSOLE"
    },
    {
      "code": "DATACHAIN_CORE",
      "label": "Datachain Core Platform",
      "description": "Datachain Generics Data & Handle Data",
      "internalUrl": "http://dc-backend:8089",
      "publicUrl": "_____PUBLIC_URL_TO_CONFIGURE______",
      "type": "DC_CORE"
    }
  ]
}

4.2 Adding the console to the deployment descriptors

services:
  dc-console:
    # REGISTRY URL TO ADAPT and product version to adapt
    image: ${HARBOR_REGISTRY}/dc/console:${PRODUCT_VERSION}
    networks:
      traefik_network:
      dc_network:
        aliases:
          - dc-console
    volumes:
      - type: bind
        # Log directory to adapt
        source: ${DC_CFG_DC_LOG_DIR_DATACHAIN}
        target: /logs
      - type: bind
        source: ./config
        target: /config
    env_file: console.env
    deploy:
      mode: replicated
      replicas: 1
      labels:
        - "traefik.enable=true"
        - "traefik.docker.network=traefik_network"
        - "traefik.http.routers.dc_console.rule=Host(`___PUBLIC_URL_OF_THE_CONSOLE___`) && PathPrefix(`/`)"
        # To adapt depending on your deployment, if secure
        #- "traefik.http.routers.dc_console.entrypoints=web"
        #- "traefik.http.routers.dc_console.entrypoints=websecure"
        #- "traefik.http.routers.dc_console.tls=true"
        #- "traefik.http.routers.dc_console.tls.certresolver=resolver1"
        - "traefik.http.routers.dc_console.service=dc_console"
        - "traefik.http.services.dc_console.loadbalancer.server.port=8093"
        - "traefik.http.routers.dc_console1.middlewares=dc_console_comp1@docker"
        - "traefik.http.middlewares.dc_console_comp1.compress=true"

networks:
  dc_network:
    external: true
    name: dc_network
  traefik_network:
    external: true
    name: traefik_network

Base configuration file

# default log configuration, which can be modified if needed
logging.level.ROOT=INFO
logging.level.com.adb=INFO
logging.level.org.springframework=INFO
logging.level.org.springframework.security=INFO
logging.level.org.springframework.data.redis=ERROR
#logging.level.org.hibernate.SQL=DEBUG
#logging.level.org.hibernate.orm.jdbc.bind=TRACE
#logging.level.org.apache.http=DEBUG
#logging.level.org.apache.http.wire=DEBUG

dc.app.console.database.main-datasource.password=___TO_BE_SPECIFIED___
dc.app.console.database.main-datasource.url=jdbc:postgresql://dc-pg:5432/dc_console

spring.security.oauth2.resourceserver.jwt.jwk-set-uri=___SSO_URL___/realms/dc-realm/protocol/openid-connect/certs
spring.security.oauth2.client.provider.oidc.issuer-uri=___SSO_URL___/realms/dc-realm
spring.security.oauth2.client.registration.oidc.client-id=dc_console
spring.security.oauth2.client.registration.oidc.client-secret=___PREVIOUSLY_GENERATED_PASSWORD___
spring.security.oauth2.client.registration.oidc.client-authentication-method=client_secret_post
spring.security.oauth2.client.registration.oidc.authorization-grant-type=authorization_code
spring.security.oauth2.client.registration.oidc.redirect-uri=___SSO_URL___/login/oauth2/code/oidc
spring.security.oauth2.client.registration.oidc.scope=openid, profile, email
management.endpoints.web.exposure.include=health, info, prometheus, loggers

# Uncomment this parameter if a proxy modifies the urls and forwards the original url via the standard X-Forwarded-XXX headers
# server.forward-headers-strategy=NATIVE

4.3 Adding the marketplace to the deployment descriptors

services:
  dc-marketplace:
    # REGISTRY URL TO ADAPT and product version to adapt
    image: ${HARBOR_REGISTRY}/dc/marketplace:${PRODUCT_VERSION}
    networks:
      traefik_network:
      dc_network:
        aliases:
          - dc-marketplace
    volumes:
      - type: bind
        # Log directory to adapt
        source: ${DC_CFG_DC_LOG_DIR_DATACHAIN}
        target: /logs
    env_file: marketplace.env
    deploy:
      mode: replicated
      replicas: 1
      labels:
        - "traefik.enable=true"
        - "traefik.docker.network=traefik_network"
        - "traefik.http.routers.dc_marketplace.rule=Host(`___PUBLIC_URL_OF_THE_MARKETPLACE___`) && PathPrefix(`/`)"
        # To adapt depending on your deployment, if secure
        #- "traefik.http.routers.dc_marketplace.entrypoints=web"
        #- "traefik.http.routers.dc_marketplace.entrypoints=websecure"
        #- "traefik.http.routers.dc_marketplace.tls=true"
        #- "traefik.http.routers.dc_marketplace.tls.certresolver=resolver1"
        - "traefik.http.routers.dc_marketplace.service=dc_marketplace"
        - "traefik.http.services.dc_marketplace.loadbalancer.server.port=8092"
        - "traefik.http.routers.dc_marketplace1.middlewares=dc_marketplace_comp1@docker"
        - "traefik.http.middlewares.dc_marketplace_comp1.compress=true"

networks:
  dc_network:
    external: true
    name: dc_network
  traefik_network:
    external: true
    name: traefik_network

Base configuration file

# default log configuration, which can be modified if needed
logging.level.ROOT=INFO
logging.level.com.adb=INFO
logging.level.org.springframework=INFO
logging.level.org.springframework.security=INFO
logging.level.org.springframework.data.redis=ERROR
#logging.level.org.hibernate.SQL=DEBUG
#logging.level.org.hibernate.orm.jdbc.bind=TRACE
#logging.level.org.apache.http=DEBUG
#logging.level.org.apache.http.wire=DEBUG

# Database configuration
dc.app.marketplace.database.main-datasource.url=jdbc:postgresql://dc-pg:5432/dc_marketplace
dc.app.marketplace.database.expose-datasource.url=jdbc:postgresql://dc-pg:5432/expose
dc.app.marketplace.database.main-datasource.password=___TO_BE_SPECIFIED___
dc.app.marketplace.database.expose-datasource.password=___TO_BE_SPECIFIED___

# Console access information
dc.app.module.console.url=http://dc-console:8093/
dc.app.module.instance-id=DC_MP

# Exposition root
dc.app.marketplace.expose.public-url=___TO_BE_SPECIFIED___

# Security configuration
dc.app.security.auth.oauth.resource-server.enable=true
dc.app.security.auth.oauth.client.enable=true
dc.app.security.auth.oauth.client.enable-oauth-logout=true

spring.security.oauth2.resourceserver.jwt.jwk-set-uri=___SSO_URL___/realms/dc-realm/protocol/openid-connect/certs
spring.security.oauth2.client.provider.oidc.issuer-uri=___SSO_URL___/realms/dc-realm
spring.security.oauth2.client.registration.oidc.client-id=dc_marketplace
spring.security.oauth2.client.registration.oidc.client-secret=___PREVIOUSLY_GENERATED_PASSWORD___
spring.security.oauth2.client.registration.oidc.client-authentication-method=client_secret_post
spring.security.oauth2.client.registration.oidc.authorization-grant-type=authorization_code
spring.security.oauth2.client.registration.oidc.redirect-uri=___SSO_URL___/login/oauth2/code/oidc
spring.security.oauth2.client.registration.oidc.scope=openid, profile, email

management.endpoints.web.exposure.include=health, info, prometheus, loggers
# Uncomment this parameter if a proxy modifies the urls and forwards the original url via the standard X-Forwarded-XXX headers
# server.forward-headers-strategy=NATIVE

4.4 Changing the dc-core configuration

In the front configuration (generally webui.env), remove the KEYCLOAK_CONFIG configuration, which is now unnecessary
In the backend configuration, add the modified SSO configuration to keycloak and the console configuration

Base configuration file

spring.security.oauth2.resourceserver.jwt.jwk-set-uri=${DC_CFG_KC_PUBLIC_URL}/realms/dc-realm/protocol/openid-connect/certs
spring.security.oauth2.client.provider.oidc.issuer-uri=${DC_CFG_KC_PUBLIC_URL}/realms/dc-realm
spring.security.oauth2.client.registration.oidc.client-id=${DC_CFG_DC_CORE_KC_CLIENT_ID}
spring.security.oauth2.client.registration.oidc.client-secret=${DC_CFG_DC_CORE_KC_CLIENT_SECRET}
spring.security.oauth2.client.registration.oidc.client-authentication-method=client_secret_post
spring.security.oauth2.client.registration.oidc.authorization-grant-type=authorization_code
spring.security.oauth2.client.registration.oidc.redirect-uri=${DC_CFG_DC_CORE_PUBLIC_URL}/login/oauth2/code/oidc
spring.security.oauth2.client.registration.oidc.scope=openid, profile, email

dc.app.security.auth.oauth.client.enable-oauth-logout=true

# Redis configuration
dc.app.notification.urls=${DC_CFG_DC_REDIS_PRIVATE_URL}

# Console activation
dc.app.module.instance-id=DATACHAIN_CORE_1
dc.app.module.console.sync.enable=true
dc.app.module.console.url=http://dc-console:8093/

# IF marketplace
dc.app.module.web.functionality.exposition_v2.available=true

# default log configuration, which can be modified if needed
logging.level.ROOT=INFO
logging.level.com.adb=INFO

# Uncomment this parameter if a proxy modifies the urls and forwards the original url via the standard X-Forwarded-XXX headers
# server.forward-headers-strategy=NATIVE

5. Deployment

Deploy the stack

6. Creating the first console administrator user (migration from a V8 platform without console)

Open a shell on the Datachain database

psql -U postgres
\c dc_console

/* Retrieve the console id, to be reported in _______INSTANCE_ID______ below */
SELECT id FROM dc_console.dcc_product_instance WHERE type='DC_CONSOLE';

/* Retrieve the id of the first administrator user, to be reported in ______USER_ID______ below */
SELECT id FROM dc_console_data.local_user WHERE mail='tests+admin@adobis.lan';

UPDATE dc_console.dcc_user_instance_account (id,"version",created_at,updated_at,roles,user_id,instance_id,organization_id) VALUES
(1,0,TIMEZONE('Europe/Paris', NOW()),TIMEZONE('Europe/Paris', NOW()),'DCC_ADMIN;DCC_ORGANIZATION_ADMIN;DCC_MEMBER', ______USER_ID______ ,_______INSTANCE_ID______ ,1) ON CONFLICT DO NOTHING;

INSERT INTO dc_console_data.local_user_roles (id,created_at,updated_at,roles,organization_id, user_id) VALUES
    (1,TIMEZONE('Europe/Paris', NOW()),TIMEZONE('Europe/Paris', NOW()),'DCC_ADMIN;DCC_ORGANIZATION_ADMIN;DCC_MEMBER',1,______USER_ID______) ON CONFLICT DO NOTHING;

7. Migrating the user roles from keycloak into the console

Open a shell on the console container and run the following command

# set the value   --dc.app.console.keycloak.migration.create-members=false
# instead of true if the console provisioning is external (ldap) so as
# not to recreate users who would no longer be in the directory.
java -jar dc-console-*.jar \
  --dc.app.console.keycloak.enable-kc-migration=true \
  --dc.app.console.keycloak.migration.create-members=true \
  --spring.profiles.active=exec,prod \
  --dc.app.console.keycloak.url=http://dc_keycloak:8080/auth \
  --dc.app.console.keycloak.realm=dc-realm \
  --dc.app.console.keycloak.sync-client-id=dc_backend_keycloak \
  --dc.app.console.keycloak.sync-client-secret=6bcf085a-44c8-40b1-9c0c-9381c9749a8c \
  --dc.app.console.keycloak.resource=dc_backend \
  --server.port=13333 --management.server.port=13334 \
  --dc.app.console.provisioning.ldap.first.active=false \
  --dc.app.security.auth.kerberos.enable=false \
  --dc.app.security.auth.oauth.resource-server.enable=false \
  --dc.app.security.auth.oauth.client.enable=false \
  --dc.app.security.auth.api-key.enable=false \
  --logging.level.ROOT=INFO \
  --logging.level.org.springframework=ERROR \
  --logging.level.org.springframework.security=ERROR \
  --logging.level.org.springframework.data.redis=ERROR \
  --logging.level.org.hibernate.SQL=ERROR \
  --logging.level.org.hibernate.orm.jdbc.bind=ERROR \
  --logging.level.com.adb=DEBUG

8. Validating the Datachain-Core migration

Database driver updates

Database type	Version	Driver compatibility matrix	Comments
PostgreSQL	42.7.7	https://jdbc.postgresql.org/download/
SQLServer	12.6.4	https://learn.microsoft.com/en-us/sql/connect/jdbc/microsoft-jdbc-driver-for-sql-server-support-matrix?view=sql-server-ver17#sql-version-compatibility
Oracle	19.22.0.0	https://www.oracle.com/uk/database/technologies/faq-jdbc.html (question: What is the JDBC and RDBMS interoperability matrix or the certification matrix?)	With internationalization support (oraI18n)
Mysql	9.4.0	https://dev.mysql.com/doc/connector-j/en/connector-j-versions.html
Mariadb	3.5.5	https://mariadb.com/docs/connectors/mariadb-connector-j/about-mariadb-connector-j

Database type

Version

Driver compatibility matrix

Comments

PostgreSQL

42.7.7

https://jdbc.postgresql.org/download/

SQLServer

12.6.4

https://learn.microsoft.com/en-us/sql/connect/jdbc/microsoft-jdbc-driver-for-sql-server-support-matrix?view=sql-server-ver17#sql-version-compatibility

Oracle

19.22.0.0

https://www.oracle.com/uk/database/technologies/faq-jdbc.html (question: What is the JDBC and RDBMS interoperability matrix or the certification matrix?)

With internationalization support (oraI18n)

Mysql

9.4.0

https://dev.mysql.com/doc/connector-j/en/connector-j-versions.html

Mariadb

3.5.5

https://mariadb.com/docs/connectors/mariadb-connector-j/about-mariadb-connector-j

Version 8.9.0

Details on the database drivers:

Database type	Version	Driver compatibility matrix	Comments
PostgreSQL	42.6.0	https://jdbc.postgresql.org/download/
SQLServer	12.2.0	https://learn.microsoft.com/en-us/sql/connect/jdbc/microsoft-jdbc-driver-for-sql-server-support-matrix?view=sql-server-ver17#sql-version-compatibility
Oracle	19.22.0.0	https://www.oracle.com/uk/database/technologies/faq-jdbc.html (question: What is the JDBC and RDBMS interoperability matrix or the certification matrix?)	With internationalization support (oraI18n)
Mysql	8.1.0	https://dev.mysql.com/doc/connector-j/en/connector-j-versions.html
Mariadb	3.2.0	https://mariadb.com/docs/connectors/mariadb-connector-j/about-mariadb-connector-j

Database type

Version

Driver compatibility matrix

Comments

PostgreSQL

42.6.0

https://jdbc.postgresql.org/download/

SQLServer

12.2.0

https://learn.microsoft.com/en-us/sql/connect/jdbc/microsoft-jdbc-driver-for-sql-server-support-matrix?view=sql-server-ver17#sql-version-compatibility

Oracle

19.22.0.0

https://www.oracle.com/uk/database/technologies/faq-jdbc.html (question: What is the JDBC and RDBMS interoperability matrix or the certification matrix?)

With internationalization support (oraI18n)

Mysql

8.1.0

https://dev.mysql.com/doc/connector-j/en/connector-j-versions.html

Mariadb

3.2.0

https://mariadb.com/docs/connectors/mariadb-connector-j/about-mariadb-connector-j

Version 8.8.1

No additional technical information.

Version 8.8.0

One or more patches modify database structure. Verify migration service execution.

Version 8.7.1

Version 8.7.0

Manual Update Procedures

Console and Marketplace Update

For users of beta console and marketplace versions, manual operations are required during migration. The data_view database and schema are replaced by marketplace database/schema, with renamed parameters for clarity.

Update procedure:

Stop non-database services
Backup databases
On data_view/marketplace database, do the following:

\c postgres
ALTER DATABASE data_view RENAME TO dc_marketplace;
\c dc_marketplace
ALTER SCHEMA data_view RENAME TO dc_marketplace;
UPDATE dc_marketplace.schema_changes SET checksum='-774607648' WHERE installed_rank=1;

On dc_console database:

\c dc_console
DELETE from dc_console.schema_changes WHERE version='1.1.1';

In marketplace (ex data_view) docker-compose file: rename image data_view to marketplace
In environment/docker-compose files: replace all dc.app.view parameters with dc.app.marketplace
If marketplace DB URL was overridden (via dc.app.marketplace.database.main-datasource), adapt URL to new database name. For example:

dc.app.marketplace.database.main-datasource.url=jdbc:postgresql://dc-pg:5432/dc_marketplace?prepareThreshold=0&preparedStatementCacheQueries=0&defaultRowFetchSize=1000

Deploy version 8.7.0

DataChain Core Update

One or more patches modify database structure. Verify migration service execution.

🚀 Technical Notes

New/modified API services for simplified interaction with connectors, repositories, and DataBlocks:

Duplicate project POST /service/projects/{id}/duplicate: Modified duplication service. Previously required full English consent text rewrite - now uses a boolean flag.
Check Connector connectivity POST /service/connectors/detailed/check_connection/{connectorId}: New generic connector validation service. ctx parameter optional.
Get connector by ID GET /service/connectors/detailed/{connectorId}: New connector details service using only ID (type no longer required).
List depots GET /service/depots: Modified service now returns project depots with their types.
Create Extraction POST /service/depots/detailed/{depotId}/extractions: New generic extraction creation using depot ID alone.
Get depot preview GET /service/depots/detailed/{depotId}/remote_file/preview: New preview service requiring only depot ID.
Execute DataBlock POST/service/datablock/exec_by_element_id: New execution service returning job key
Get execution results GET /service/datablock/exec/result/detailed: New service using job key to display detailed results

Version 8.6.0

One or more patches modify database structure. Verify migration service execution.

Spark container ports aligned with monitoring standards - Spark now listens on port 9090. Update Spark health checks in docker-compose and Traefik rules for actuator/monitoring access. Port list available here.

🚀 Technical Notes

Integration of information on:

Solution architecture and deployment
Security
Technical administration (including backup/restore procedures)
Advanced service descriptions (ports, volumes)
DataChain Core monitoring

Version 8.5.4

No additional technical information.

Version 8.5.3

Added PostgreSQL function. No data structure changes.

Version 8.5.2

No additional technical information.

Version 8.5.1

Database driver updates:

Database Type	Version	Comments
PostgreSQL	42.6.0
SQLServer	12.2.0
Oracle	19.22.0.0	Internationalization support (oraI18n)
MySQL	8.1.0
MariaDB	3.2.0

Version 8.5.0

Database migrations applied during installation. Verify patches success via migration logs.

Driver updates:

Database Type	Version	Comments
PostgreSQL	42.6.0
SQLServer	12.2.0
Oracle	19.19.0.0	Internationalization support (oraI18n)
MySQL	8.1.0
MariaDB	3.2.0

Version 8.4.0

Database migrations applied. Verify migration logs.
Added SMTP configuration parameters for deployment-level setup.

Version 8.3.1/8.3.2

No additional technical information.

Version 8.3.0

🚀 Technical Notes

Database Model

Database migrations applied. Verify patches success via migration logs.

Version 8.2.0

🚀 Technical Notes

During a new data exposure, the data to be replaced remains available for the duration of the exposure, to guarantee continued access to the data.

Database Model

Multiple database migrations applied. Verify patches success via migration logs.

Patches update exposure database. Verify pg_migration configuration. Configuration details here.

Dernière mise à jour le 21 octobre 2026 à 10:54