Context

After upgrading LangSmith to version 0.12.5, you may encounter ClickHouse migration errors when running the migration script like:

AS combined_insert; (details: code: 397, message: UNION is not supported for MATERIALIZED VIEW)

This error typically occur due to ClickHouse version compatibility issues or specific ClickHouse settings that need to be configured for the new migration scripts to work properly.

Answer

To resolve ClickHouse migration errors after upgrading LangSmith, follow these steps:

Step 1: Check Your ClickHouse Version

First, verify your current ClickHouse version:

kubectl exec -it deployments/clickhouse -- bash -c \
  'clickhouse-client --query "SELECT version();"'

Step 2: Upgrade ClickHouse if Necessary

LangSmith version 0.12.5 requires ClickHouse version 24.12+ or higher. If your version is lower, you'll need to upgrade ClickHouse to version 25.4 or later.

Step 3: Check Migration State

Check if there are any dirty migrations in your ClickHouse database:

kubectl exec -it deployments/clickhouse -- bash -c \
  'clickhouse-client --database=<CLICKHOUSE_DB> --query \
  "SELECT * FROM schema_migrations WHERE dirty=1 ORDER BY version DESC;"'

Step 4: Force Migration Version (if needed)

If you have dirty migrations, you may need to force the migration to a specific version. Use the version number that's one less than the error version:

kubectl exec -it deployments/langsmith-backend -- bash -c 'migrate -source "<file://clickhouse/migrations>" -database "clickhouse://$CLICKHOUSE_HOST:$CLICKHOUSE_NATIVE_PORT?username=$CLICKHOUSE_USER&password=$CLICKHOUSE_PASSWORD&database=$CLICKHOUSE_DB&x-multi-statement=true&x-migrations-table-engine=MergeTree&secure=$CLICKHOUSE_TLS" force <version>'

Step 5: Configure ClickHouse Setting

If you encounter errors related to column names in materialized views like:

details: code: 8, message: SELECT query outputs column with name 'kv', which is not found in the target table. Use 'AS' to assign alias that matches a column name

you need to set a specific ClickHouse setting allow_experimental_analyzer to 1 before running the migration script.

Step 6: Run the Migration Script

After completing the above steps, run the ClickHouse migration script:

kubectl exec -it deployments/langsmith-backend -- bash -c './ch_migration_entrypoint.sh'

The migration script should now complete successfully. If you continue to experience issues, ensure that your ClickHouse version is 24.12+ and that the allow_experimental_analyzer setting is properly configured.

Resources

https://docs.langchain.com/langsmith/self-hosted

https://clickhouse.com/docs/whats-new/changelog/2024#a-id249a-clickhouse-release-249-2024-09-26

https://clickhouse.com/docs/operations/settings/settings#allow_materialized_view_with_bad_select