Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions docs-mintlify/admin/connect-to-data/data-sources/clickhouse.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ CUBEJS_DB_PASS=**********
| [`CUBEJS_DB_PASS`](/reference/configuration/environment-variables#cubejs_db_pass) | The password used to connect to the database | A valid database password | ✅ |
| [`CUBEJS_DB_CLICKHOUSE_READONLY`](/reference/configuration/environment-variables#cubejs_db_clickhouse_readonly) | Whether the ClickHouse user has read-only access or not | `true`, `false` | ❌ |
| [`CUBEJS_DB_CLICKHOUSE_COMPRESSION`](/reference/configuration/environment-variables#cubejs_db_clickhouse_compression) | Whether the ClickHouse client has compression enabled or not | `true`, `false` | ❌ |
| [`CUBEJS_DB_SSL`](/reference/configuration/environment-variables#cubejs_db_ssl) | If `true`, enables SSL/TLS encryption between Cube and the database | `true`, `false` | ❌ |
| [`CUBEJS_DB_SSL_CA`](/reference/configuration/environment-variables#cubejs_db_ssl_ca) | The CA certificate, or a path to it. Required to enable mutual TLS | A valid certificate or path | ❌ |
| [`CUBEJS_DB_SSL_CERT`](/reference/configuration/environment-variables#cubejs_db_ssl_cert) | The client certificate, or a path to it. Set with `CUBEJS_DB_SSL_KEY` for mutual TLS | A valid certificate or path | ❌ |
| [`CUBEJS_DB_SSL_KEY`](/reference/configuration/environment-variables#cubejs_db_ssl_key) | The client private key, or a path to it. Set with `CUBEJS_DB_SSL_CERT` for mutual TLS | A valid key or path | ❌ |
| [`CUBEJS_DB_MAX_POOL`](/reference/configuration/environment-variables#cubejs_db_max_pool) | The maximum number of concurrent database connections to pool. Default is `20` | A valid number | ❌ |
| [`CUBEJS_CONCURRENCY`](/reference/configuration/environment-variables#cubejs_concurrency) | The number of [concurrent queries][ref-data-source-concurrency] to the data source | A valid number | ❌ |

Expand Down Expand Up @@ -122,6 +126,56 @@ To enable SSL-encrypted connections between Cube and ClickHouse, set the
configure custom certificates, please check out [Enable SSL Connections to the
Database][ref-recipe-enable-ssl].

### Mutual TLS (client certificates)

ClickHouse can authenticate users by client certificate (users created with
[`IDENTIFIED WITH ssl_certificate`](https://clickhouse.com/docs/en/operations/external-authenticators/ssl-x509)).
To connect Cube using mutual TLS, provide a CA certificate together with a
client certificate and key via the standard SSL environment variables:

```dotenv
CUBEJS_DB_SSL=true
CUBEJS_DB_SSL_CA=/path/to/ca.pem
CUBEJS_DB_SSL_CERT=/path/to/client.crt
CUBEJS_DB_SSL_KEY=/path/to/client.key
```

These variables accept either the certificate/key contents or a path to a file.
When `CUBEJS_DB_SSL_CERT` and `CUBEJS_DB_SSL_KEY` are both set, the driver
authenticates using the client certificate; the matching ClickHouse user should
be created with `IDENTIFIED WITH ssl_certificate CN '<common-name>'` and Cube's
`CUBEJS_DB_USER` set to that user.

<Info>

`@clickhouse/client` requires a CA certificate (`CUBEJS_DB_SSL_CA`) to be present
in order to enable mutual TLS.

</Info>

For dynamic, per-connection certificates — for example, authenticating each
tenant as a distinct ClickHouse user — pass the certificate material
programmatically from a [`driverFactory`](/reference/configuration/config#driverfactory)
using the `ssl` option. Unlike the environment variables, values passed here are
treated as raw certificate/key contents (string or `Buffer`), not file paths:

```javascript
const { ClickHouseDriver } = require('@cubejs-backend/clickhouse-driver');

module.exports = {
driverFactory: ({ securityContext }) => {
const { ca, cert, key } = loadTenantCertificate(securityContext.tenantId);

return new ClickHouseDriver({
host: process.env.CUBEJS_DB_HOST,
port: '8443',
username: securityContext.tenantId, // must match the certificate's CN-mapped user
ssl: { ca, cert, key },
});
},
};
```

## Additional Configuration

You can connect to a ClickHouse database when your user's permissions are
Expand Down
81 changes: 80 additions & 1 deletion packages/cubejs-clickhouse-driver/src/ClickHouseDriver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,22 @@ const ClickhouseTypeToGeneric: Record<string, string> = {
enum16: 'text',
};

/**
* TLS/SSL material used to connect to ClickHouse over HTTPS.
*
* Values are the certificate/key contents (PEM), either as a string or a
* Buffer. Supplying `cert` and `key` in addition to `ca` enables mutual TLS
* (client certificate authentication). Unlike the `CUBEJS_DB_SSL_*`
* environment variables, values passed here are treated as raw contents, not
* file paths — which makes it convenient to inject per-tenant certificates
* from a `driverFactory`.
*/
export interface ClickHouseDriverSslOptions {
ca?: string | Buffer,
cert?: string | Buffer,
key?: string | Buffer,
}

export interface ClickHouseDriverOptions {
host?: string,
port?: string,
Expand All @@ -70,6 +86,16 @@ export interface ClickHouseDriverOptions {
protocol?: string,
database?: string,
readOnly?: boolean,

/**
* TLS/SSL options for connecting to ClickHouse over HTTPS.
*
* When provided (or when the standard `CUBEJS_DB_SSL_*` environment
* variables are set) the driver connects over HTTPS. Provide `cert` and
* `key` alongside `ca` to enable mutual TLS, i.e. authenticate as a
* ClickHouse user created with `IDENTIFIED WITH ssl_certificate`.
*/
ssl?: ClickHouseDriverSslOptions,
/**
* Timeout in milliseconds for requests to ClickHouse.
* Default is 10 minutes
Expand Down Expand Up @@ -112,6 +138,14 @@ interface ClickhouseDriverExportKeySecretAWS extends ClickhouseDriverExportRequi
interface ClickhouseDriverExportAWS extends ClickhouseDriverExportKeySecretAWS {
}

/**
* TLS material in the shape expected by `@clickhouse/client`'s `createClient`.
* `ca_cert` alone enables basic TLS; adding `cert` and `key` enables mutual TLS.
*/
type ClickHouseTLSOptions =
| { ca_cert: Buffer }
| { ca_cert: Buffer, cert: Buffer, key: Buffer };

type ClickHouseDriverConfig = {
url: string,
username: string,
Expand All @@ -122,6 +156,7 @@ type ClickHouseDriverConfig = {
exportBucket: ClickhouseDriverExportAWS | null,
compression: { response?: boolean; request?: boolean },
clickhouseSettings: ClickHouseSettings,
tls?: ClickHouseTLSOptions,
};

export class ClickHouseDriver extends BaseDriver implements DriverInterface {
Expand Down Expand Up @@ -153,7 +188,8 @@ export class ClickHouseDriver extends BaseDriver implements DriverInterface {
const preAggregations = config.preAggregations || false;
const host = config.host ?? getEnv('dbHost', { dataSource, preAggregations });
const port = config.port ?? getEnv('dbPort', { dataSource, preAggregations }) ?? 8123;
const protocol = config.protocol ?? (getEnv('dbSsl', { dataSource, preAggregations }) ? 'https:' : 'http:');
const tls = this.buildTls(config, dataSource, preAggregations);
const protocol = config.protocol ?? ((getEnv('dbSsl', { dataSource, preAggregations }) || tls) ? 'https:' : 'http:');
const url = `${protocol}//${host}:${port}`;

const username = config.username ?? getEnv('dbUser', { dataSource, preAggregations });
Expand All @@ -175,6 +211,7 @@ export class ClickHouseDriver extends BaseDriver implements DriverInterface {
exportBucket: this.getExportBucket(dataSource, preAggregations),
readOnly: !!config.readOnly,
requestTimeout,
tls,
compression: {
// Response compression can't be enabled for a user with readonly=1, as ClickHouse will not allow settings modifications for such user.
response: this.readOnlyMode ? false : getEnv('clickhouseCompression', { dataSource, preAggregations }),
Expand Down Expand Up @@ -246,9 +283,51 @@ export class ClickHouseDriver extends BaseDriver implements DriverInterface {
clickhouse_settings: this.config.clickhouseSettings,
request_timeout: this.config.requestTimeout,
max_open_connections: maxPoolSize,
tls: this.config.tls,
});
}

/**
* Resolves TLS material into the shape `@clickhouse/client` expects.
*
* Options passed directly via `config.ssl` take precedence; otherwise the
* standard `CUBEJS_DB_SSL_*` environment variables are used (via
* `getSslOptions`, which also supports file paths). Mutual TLS is enabled
* only when `ca`, `cert` and `key` are all present.
*/
private buildTls(
config: ClickHouseDriverOptions,
dataSource: string,
preAggregations?: boolean,
): ClickHouseTLSOptions | undefined {
const source = config.ssl ?? this.getSslOptions(dataSource, preAggregations);
if (!source) {
return undefined;
}

const caCert = ClickHouseDriver.toCertBuffer(source.ca as string | Buffer | undefined);
if (!caCert) {
// @clickhouse/client requires a CA certificate to enable TLS;
// plain HTTPS (system CAs) needs no `tls` object at all.
return undefined;
}

const cert = ClickHouseDriver.toCertBuffer(source.cert as string | Buffer | undefined);
const key = ClickHouseDriver.toCertBuffer(source.key as string | Buffer | undefined);
if (cert && key) {
return { ca_cert: caCert, cert, key };
}

return { ca_cert: caCert };
}

private static toCertBuffer(value: string | Buffer | undefined): Buffer | undefined {
if (value === undefined || value === null) {
return undefined;
}
return Buffer.isBuffer(value) ? value : Buffer.from(value);
}

public async testConnection() {
await this.query('SELECT 1', []);
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
import { ClickHouseDriver } from '../src';

// These tests exercise how TLS/SSL options are resolved into the shape that
// `@clickhouse/client` expects. They do not require a running ClickHouse
// instance: the client only establishes a connection lazily, so we can inspect
// the resolved driver config after construction.
describe('ClickHouseDriver TLS options', () => {
const ca = '-----BEGIN CERTIFICATE-----\nca\n-----END CERTIFICATE-----';
const cert = '-----BEGIN CERTIFICATE-----\nclient\n-----END CERTIFICATE-----';
const key = '-----BEGIN PRIVATE KEY-----\nkey\n-----END PRIVATE KEY-----';

const SSL_ENV_KEYS = [
'CUBEJS_DB_SSL',
'CUBEJS_DB_SSL_CA',
'CUBEJS_DB_SSL_CERT',
'CUBEJS_DB_SSL_KEY',
'CUBEJS_DB_SSL_REJECT_UNAUTHORIZED',
];

const tlsOf = async (config: any) => {
const driver = new ClickHouseDriver({ host: 'localhost', port: '8123', ...config });
try {
return {
tls: (driver as any).config.tls,
url: (driver as any).config.url,
};
} finally {
await driver.release();
}
};

beforeEach(() => {
SSL_ENV_KEYS.forEach((k) => { delete process.env[k]; });
});

afterEach(() => {
SSL_ENV_KEYS.forEach((k) => { delete process.env[k]; });
});

it('does not configure TLS and uses http when no SSL is provided', async () => {
const { tls, url } = await tlsOf({});
expect(tls).toBeUndefined();
expect(url).toMatch(/^http:\/\//);
});

it('enables basic TLS (ca only) over https', async () => {
const { tls, url } = await tlsOf({ ssl: { ca } });
expect(tls).toEqual({ ca_cert: Buffer.from(ca) });
expect(url).toMatch(/^https:\/\//);
});

it('enables mutual TLS when ca, cert and key are provided', async () => {
const { tls, url } = await tlsOf({ ssl: { ca, cert, key } });
expect(tls).toEqual({
ca_cert: Buffer.from(ca),
cert: Buffer.from(cert),
key: Buffer.from(key),
});
expect(url).toMatch(/^https:\/\//);
});

it('preserves Buffer values as-is', async () => {
const caBuf = Buffer.from(ca);
const { tls } = await tlsOf({ ssl: { ca: caBuf } });
expect((tls as any).ca_cert).toBe(caBuf);
});

it('falls back to basic TLS when only cert/key (no ca) are provided', async () => {
// @clickhouse/client requires a CA certificate to build a `tls` object,
// so cert/key without a ca cannot enable mutual TLS.
const { tls } = await tlsOf({ ssl: { cert, key } });
expect(tls).toBeUndefined();
});

it('reads TLS material from CUBEJS_DB_SSL_* environment variables', async () => {
process.env.CUBEJS_DB_SSL = 'true';
process.env.CUBEJS_DB_SSL_CA = ca;
process.env.CUBEJS_DB_SSL_CERT = cert;
process.env.CUBEJS_DB_SSL_KEY = key;

const { tls, url } = await tlsOf({});
expect(tls).toEqual({
ca_cert: Buffer.from(ca),
cert: Buffer.from(cert),
key: Buffer.from(key),
});
expect(url).toMatch(/^https:\/\//);
});

it('prefers explicit ssl config over environment variables', async () => {
process.env.CUBEJS_DB_SSL = 'true';
process.env.CUBEJS_DB_SSL_CA = ca;

const otherCa = '-----BEGIN CERTIFICATE-----\nother\n-----END CERTIFICATE-----';
const { tls } = await tlsOf({ ssl: { ca: otherCa } });
expect(tls).toEqual({ ca_cert: Buffer.from(otherCa) });
});
});