You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/en/altinity-kb-setup-and-maintenance/altinity-kb-data-migration/rsync.md
+9-6Lines changed: 9 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,6 +10,13 @@ When migrating a large, live ClickHouse cluster (multi-terabyte scale) to a new
10
10
1.**Prepare the new cluster**
11
11
- Ensure the new cluster is set up with its own ZooKeeper (or Keeper).
12
12
- Configure ClickHouse but keep it stopped initially.
13
+
- For clickhouse-operator instances, you can stop all pods by CHI definition:
14
+
```
15
+
spec:
16
+
stop: "true"
17
+
```
18
+
and attach volumes (PVC) to a service pod.
19
+
13
20
2.**Initial data sync**
14
21
15
22
Run a full recursive sync of the data directory from the old server to the new one:
@@ -27,7 +34,7 @@ When migrating a large, live ClickHouse cluster (multi-terabyte scale) to a new
27
34
- `W`: copy whole files instead of using rsync’s delta algorithm (faster for large DB files).
28
35
- --delete: remove files from the destination that don’t exist on the source.
29
36
30
-
If you plan to run several replicas on a new cluster, rsync data to all of them. To save the performance of production servers, you can copy data to 1 new replica and then use it as a sourcefor others. However, you can start from a single replica and add more after switching.
37
+
If you plan to run several replicas on a new cluster, rsync data to all of them. To save the performance of production servers, you can copy data to 1 new replica and then use it as a sourcefor others. You can start with a single replica and add more after switching, but it will take more time afterward, as additional replicas need to pull all the data.
31
38
32
39
Add --bwlimit=100000 to preserve the performance of the production cluster while copying a lot of data.
33
40
@@ -71,9 +78,5 @@ where engine ilike 'Replicated%'
71
78
You'll need to use `/data1/clickhouse` instead of `/var/lib/clickhouse`in the rsync paths.
72
79
73
80
3. ClickHouse Docker container image does not have rsync installed. Add it using apt-get or run sidecar in k8s or run a service pod with volumes attached.
74
-
For clickhouse-operator instances, you can to stop all pods by CHI definition.
0 commit comments