From 951b85d6201e4ba61fa544b8c8e2d7e495f93886 Mon Sep 17 00:00:00 2001 From: Christoph Dyllick-Brenzinger Date: Tue, 28 Oct 2025 12:53:00 +0100 Subject: [PATCH 1/3] update base export --- .../configuration/authentication/saml-team.md | 4 +- docs/maintenance/base-export.md | 90 +++++++++-- docs/maintenance/helper-scripts.md | 140 ++++++++++-------- 3 files changed, 153 insertions(+), 81 deletions(-) diff --git a/docs/configuration/authentication/saml-team.md b/docs/configuration/authentication/saml-team.md index 58d9b2511..ada5aa3ba 100644 --- a/docs/configuration/authentication/saml-team.md +++ b/docs/configuration/authentication/saml-team.md @@ -1,7 +1,7 @@ --- status: wip -search: - exclude: true +#search: +# exclude: true --- # SAML Authentication (for multi-tenancy environments) diff --git a/docs/maintenance/base-export.md b/docs/maintenance/base-export.md index 7022fb196..21297788d 100644 --- a/docs/maintenance/base-export.md +++ b/docs/maintenance/base-export.md @@ -4,39 +4,101 @@ status: wip # Exporting a Base from the Command Line -SeaTable allows you to export a base using the command line, bypassing typical limitations from the web interface. +SeaTable provides two ways to export a base using the command line. You can perform a simple export to a single `.dtable` file or an advanced export that includes all data, including big data content. Both methods bypass the limits of the web interface. -!!! warning "Big Data Exclusion" +## Simple Export: Create a .dtable File - Please note that the base export does not include big data. +The simple export generates a ready-to-use `.dtable` file that can be imported into another SeaTable instance. -## Export Command +### Advantages -Use the following command to export a base by its `base_uuid`. The dtable file will be saved to the `/templates` folder inside the docker container. +- Easy to run and import +- Produces a single `.dtable` file -``` +### Limitations + +- Maximum export size: 100 MB +- Does not include big data content from the SeaTable Big Data backend + +### Export command + +Use the following command to export a base by its `base_uuid`. The `.dtable` file will be saved to your current directory. In this example, the `/shared` folder is used because it is accessible from the SeaTable host. + +```bash docker exec -it seatable-server bash -cd /templates +cd /shared seatable.sh python-env /opt/seatable/seatable-server-latest/dtable-web/manage.py export_dtable ``` -Replace `` with the actual UUID of the base. +Replace `` with your actual base UUID. -## Exporting Without Assets +### Excluding Attached Files and Images -By default, the export includes all data and content from image and file columns. To exclude file/image columns and export only base data, use the `--ignore-asset` parameter: +To export only the structured base data and omit all assets (images, files), use the `--ignore-asset` option: ``` seatable.sh python-env /opt/seatable/seatable-server-latest/dtable-web/manage.py export_dtable --ignore-asset true ``` -## Troubleshooting Large Asset Exports +### If the Export Fails Due to Size Limits -If your base contains numerous assets, SeaTable may refuse to export with assets included. To resolve this, increase the `max_download_dir_size` value in the `seafile.conf` file and [restart SeaTable](../maintenance/restart-seatable.md): +If your base contains many attached files/images, you might encounter the `max_download_dir_size` limitation. You can increase this limit (default is 100 MB) in `seafile.conf`: -``` +```ini [fileserver] max_download_dir_size=1000 ``` -This adjustment allows the creation of dtable files up to 1 GB in size. +Then [restart SeaTable](../maintenance/restart-seatable.md) +This allows `.dtable` exports up to 1 GB in size. + +## Advanced Export: Export to a Folder with All Data + + + +The advanced export uses the `export_dtable_folder` command. It creates a local folder that contains all base data, assets, and big data content. This approach is ideal for complete backups or when moving large datasets. + +### Advantages + +- Includes data from the Big Data backend +- No size limitation (overcomes the 100 MB default limit) + +### Limitations + +- The export produces a folder structure, not a single `.dtable` file +- Requires manual handling when re-importing or transferring + +### Export Command + +Run the following command to export all content, including big data, to your current folder. In this example, the `/shared` folder is used because it is accessible from the SeaTable host. + +```bash +docker exec -it seatable-server bash +cd /shared +seatable.sh python-env /opt/seatable/seatable-server-latest/dtable-web/manage.py export_dtable_folder +``` + +This command creates: + +- a folder with the name of the base +- inside this folder: + - a content.json file with the base’s structure and data + - one or more subfolders for assets and big data files + +### Optional parameters + +- `--ignore-archive-backup true`: Skips the export of archived backup data. Use this if archived backups are not required in the exported result. + +## Importing an Advanced Export + + + +To import an exported base folder into another group/workspace, use the following command. + +```bash +seatable.sh python-env /opt/seatable/seatable-server-latest/dtable-web/manage.py import_dtable_folder --workspace-id="" --path="" +``` + +Replace `` with the ID of the target workspace/group and provide the path to the folder containing the export. + +After running the command, check that the output indicates a successful import. \ No newline at end of file diff --git a/docs/maintenance/helper-scripts.md b/docs/maintenance/helper-scripts.md index 548766d56..5ab25a16f 100644 --- a/docs/maintenance/helper-scripts.md +++ b/docs/maintenance/helper-scripts.md @@ -13,67 +13,73 @@ After installing SeaTable, `/opt/seatable-compose/tools` contains the following This script allows you to enable an **existing** SeaTable user inside the database. Simply provide the user's contact email address as a parameter: -```bash -./tools/activate-user.sh "user@email.com" -``` +=== "Input" -**Output** + ```bash + ./tools/activate-user.sh "user@email.com" + ``` -``` --------------- -UPDATE ccnet_db.EmailUser SET is_active = 1 WHERE email = (SELECT user FROM dtable_db.profile_profile WHERE contact_email = 'user@email.com') --------------- +=== "Output" -Query OK, 1 row affected (0.002 sec) -Rows matched: 1 Changed: 1 Warnings: 0 + ``` + -------------- + UPDATE ccnet_db.EmailUser SET is_active = 1 WHERE email = (SELECT user FROM dtable_db.profile_profile WHERE contact_email = 'user@email.com') + -------------- -Bye -Success: Activated user user@email.com -``` + Query OK, 1 row affected (0.002 sec) + Rows matched: 1 Changed: 1 Warnings: 0 + + Bye + Success: Activated user user@email.com + ``` ### deactivate-user.sh This script allows you to disable an existing SeaTable user inside the database. Simply provide the user's email address as a parameter: -```bash -./tools/deactivate-user.sh "user@email.com" -``` +=== "Input" + + ```bash + ./tools/deactivate-user.sh "user@email.com" + ``` -**Output** +=== "Output" -``` --------------- -UPDATE ccnet_db.EmailUser SET is_active = 0 WHERE email = (SELECT user FROM dtable_db.profile_profile WHERE contact_email = 'user@email.com') --------------- + ``` + -------------- + UPDATE ccnet_db.EmailUser SET is_active = 0 WHERE email = (SELECT user FROM dtable_db.profile_profile WHERE contact_email = 'user@email.com') + -------------- -Query OK, 1 row affected (0.002 sec) -Rows matched: 1 Changed: 1 Warnings: 0 + Query OK, 1 row affected (0.002 sec) + Rows matched: 1 Changed: 1 Warnings: 0 -Bye -Success: Deactivated user user@email.com -``` + Bye + Success: Deactivated user user@email.com + ``` ### user-stats.sh This script will print your current license limit and query the database for the number of enabled users: -```bash -./tools/user-stats.sh -``` +=== "Input" + + ```bash + ./tools/user-stats.sh + ``` -**Output** +=== "Output" -``` -User limit according to license file: 3 + ``` + User limit according to license file: 3 -Users in database: -+-------+--------------+ -| users | active_users | -+-------+--------------+ -| 2 | 2 | -+-------+--------------+ -``` + Users in database: + +-------+--------------+ + | users | active_users | + +-------+--------------+ + | 2 | 2 | + +-------+--------------+ + ``` ## Database management @@ -81,44 +87,48 @@ Users in database: This script will give you an interactive shell inside the MariaDB container. You can use this to directly run SQL commands. -```bash -./tools/db-shell.sh -``` +=== "Input" -**Output** + ```bash + ./tools/db-shell.sh + ``` -``` -Welcome to the MariaDB monitor. Commands end with ; or \g. -Your MariaDB connection id is 73 -Server version: 11.4.3-MariaDB-ubu2404 mariadb.org binary distribution +=== "Output" -Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. + ``` + Welcome to the MariaDB monitor. Commands end with ; or \g. + Your MariaDB connection id is 73 + Server version: 11.4.3-MariaDB-ubu2404 mariadb.org binary distribution -Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. + Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. -MariaDB [(none)]> -``` + Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. + + MariaDB [(none)]> + ``` ### dump-database.sh This script will dump all three databases (`ccnet_db`, `dtable_db` and `seahub_db`) to `/opt/seatable-backup` on the host. -```bash -./tools/dump-database.sh -``` +=== "Input" + + ```bash + ./tools/dump-database.sh + ``` -**Output** +=== "Output" -``` -Success: Dumped databases into /opt/seatable-backup -``` + ``` + Success: Dumped databases into /opt/seatable-backup + ``` -**Created Files** +=== "Created Files" -``` -total 228 --rw-r--r-- 1 root root 12184 May 23 11:30 ccnet_db-2025-05-23-11-30-27.sql --rw-r--r-- 1 root root 177499 May 23 11:30 dtable_db-2025-05-23-11-30-27.sql --rw-r--r-- 1 root root 38402 May 23 11:30 seafile_db-2025-05-23-11-30-27.sql -``` + ``` + total 228 + -rw-r--r-- 1 root root 12184 May 23 11:30 ccnet_db-2025-05-23-11-30-27.sql + -rw-r--r-- 1 root root 177499 May 23 11:30 dtable_db-2025-05-23-11-30-27.sql + -rw-r--r-- 1 root root 38402 May 23 11:30 seafile_db-2025-05-23-11-30-27.sql + ``` From bee78b594188f74b0962dc427fa56024ccfb4c2a Mon Sep 17 00:00:00 2001 From: Christoph Dyllick-Brenzinger Date: Tue, 28 Oct 2025 13:15:29 +0100 Subject: [PATCH 2/3] add big data cleanup --- docs/installation/components/ofelia.md | 2 +- docs/maintenance/big-data-cleanup.md | 58 +++++++++++++++++++ ...rbage-collection.md => storage-cleanup.md} | 2 +- mkdocs.yml | 3 +- 4 files changed, 62 insertions(+), 3 deletions(-) create mode 100644 docs/maintenance/big-data-cleanup.md rename docs/maintenance/{garbage-collection.md => storage-cleanup.md} (98%) diff --git a/docs/installation/components/ofelia.md b/docs/installation/components/ofelia.md index bc4e6d380..3b76f0d46 100644 --- a/docs/installation/components/ofelia.md +++ b/docs/installation/components/ofelia.md @@ -37,7 +37,7 @@ By default, `ofelia.yml` adds labels to the `seatable-server` service. These lab | Slug | Description | Schedule | | ----------- | ------------------------------------------------------------------------------------------------------------------------------- | --------- | | `clean-tmp` | Removes any files older than 3 days from the `/tmp` directory inside the container to the system from running out of disk space | `@weekly` | -| `gc` | Runs the [garbage collection](../../maintenance/garbage-collection.md) process to clean up unused blocks | `@weekly` | +| `gc` | Runs the [garbage collection](../../maintenance/storage-cleanup.md) process to clean up unused blocks | `@weekly` | | `seaf-fsck` | Runs the [`seaf-fsck`](../../maintenance/filesystem-check.md) script to check the integrity of all Seafile libraries | `@weekly` | You are free to modify the schedule of these commands and/or add your own commands via additional labels on the `seatable-server` container. diff --git a/docs/maintenance/big-data-cleanup.md b/docs/maintenance/big-data-cleanup.md new file mode 100644 index 000000000..0b0d8a445 --- /dev/null +++ b/docs/maintenance/big-data-cleanup.md @@ -0,0 +1,58 @@ +# Big Data Cleanup + + + +Unused or deleted big data storage is not automatically reclaimed. To free up disk space for bases that use large amounts of big data storage, run the following commands as needed. + +## Detecting Bases with Large Big Data Storage + +Check the size of the folders stored in `/opt/seatable-server/seatable/db-data`. +Using the `ncdu` tool is recommended, as it shows the storage usage per folder. + +Example output: + +``` +--- /opt/dtable-db/seatable/db-data ------------------------------------------ + 135.4 MiB [##########################] /9ba24950-cd00-42dc-b5e9-233f3de2c2b6 + 54.0 MiB [########## ] /281b15c1-2a12-4da3-814d-145cb703a55b + 40.5 MiB [####### ] /76a4c7e8-2422-4c3f-9282-40e1e9fd5afc + 33.3 MiB [###### ] /35a51052-0f29-4628-91f3-9aad177e1bd2 + 33.1 MiB [###### ] /52fd3b11-d00f-4fdf-b2ce-d306e08c4697 + 30.4 MiB [##### ] /9220dada-2a6a-4392-8dee-0d77af0f5a02 + 28.9 MiB [##### ] /3adc3d74-1783-40a9-9d05-245e9dc03f21 +``` + +Each folder name corresponds to the UUID of a base. + +## Garbage Collection for Big Data + +Run garbage collection on a base to remove unused or deleted big data entries: + +```bash +docker exec -it seatable-server bash +cd /opt/seatable/seatable-server-latest/ +./dtable-db-admin.sh gc "" +``` + +Replace `` with the actual UUID of the base. + +This command reclaims unreferenced data blocks and cleans up invalid references in the big data store. + +## Flatten Command + +After running garbage collection, you may see log warnings such as: + +``` +Unable to read: Key: xxx, file with ID not found +``` + +These messages can appear during backup operations. To clean up remaining invalid keys, run the flatten command: + +```bash +./dtable-db-admin.sh flatten "" +``` + +!!! note "Archived keys and values are stored in separate files in `dtable-db`" + + The warning occurs when garbage collection removes value files but leaves behind invalid key entries. + This does not affect data integrity and is primarily an internal maintenance issue. diff --git a/docs/maintenance/garbage-collection.md b/docs/maintenance/storage-cleanup.md similarity index 98% rename from docs/maintenance/garbage-collection.md rename to docs/maintenance/storage-cleanup.md index c319cb690..8112597b5 100644 --- a/docs/maintenance/garbage-collection.md +++ b/docs/maintenance/storage-cleanup.md @@ -1,4 +1,4 @@ -# Garbage Collection +# Storage Cleanup (Garbage Collection) SeaTable uses storage de-duplication technology to reduce storage usage. The underlying data blocks will not be removed immediately after you delete a file or a base. As a result, the number of unused data blocks will increase on SeaTable server. diff --git a/mkdocs.yml b/mkdocs.yml index 773bdb509..b219032ac 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -269,7 +269,8 @@ nav: - Restart SeaTable: maintenance/restart-seatable.md - Backup and Recovery: maintenance/backup-recovery.md - Clean Database: maintenance/database-cleanup.md - - Clean Storage: maintenance/garbage-collection.md + - Clean Storage: maintenance/storage-cleanup.md + - Clean Big Data: maintenance/big-data-cleanup.md - Filesystem check: maintenance/filesystem-check.md - Change URL: maintenance/domain-change.md - Export Base: maintenance/base-export.md From 9e5faf696a13cf3664842779fe29e6c2daeddc4a Mon Sep 17 00:00:00 2001 From: Christoph Dyllick-Brenzinger Date: Tue, 28 Oct 2025 13:52:11 +0100 Subject: [PATCH 3/3] add enable_generate_web_api_token_in_web --- docs/configuration/dtable-web-settings.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/configuration/dtable-web-settings.md b/docs/configuration/dtable-web-settings.md index f3637c9ed..6819776d7 100644 --- a/docs/configuration/dtable-web-settings.md +++ b/docs/configuration/dtable-web-settings.md @@ -116,9 +116,12 @@ SESSION_EXPIRE_AT_BROWSER_CLOSE = False # Whether to save the session data on every request. Default is `False` SESSION_SAVE_EVERY_REQUEST = False -# Whether to allow a user to create a base in personal workspace. Default by `False` +# Whether to allow a user to create a base in personal workspace. Default is `False` DISABLE_ADDING_PERSONAL_BASES = False +# Whether to show the option to generate an account token in the personal settings Default is `True` +ENABLE_GENERATE_WEB_API_TOKEN_IN_WEB = True + ``` ## Other options