Export base

docs/configuration/authentication/saml-team.md

@@ -1,7 +1,7 @@
 ---
 status: wip
-search:
-  exclude: true
+#search:
+#  exclude: true
 ---
 
 # SAML Authentication (for multi-tenancy environments)

docs/configuration/dtable-web-settings.md

@@ -116,9 +116,12 @@ SESSION_EXPIRE_AT_BROWSER_CLOSE = False
 # Whether to save the session data on every request. Default is `False`
 SESSION_SAVE_EVERY_REQUEST = False
 
-# Whether to allow a user to create a base in personal workspace. Default by `False`
+# Whether to allow a user to create a base in personal workspace. Default is `False`
 DISABLE_ADDING_PERSONAL_BASES = False
 
+# Whether to show the option to generate an account token in the personal settings Default is `True`
+ENABLE_GENERATE_WEB_API_TOKEN_IN_WEB = True
+
 ```
 
 ## Other options

docs/installation/components/ofelia.md

@@ -37,7 +37,7 @@ By default, `ofelia.yml` adds labels to the `seatable-server` service. These lab
 | Slug        | Description                                                                                                                     | Schedule  |
 | ----------- | ------------------------------------------------------------------------------------------------------------------------------- | --------- |
 | `clean-tmp` | Removes any files older than 3 days from the `/tmp` directory inside the container to the system from running out of disk space | `@weekly` |
-| `gc`        | Runs the [garbage collection](../../maintenance/garbage-collection.md) process to clean up unused blocks                        | `@weekly` |
+| `gc`        | Runs the [garbage collection](../../maintenance/storage-cleanup.md) process to clean up unused blocks                           | `@weekly` |
 | `seaf-fsck` | Runs the [`seaf-fsck`](../../maintenance/filesystem-check.md) script to check the integrity of all Seafile libraries            | `@weekly` |
 
 You are free to modify the schedule of these commands and/or add your own commands via additional labels on the `seatable-server` container.

docs/maintenance/base-export.md

@@ -4,39 +4,101 @@ status: wip
 
 # Exporting a Base from the Command Line
 
-SeaTable allows you to export a base using the command line, bypassing typical limitations from the web interface.
+SeaTable provides two ways to export a base using the command line. You can perform a simple export to a single `.dtable` file or an advanced export that includes all data, including big data content. Both methods bypass the limits of the web interface.
 
-!!! warning "Big Data Exclusion"
+## Simple Export: Create a .dtable File
 
-    Please note that the base export does not include big data.
+The simple export generates a ready-to-use `.dtable` file that can be imported into another SeaTable instance.
 
-## Export Command
+### Advantages
 
-Use the following command to export a base by its `base_uuid`. The dtable file will be saved to the `/templates` folder inside the docker container.
+- Easy to run and import
+- Produces a single `.dtable` file
 
-```
+### Limitations
+
+- Maximum export size: 100 MB
+- Does not include big data content from the SeaTable Big Data backend
+
+### Export command
+
+Use the following command to export a base by its `base_uuid`. The `.dtable` file will be saved to your current directory. In this example, the `/shared` folder is used because it is accessible from the SeaTable host.
+
+```bash
 docker exec -it seatable-server bash
-cd /templates
+cd /shared
 seatable.sh python-env /opt/seatable/seatable-server-latest/dtable-web/manage.py export_dtable <base_uuid>
 ```
 
-Replace `<base_uuid>` with the actual UUID of the base.
+Replace `<base_uuid>` with your actual base UUID.
 
-## Exporting Without Assets
+### Excluding Attached Files and Images
 
-By default, the export includes all data and content from image and file columns. To exclude file/image columns and export only base data, use the `--ignore-asset` parameter:
+To export only the structured base data and omit all assets (images, files), use the `--ignore-asset` option:
 
 ```
 seatable.sh python-env /opt/seatable/seatable-server-latest/dtable-web/manage.py export_dtable <base_uuid> --ignore-asset true
 ```
 
-## Troubleshooting Large Asset Exports
+### If the Export Fails Due to Size Limits
 
-If your base contains numerous assets, SeaTable may refuse to export with assets included. To resolve this, increase the `max_download_dir_size` value in the `seafile.conf` file and [restart SeaTable](../maintenance/restart-seatable.md):
+If your base contains many attached files/images, you might encounter the `max_download_dir_size` limitation. You can increase this limit (default is 100 MB) in `seafile.conf`: 
 
-```
+```ini
 [fileserver]
 max_download_dir_size=1000
 ```
 
-This adjustment allows the creation of dtable files up to 1 GB in size.
+Then [restart SeaTable](../maintenance/restart-seatable.md)
+This allows `.dtable` exports up to 1 GB in size.
+
+## Advanced Export: Export to a Folder with All Data
+
+<!-- md:version 6.0 -->
+
+The advanced export uses the `export_dtable_folder` command. It creates a local folder that contains all base data, assets, and big data content. This approach is ideal for complete backups or when moving large datasets.
+
+### Advantages
+
+- Includes data from the Big Data backend
+- No size limitation (overcomes the 100 MB default limit)
+
+### Limitations
+
+- The export produces a folder structure, not a single `.dtable` file
+- Requires manual handling when re-importing or transferring
+
+### Export Command
+
+Run the following command to export all content, including big data, to your current folder. In this example, the `/shared` folder is used because it is accessible from the SeaTable host.
+
+```bash
+docker exec -it seatable-server bash
+cd /shared
+seatable.sh python-env /opt/seatable/seatable-server-latest/dtable-web/manage.py export_dtable_folder <base_uuid>
+```
+
+This command creates:
+
+- a folder with the name of the base
+- inside this folder:
+    - a content.json file with the base’s structure and data
+    - one or more subfolders for assets and big data files
+
+### Optional parameters
+
+- `--ignore-archive-backup true`: Skips the export of archived backup data. Use this if archived backups are not required in the exported result.
+
+## Importing an Advanced Export
+
+<!-- md:version 6.0 -->
+
+To import an exported base folder into another group/workspace, use the following command. 
+
+```bash
+seatable.sh python-env /opt/seatable/seatable-server-latest/dtable-web/manage.py import_dtable_folder --workspace-id="<workspace id>" --path="<path to folder>"
+```
+
+Replace `<workspace id>` with the ID of the target workspace/group and provide the path to the folder containing the export.
+
+After running the command, check that the output indicates a successful import.

docs/maintenance/big-data-cleanup.md

@@ -0,0 +1,58 @@
+# Big Data Cleanup
+
+<!-- md:version 6.0 -->
+
+Unused or deleted big data storage is not automatically reclaimed. To free up disk space for bases that use large amounts of big data storage, run the following commands as needed.
+
+## Detecting Bases with Large Big Data Storage
+
+Check the size of the folders stored in `/opt/seatable-server/seatable/db-data`.  
+Using the `ncdu` tool is recommended, as it shows the storage usage per folder.
+
+Example output:
+
+```
+--- /opt/dtable-db/seatable/db-data ------------------------------------------
+  135.4 MiB [##########################] /9ba24950-cd00-42dc-b5e9-233f3de2c2b6
+   54.0 MiB [##########                ] /281b15c1-2a12-4da3-814d-145cb703a55b
+   40.5 MiB [#######                   ] /76a4c7e8-2422-4c3f-9282-40e1e9fd5afc
+   33.3 MiB [######                    ] /35a51052-0f29-4628-91f3-9aad177e1bd2
+   33.1 MiB [######                    ] /52fd3b11-d00f-4fdf-b2ce-d306e08c4697
+   30.4 MiB [#####                     ] /9220dada-2a6a-4392-8dee-0d77af0f5a02
+   28.9 MiB [#####                     ] /3adc3d74-1783-40a9-9d05-245e9dc03f21
+```
+
+Each folder name corresponds to the UUID of a base.
+
+## Garbage Collection for Big Data
+
+Run garbage collection on a base to remove unused or deleted big data entries:
+
+```bash
+docker exec -it seatable-server bash
+cd /opt/seatable/seatable-server-latest/
+./dtable-db-admin.sh gc "<base_uuid>"
+```
+
+Replace `<base_uuid>` with the actual UUID of the base.
+
+This command reclaims unreferenced data blocks and cleans up invalid references in the big data store.
+
+## Flatten Command
+
+After running garbage collection, you may see log warnings such as:
+
+```
+Unable to read: Key: xxx, file with ID not found
+```
+
+These messages can appear during backup operations. To clean up remaining invalid keys, run the flatten command:
+
+```bash
+./dtable-db-admin.sh flatten "<base_uuid>"
+```
+
+!!! note "Archived keys and values are stored in separate files in `dtable-db`"
+
+    The warning occurs when garbage collection removes value files but leaves behind invalid key entries.  
+    This does not affect data integrity and is primarily an internal maintenance issue.

docs/maintenance/helper-scripts.md

@@ -13,112 +13,122 @@ After installing SeaTable, `/opt/seatable-compose/tools` contains the following
 This script allows you to enable an **existing** SeaTable user inside the database.
 Simply provide the user's contact email address as a parameter:
 
-```bash
-./tools/activate-user.sh "user@email.com"
-```
+=== "Input"
 
-**Output**
+    ```bash
+    ./tools/activate-user.sh "user@email.com"
+    ```
 
-```
---------------
-UPDATE ccnet_db.EmailUser SET is_active = 1 WHERE email = (SELECT user FROM dtable_db.profile_profile WHERE contact_email = 'user@email.com')
---------------
+=== "Output"
 
-Query OK, 1 row affected (0.002 sec)
-Rows matched: 1  Changed: 1  Warnings: 0
+    ```
+    --------------
+    UPDATE ccnet_db.EmailUser SET is_active = 1 WHERE email = (SELECT user FROM dtable_db.profile_profile WHERE contact_email = 'user@email.com')
+    --------------
 
-Bye
-Success: Activated user user@email.com
-```
+    Query OK, 1 row affected (0.002 sec)
+    Rows matched: 1  Changed: 1  Warnings: 0
+
+    Bye
+    Success: Activated user user@email.com
+    ```
 
 ### deactivate-user.sh
 
 This script allows you to disable an existing SeaTable user inside the database.
 Simply provide the user's email address as a parameter:
 
-```bash
-./tools/deactivate-user.sh "user@email.com"
-```
+=== "Input"
+
+    ```bash
+    ./tools/deactivate-user.sh "user@email.com"
+    ```
 
-**Output**
+=== "Output"
 
-```
---------------
-UPDATE ccnet_db.EmailUser SET is_active = 0 WHERE email = (SELECT user FROM dtable_db.profile_profile WHERE contact_email = 'user@email.com')
---------------
+    ```
+    --------------
+    UPDATE ccnet_db.EmailUser SET is_active = 0 WHERE email = (SELECT user FROM dtable_db.profile_profile WHERE contact_email = 'user@email.com')
+    --------------
 
-Query OK, 1 row affected (0.002 sec)
-Rows matched: 1  Changed: 1  Warnings: 0
+    Query OK, 1 row affected (0.002 sec)
+    Rows matched: 1  Changed: 1  Warnings: 0
 
-Bye
-Success: Deactivated user user@email.com
-```
+    Bye
+    Success: Deactivated user user@email.com
+    ```
 
 ### user-stats.sh
 
 This script will print your current license limit and query the database for the number of enabled users:
 
-```bash
-./tools/user-stats.sh
-```
+=== "Input"
+
+    ```bash
+    ./tools/user-stats.sh
+    ```
 
-**Output**
+=== "Output"
 
-```
-User limit according to license file: 3
+    ```
+    User limit according to license file: 3
 
-Users in database:
-+-------+--------------+
-| users | active_users |
-+-------+--------------+
-|     2 |            2 |
-+-------+--------------+
-```
+    Users in database:
+    +-------+--------------+
+    | users | active_users |
+    +-------+--------------+
+    |     2 |            2 |
+    +-------+--------------+
+    ```
 
 ## Database management
 
 ### db-shell.sh
 
 This script will give you an interactive shell inside the MariaDB container. You can use this to directly run SQL commands.
 
-```bash
-./tools/db-shell.sh
-```
+=== "Input"
 
-**Output**
+    ```bash
+    ./tools/db-shell.sh
+    ```
 
-```
-Welcome to the MariaDB monitor.  Commands end with ; or \g.
-Your MariaDB connection id is 73
-Server version: 11.4.3-MariaDB-ubu2404 mariadb.org binary distribution
+=== "Output"
 
-Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
+    ```
+    Welcome to the MariaDB monitor.  Commands end with ; or \g.
+    Your MariaDB connection id is 73
+    Server version: 11.4.3-MariaDB-ubu2404 mariadb.org binary distribution
 
-Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
+    Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
 
-MariaDB [(none)]>
-```
+    Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
+
+    MariaDB [(none)]>
+    ```
 
 ### dump-database.sh
 
 This script will dump all three databases (`ccnet_db`, `dtable_db` and `seahub_db`) to `/opt/seatable-backup` on the host.
 
-```bash
-./tools/dump-database.sh
-```
+=== "Input"
+
+    ```bash
+    ./tools/dump-database.sh
+    ```
 
-**Output**
+=== "Output"
 
-```
-Success: Dumped databases into /opt/seatable-backup
-```
+    ```
+    Success: Dumped databases into /opt/seatable-backup
+    ```
 
-**Created Files**
+=== "Created Files"
 
-```
-total 228
--rw-r--r-- 1 root root  12184 May 23 11:30 ccnet_db-2025-05-23-11-30-27.sql
--rw-r--r-- 1 root root 177499 May 23 11:30 dtable_db-2025-05-23-11-30-27.sql
--rw-r--r-- 1 root root  38402 May 23 11:30 seafile_db-2025-05-23-11-30-27.sql
-```
+    ```
+    total 228
+    -rw-r--r-- 1 root root  12184 May 23 11:30 ccnet_db-2025-05-23-11-30-27.sql
+    -rw-r--r-- 1 root root 177499 May 23 11:30 dtable_db-2025-05-23-11-30-27.sql
+    -rw-r--r-- 1 root root  38402 May 23 11:30 seafile_db-2025-05-23-11-30-27.sql
+    ```