diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8c143ba905..712128f5a7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -104,8 +104,7 @@ cargo 1.69.0 (6e9a83356 2023-04-12) Currently, iceberg-rust uses Docker to set up environment for integration tests. Native Docker has some limitations, please check (https://github.com/apache/iceberg-rust/pull/748). Please use Orbstack or Podman. For MacOS users, you can install [OrbStack as a docker alternative](docs/contributing/orbstack.md). - -For podman, refer to [Using Podman instead of Docker](docs/contributing/podman.md) +For Podman users, refer to [Using Podman instead of Docker](docs/contributing/podman.md) ## Build diff --git a/crates/integration_tests/testdata/docker-compose.yaml b/crates/integration_tests/testdata/docker-compose.yaml index f9851f3c71..e2f9b7da36 100644 --- a/crates/integration_tests/testdata/docker-compose.yaml +++ b/crates/integration_tests/testdata/docker-compose.yaml @@ -80,6 +80,3 @@ services: - AWS_ACCESS_KEY_ID=admin - AWS_SECRET_ACCESS_KEY=password - AWS_REGION=us-east-1 - links: - - rest:rest - - minio:minio diff --git a/crates/integration_tests/testdata/spark/provision.py b/crates/integration_tests/testdata/spark/provision.py index 1929266b4a..899a49f6ac 100755 --- a/crates/integration_tests/testdata/spark/provision.py +++ b/crates/integration_tests/testdata/spark/provision.py @@ -30,6 +30,8 @@ .getOrCreate() ) +spark.sql(f"""CREATE NAMESPACE IF NOT EXISTS rest.default""") + spark.sql( f""" CREATE OR REPLACE TABLE rest.default.test_positional_merge_on_read_deletes ( diff --git a/docs/contributing/podman.md b/docs/contributing/podman.md index 3281ad4da5..eee93a0b35 100644 --- a/docs/contributing/podman.md +++ b/docs/contributing/podman.md @@ -19,15 +19,16 @@ # Using Podman instead of Docker -Iceberg-rust does not require containerization, except for integration tests, where "docker" and "docker-compose" are used to start containers for minio and various catalogs. Below instructions setup "rootful podman" and docker's official docker-compose plugin to run integration tests as an alternative to docker or Orbstack. +Iceberg-rust does not require containerization, except for integration tests, where "docker" and "docker-compose" are used to start containers for minio and various catalogs. Below instructions setup "rootful podman" and docker's official docker-compose plugin to run integration tests as an alternative to docker or Orbstack. 1. Have podman v4 or newer. - ```console + ```shell $ podman --version podman version 4.9.4-rhel ``` - -2. Open file `/usr/bin/docker` and add the below contents: +2. Create a docker wrapper script: + + * Create a fresh `/usr/bin/docker` file and add the below contents: ```bash #!/bin/sh [ -e /etc/containers/nodocker ] || \ @@ -35,8 +36,13 @@ Iceberg-rust does not require containerization, except for integration tests, wh exec sudo /usr/bin/podman "$@" ``` + * Set new `/usr/bin/docker` file to executable. + ```shell + sudo chmod +x /usr/bin/docker + ``` + 3. Install the [docker compose plugin](https://docs.docker.com/compose/install/linux). Check for successful installation. - ```console + ```shell $ docker compose version Docker Compose version v2.28.1 ``` @@ -53,7 +59,7 @@ Iceberg-rust does not require containerization, except for integration tests, wh ``` 6. Check that the following symlink exists. - ```console + ```shell $ ls -al /var/run/docker.sock lrwxrwxrwx 1 root root 27 Jul 24 12:18 /var/run/docker.sock -> /var/run/podman/podman.sock ``` @@ -83,3 +89,12 @@ As of podman v4, ["To be succinct and simple, when running rootless containers, * * + +# Debugging Note: + - Fix for error: `Error: short-name "apache/iceberg-rest-fixture" did not resolve to an alias and no unqualified-search registries are defined in "/etc/containers/registries.conf"` + - Add or modify the `/etc/containers/registries.conf` file: + ```toml + [[registry]] + prefix = "docker.io" + location = "docker.io" + ```