diff --git a/docs/content.zh/docs/deployment/filesystems/gcs.md b/docs/content.zh/docs/deployment/filesystems/gcs.md index 0b971b24747ba..929400f39deec 100644 --- a/docs/content.zh/docs/deployment/filesystems/gcs.md +++ b/docs/content.zh/docs/deployment/filesystems/gcs.md @@ -71,7 +71,7 @@ Note that these examples are *not* exhaustive and you can use GCS in other place Flink provides the `flink-gs-fs-hadoop` file system to write to GCS. This implementation is self-contained with no dependency footprint, so there is no need to add Hadoop to the classpath to use it. -`flink-gs-fs-hadoop` registers a `FileSystem` wrapper for URIs with the *gs://* scheme. It uses Google's [gcs-connector](https://mvnrepository.com/artifact/com.google.cloud.bigdataoss/gcs-connector/hadoop3-2.2.18) Hadoop library to access GCS. It also uses Google's [google-cloud-storage](https://mvnrepository.com/artifact/com.google.cloud/google-cloud-storage/2.68.0) library to provide `RecoverableWriter` support. +`flink-gs-fs-hadoop` registers a `FileSystem` wrapper for URIs with the *gs://* scheme. It uses Google's [gcs-connector](https://mvnrepository.com/artifact/com.google.cloud.bigdataoss/gcs-connector/3.1.17) Hadoop library to access GCS. It also uses Google's [google-cloud-storage](https://mvnrepository.com/artifact/com.google.cloud/google-cloud-storage/2.68.0) library to provide `RecoverableWriter` support. This file system can be used with the [FileSystem connector]({{< ref "docs/connectors/datastream/filesystem.md" >}}). @@ -84,7 +84,7 @@ cp ./opt/flink-gs-fs-hadoop-{{< version >}}.jar ./plugins/gs-fs-hadoop/ ### Configuration -The underlying Hadoop file system can be configured using the [Hadoop configuration keys](https://github.com/GoogleCloudDataproc/hadoop-connectors/blob/v2.2.18/gcs/CONFIGURATION.md) for `gcs-connector` by adding the configurations to your [Flink configuration file]({{< ref "docs/deployment/config#flink-配置文件" >}}). +The underlying Hadoop file system can be configured using the [Hadoop configuration keys](https://github.com/GoogleCloudDataproc/hadoop-connectors/blob/v3.1.17/gcs/CONFIGURATION.md) for `gcs-connector` by adding the configurations to your [Flink configuration file]({{< ref "docs/deployment/config#flink-配置文件" >}}). For example, `gcs-connector` has a `fs.gs.http.connect-timeout` configuration key. If you want to change it, you need to set `gs.http.connect-timeout: xyz` in [Flink configuration file]({{< ref "docs/deployment/config#flink-配置文件" >}}). Flink will internally translate this back to `fs.gs.http.connect-timeout`. diff --git a/docs/content/docs/deployment/filesystems/gcs.md b/docs/content/docs/deployment/filesystems/gcs.md index 164dbcc26156a..775c58d15ac8b 100644 --- a/docs/content/docs/deployment/filesystems/gcs.md +++ b/docs/content/docs/deployment/filesystems/gcs.md @@ -71,7 +71,7 @@ Note that these examples are *not* exhaustive and you can use GCS in other place Flink provides the `flink-gs-fs-hadoop` file system to write to GCS. This implementation is self-contained with no dependency footprint, so there is no need to add Hadoop to the classpath to use it. -`flink-gs-fs-hadoop` registers a `FileSystem` wrapper for URIs with the *gs://* scheme. It uses Google's [gcs-connector](https://mvnrepository.com/artifact/com.google.cloud.bigdataoss/gcs-connector/hadoop3-2.2.18) Hadoop library to access GCS. It also uses Google's [google-cloud-storage](https://mvnrepository.com/artifact/com.google.cloud/google-cloud-storage/2.68.0) library to provide `RecoverableWriter` support. +`flink-gs-fs-hadoop` registers a `FileSystem` wrapper for URIs with the *gs://* scheme. It uses Google's [gcs-connector](https://mvnrepository.com/artifact/com.google.cloud.bigdataoss/gcs-connector/3.1.17) Hadoop library to access GCS. It also uses Google's [google-cloud-storage](https://mvnrepository.com/artifact/com.google.cloud/google-cloud-storage/2.68.0) library to provide `RecoverableWriter` support. This file system can be used with the [FileSystem connector]({{< ref "docs/connectors/datastream/filesystem.md" >}}). @@ -84,7 +84,7 @@ cp ./opt/flink-gs-fs-hadoop-{{< version >}}.jar ./plugins/gs-fs-hadoop/ ### Configuration -The underlying Hadoop file system can be configured using the [Hadoop configuration keys](https://github.com/GoogleCloudDataproc/hadoop-connectors/blob/v2.2.18/gcs/CONFIGURATION.md) for `gcs-connector` by adding the configurations to your [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). +The underlying Hadoop file system can be configured using the [Hadoop configuration keys](https://github.com/GoogleCloudDataproc/hadoop-connectors/blob/v3.1.17/gcs/CONFIGURATION.md) for `gcs-connector` by adding the configurations to your [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). For example, `gcs-connector` has a `fs.gs.http.connect-timeout` configuration key. If you want to change it, you need to set `gs.http.connect-timeout: xyz` in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). Flink will internally translate this back to `fs.gs.http.connect-timeout`. diff --git a/flink-filesystems/flink-gs-fs-hadoop/pom.xml b/flink-filesystems/flink-gs-fs-hadoop/pom.xml index 8372a8f1b9757..54b17ca8378ff 100644 --- a/flink-filesystems/flink-gs-fs-hadoop/pom.xml +++ b/flink-filesystems/flink-gs-fs-hadoop/pom.xml @@ -35,7 +35,7 @@ under the License. 2.68.0 - hadoop3-2.2.18 + 3.1.17 0.128.7 1.81.0 diff --git a/flink-filesystems/flink-gs-fs-hadoop/src/main/resources/META-INF/NOTICE b/flink-filesystems/flink-gs-fs-hadoop/src/main/resources/META-INF/NOTICE index 5067d42c4cfd5..dafa6d4154499 100644 --- a/flink-filesystems/flink-gs-fs-hadoop/src/main/resources/META-INF/NOTICE +++ b/flink-filesystems/flink-gs-fs-hadoop/src/main/resources/META-INF/NOTICE @@ -24,10 +24,10 @@ This project bundles the following dependencies under the Apache Software Licens - com.google.apis:google-api-services-iamcredentials:v1-rev20211203-2.0.0 - com.google.apis:google-api-services-storage:v1-rev20260204-2.0.0 - com.google.auto.value:auto-value-annotations:1.11.0 -- com.google.cloud.bigdataoss:gcs-connector:hadoop3-2.2.18 -- com.google.cloud.bigdataoss:gcsio:2.2.18 -- com.google.cloud.bigdataoss:util-hadoop:hadoop3-2.2.18 -- com.google.cloud.bigdataoss:util:2.2.18 +- com.google.cloud.bigdataoss:gcs-connector:3.1.17 +- com.google.cloud.bigdataoss:gcsio:3.1.17 +- com.google.cloud.bigdataoss:util-hadoop:3.1.17 +- com.google.cloud.bigdataoss:util:3.1.17 - com.google.cloud.opentelemetry:detector-resources-support:0.33.0 - com.google.cloud.opentelemetry:exporter-metrics:0.33.0 - com.google.cloud.opentelemetry:shared-resourcemapping:0.33.0