diff --git a/docs/content/docs/get-started/installation.md b/docs/content/docs/get-started/installation.md index 4a0c8d989..c3e1afd7f 100644 --- a/docs/content/docs/get-started/installation.md +++ b/docs/content/docs/get-started/installation.md @@ -31,7 +31,48 @@ The sections below show how to install the required dependencies. __NOTE:__ To run on a Flink cluster, Flink-Agents requires a stable release of Flink version 1.20.3 or higher. We highly recommend using the **latest stable release of your chosen Flink minor version** (e.g., for minor version 2.2, use the latest 2.2.x release). {{< /hint >}} -## Install Apache Flink +## Prerequisites + +Both the script-based and manual installation paths require: + +* **Java 11+** on your `PATH`. Java 21+ is recommended when using the Java API — see the table below for the version-specific limitations. +* **Python 3.10, 3.11, or 3.12** (only required if you plan to use the Python API or PyFlink). + +For building Flink Agents from source, you additionally need: +- Unix-like environment (Linux, macOS, Cygwin, or WSL) +- Git +- Maven 3 + +### Java Versions + +For running an agent built with the **Python API**, you can use any Java version 11 or higher. + +When using the **Java API**, there are some functionality limitations for earlier Java versions, as detailed below: + +| Java Version | Limitations | +|--------------|------------------------| +| Java 21+ | No limitations. | +| Java 11-20 | Async execution is unavailable. | + +{{< hint info >}} +**Note**: If you want to use python 3.12, you need install Flink above 2.1 (including 2.1). +{{< /hint >}} + +## Quick install (recommended) + +The `install.sh` script provisions everything you need: it downloads Apache Flink, creates a Python virtual environment, installs `flink-agents` and `apache-flink` into it, and copies the required JARs into `$FLINK_HOME/lib`. + +Run the one-liner from any directory you want the virtual environment created in: + +```shell +curl -fsSL https://raw.githubusercontent.com/apache/flink-agents/main/tools/install.sh | bash +``` + +## Manual installation + +Use this path when you want full control over each step, when running `install.sh` is not feasible in your environment, or when building Flink Agents from source. + +### Install Apache Flink Before installing Flink Agents, you need to have Apache Flink installed. @@ -56,32 +97,6 @@ cp $FLINK_HOME/opt/flink-python-${FLINK_VERSION}.jar $FLINK_HOME/lib/ **Note:** For more detailed Flink installation instructions, refer to the [Flink local installation guide](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/try-flink/local_installation/). {{< /hint >}} -{{< hint info >}} -**Note**: If you want to use python 3.12, you need install Flink above 2.1 (including 2.1). -{{< /hint >}} - -## Install Flink Agents - -### Prerequisites - -* Python 3.10, 3.11 or 3.12 -* For building from source, you also need: - - Unix-like environment (Linux, Mac OS X, Cygwin, or WSL) - - Git - - Maven 3 - - Java 21+ (full functionality), or Java 11+ (some features unavailable) - -### Java Versions - -For running an agent built with **Python API**, you can use any Java version 11 or higher. - -When using **Java API**, there are some functionality limitations for earlier Java versions, as detailed below: - -| Java Version | Limitations | -|--------------|------------------------| -| Java 21+ | No limitations. | -| Java 11-20 | Async execution is unavailable. | - ### Set Up Python Environment (Recommended) We recommend using a Python virtual environment to isolate Flink Agents dependencies from your system Python packages. @@ -158,6 +173,60 @@ After building: - The Python package is installed and ready to use - The distribution JAR is located at: `dist/flink-${FLINK_VERSION%.*}/target/flink-agents-dist-*.jar` +### Configure PYTHONPATH + +Flink runs in its own JVM process and needs the `PYTHONPATH` environment variable to locate the flink-agents Python package. You need to set `PYTHONPATH` to the directory where flink-agents is installed. + +**Determine your Python package installation path:** + +The path depends on your Python environment setup: +- If using a virtual environment, it's the site-packages directory within your venv +- If using system Python, it's the system site-packages directory + +**Tip:** You can use this command to help find the path: +```shell +python3 -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])' +``` + +**Set PYTHONPATH before starting Flink:** + +```shell +# Set PYTHONPATH to your Python site-packages directory + +export PYTHONPATH=$(python -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])') +``` + +{{< hint info >}} +**Note:** You can add the `export PYTHONPATH=...` line to your shell profile (`~/.bash_profile`, `~/.bashrc`, `~/.zprofile`, or `~/.zshrc`) to set it permanently. This way, it will be automatically configured in all future terminal sessions. +{{< /hint >}} + +### Install Flink Agents Java Library + +Copy the Flink Agents distribution JAR to your Flink installation's `lib` directory: + +{{< tabs "Install Flink Agents Java Library" >}} + +{{< tab "From Official Release" >}} +The Flink Agents JAR is bundled inside the Python package. Use the PYTHONPATH you configured above to locate and copy it: + +```shell +# Copy the JAR from the Python package to Flink's lib directory +cp $PYTHONPATH/flink_agents/lib/flink-${FLINK_VERSION%.*}/flink-agents-dist-*.jar $FLINK_HOME/lib/ +``` + +{{< /tab >}} + +{{< tab "From Source" >}} +After building from source, the distribution JAR is located in the `dist/target/` directory: + +```shell +# Copy the JAR to Flink's lib directory +cp dist/flink-${FLINK_VERSION%.*}/target/flink-agents-dist-*.jar $FLINK_HOME/lib/ +``` +{{< /tab >}} + +{{< /tabs >}} + ### Build Environment Variables The following environment variables can be used to control how JARs are resolved during `pip install flink-agents` (from sdist) or `python -m build`: @@ -229,67 +298,9 @@ For execution in IDE, enable the feature `add dependencies with provided scope t ## Deploy to Flink Cluster -After installing Flink Agents package, you need to deploy it to your Flink cluster so that Flink can run your agent jobs. - -### Configure PYTHONPATH - -Flink runs in its own JVM process and needs the `PYTHONPATH` environment variable to locate the flink-agents Python package. You need to set `PYTHONPATH` to the directory where flink-agents is installed. - -**Determine your Python package installation path:** - -The path depends on your Python environment setup: -- If using a virtual environment, it's the site-packages directory within your venv -- If using system Python, it's the system site-packages directory - -**Tip:** You can use this command to help find the path: -```shell -python3 -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])' -``` - -**Set PYTHONPATH before starting Flink:** - -```shell -# Set PYTHONPATH to your Python site-packages directory - -export PYTHONPATH=$(python -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])') -``` - -{{< hint info >}} -**Note:** You can add the `export PYTHONPATH=...` line to your shell profile (`~/.bash_profile`, `~/.bashrc`, `~/.zprofile`, or `~/.zshrc`) to set it permanently. This way, it will be automatically configured in all future terminal sessions. -{{< /hint >}} - -### Install Flink Agents Java Library - -Copy the Flink Agents distribution JAR to your Flink installation's `lib` directory: - -{{< tabs "Install Flink Agents Java Library" >}} - -{{< tab "From Official Release" >}} -The Flink Agents JAR is bundled inside the Python package. Use the PYTHONPATH you configured above to locate and copy it: - -```shell -# Copy the JAR from the Python package to Flink's lib directory -cp $PYTHONPATH/flink_agents/lib/flink-${FLINK_VERSION%.*}/flink-agents-dist-*.jar $FLINK_HOME/lib/ -``` - -{{< /tab >}} - -{{< tab "From Source" >}} -After building from source, the distribution JAR is located in the `dist/target/` directory: - -```shell -# Copy the JAR to Flink's lib directory -cp dist/flink-${FLINK_VERSION%.*}/target/flink-agents-dist-*.jar $FLINK_HOME/lib/ -``` -{{< /tab >}} - -{{< /tabs >}} - -### Start Flink Cluster or Submit Job - Once the Java library is installed and PYTHONPATH is configured, you can start your Flink cluster or submit jobs: -{{< tabs "Start Flink Cluster or Submit Job" >}} +{{< tabs "Deploy to Flink Cluster" >}} {{< tab "Python" >}} ```bash diff --git a/tools/install.sh b/tools/install.sh index c71e3fa31..f43499c05 100755 --- a/tools/install.sh +++ b/tools/install.sh @@ -473,7 +473,6 @@ PYTHON_BIN="${PYTHON_BIN:-}" NO_PROMPT="${NO_PROMPT:-0}" VERBOSE="${FLINK_AGENTS_VERBOSE:-0}" DRY_RUN="${FLINK_AGENTS_DRY_RUN:-0}" -VERIFY_INSTALL="${FLINK_AGENTS_VERIFY_INSTALL:-0}" HELP=0 PYFLINK_ACTUALLY_ENABLED=0 @@ -491,7 +490,6 @@ Options: --enable-pyflink Enable PyFlink and install Python packages --verbose Print debug output (set -x) --dry-run Print install plan without making changes - --verify Run post-install verification checks --python Path to a Python3 interpreter (overrides PATH lookup) --flink-version Apache Flink version (e.g. 2.2.0); overrides the interactive picker --flink-agents-version @@ -513,7 +511,6 @@ Environment variables: NO_PROMPT 1 to disable all prompts FLINK_AGENTS_VERBOSE 1 to enable verbose output FLINK_AGENTS_DRY_RUN 1 to enable dry-run mode - FLINK_AGENTS_VERIFY_INSTALL 1 to enable post-install verification Examples: bash install.sh --install-flink --enable-pyflink --non-interactive @@ -1670,17 +1667,9 @@ show_install_plan() { if [[ "$DRY_RUN" == "1" ]]; then ui_kv "Dry run" "yes" fi - if [[ "$VERIFY_INSTALL" == "1" ]]; then - ui_kv "Verify" "yes" - fi } verify_installation() { - if [[ "${VERIFY_INSTALL}" != "1" ]]; then - return 0 - fi - - ui_stage "Verifying installation" if [[ -x "$FLINK_HOME/bin/flink" ]]; then local flink_ver @@ -1703,7 +1692,7 @@ verify_installation() { fi if [[ "$PYFLINK_ACTUALLY_ENABLED" -eq 1 ]]; then - if python -c "import flink_agents; print('flink-agents', flink_agents.__version__)" 2>/dev/null; then + if "$VENV_DIR/bin/python3" -c "import flink_agents; print('flink-agents ready')" 2>/dev/null; then ui_success "flink-agents Python package verified" else ui_error "flink-agents Python package import failed" @@ -1720,6 +1709,24 @@ verify_installation() { ui_success "Verification complete" } +show_next_step(){ + echo "" + ui_celebrate "Apache Flink Agents installation finished!" + echo "" + ui_section "Next steps" + ui_success "1) Point FLINK_HOME at this install (Flink CLI and clients read it):" + ui_info " export FLINK_HOME=${FLINK_HOME}" + if [[ "$PYFLINK_ACTUALLY_ENABLED" -eq 1 ]]; then + ui_success "2) Activate the Python venv (PyFlink + flink-agents are installed there):" + ui_info " source ${VENV_DIR}/bin/activate" + ui_success "3) To make both permanent, append the two lines above to your shell rc" + ui_info " (~/.zshrc or ~/.bashrc)." + else + ui_success "2) To make it permanent, append the line above to your shell rc" + ui_info " (~/.zshrc or ~/.bashrc)." + fi +} + show_footer_links() { local docs_url="https://nightlies.apache.org/flink/flink-agents-docs-latest/" local issues_url="https://github.com/apache/flink-agents/issues" @@ -1751,10 +1758,6 @@ parse_args() { DRY_RUN=1 shift ;; - --verify) - VERIFY_INSTALL=1 - shift - ;; --python) if [[ $# -lt 2 ]]; then die "--python requires a path argument" @@ -1821,6 +1824,10 @@ main() { return 0 fi + if [[ "${NO_PROMPT:-0}" != "1" ]] && [[ ! -t 0 ]] && { : /dev/null; then + exec sets PYTHON_BIN" { parse_args --python /opt/py/bin/python3 [ "$PYTHON_BIN" = "/opt/py/bin/python3" ] @@ -74,9 +69,8 @@ setup() { } @test "parse_args: combined flags all apply" { - parse_args --non-interactive --install-flink --enable-pyflink --verify + parse_args --non-interactive --install-flink --enable-pyflink [ "$NO_PROMPT" = "1" ] [ "$INSTALL_FLINK" = "Yes" ] [ "$ENABLE_PYFLINK" = "Yes" ] - [ "$VERIFY_INSTALL" = "1" ] }