refactor VipGirder + fix tests

doc/history.md

@@ -22,12 +22,12 @@
 
 ### June 2023
 
-- Class [`VipLauncher`](#viplauncher) is introduced for specific user needs, as a parent of `VipSession` & `VipCI`;
+- Class [`VipLauncher`](#viplauncher) is introduced for specific user needs, as a parent of `VipSession` & `VipGirder`;
 - Session properties (`session_name`, `pipeline_id`, *etc.*) can be safely accessed and modified in all "`Vip*`" classes;
 - A list of available pipelines and detailed informations about each pipeline can be displayed through new class method `show_pipeline()`;
 
 ### April 2023
-- Class [`VipCI`](#vipci) to interacts with Girder datasets (tailored for CI tests in the ReproVIP project).
+- Class [`VipGirder`](#VipGirder) to interacts with Girder datasets (tailored for CI tests in the ReproVIP project).
 
 ### March 2023
 - Class [`VipSession`](#vipsession): user-friendly interface to run VIP jobs on local datasets.

doc/source.md

@@ -12,7 +12,7 @@ The most user-friendly class to interact with VIP. See the documentation [here](
 
 A parent class of `VipSession` that implements everything needed to launch VIP applications on remote data sets. *More information to come*.
 
-### [vip_client.classes.**VipCI**](../src/vip_client/classes/VipCI.py)
+### [vip_client.classes.**VipGirder**](../src/vip_client/classes/VipGirder.py)
 
 [Prototype] A `VipLauncher` implementation to launch VIP application on [Girder](https://girder.readthedocs.io/en/latest/) datasets. Currently used for continuous integration (CI) tests on the VIP platform.
 

doc/vipsession.md

@@ -169,7 +169,10 @@ When all properties are set, the full *upload-run-download* process ([steps 2-5]
 ```python
 session.run_session()
 ```
-*Do not forget to remove your temporary data from VIP after downloading the outputs (`session.finish()`).*
+*Do not forget to remove your temporary data from VIP after downloading the outputs (`session.finish(keep_output=False)`).*
+
+> [!NOTE]
+> By default `session.finish()` will remove the outputs from the VIP platform, but if you want to keep them you use the option `keep_output=True`
 
 All `VipSession` methods can be run in cascade, so everything holds in a single command:
 ```python

examples/tutorials/demo-vipsession.ipynb

@@ -499,6 +499,84 @@
     "! tree {new_session.output_dir}"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Re-use a same dataset multiples times\n",
+    "You can re-use the same dataset multiples times. For that you will only need to keep the trace of where the inputs where uploaded on VIP.  \n",
+    "You will also need to not run `finish()` at the end of the session which has uploaded the dataset, it will prevent your dataset from being deleted.  \n",
+    "When you will reuse the data don't forget to adapt the paths in `inputs_settings` by using the path where they were initially uploaded."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "session = VipSession(\"session-A\")\n",
+    "session.upload_inputs(input_dir)\n",
+    "\n",
+    "inputs_settings = {\n",
+    "    \"file\": \"initial.file\",\n",
+    "    \"value\": 5\n",
+    "}\n",
+    "\n",
+    "input_folder_on_vip = session._vip_input_dir\n",
+    "print(f\"The dataset is located on VIP here: {input_folder_on_vip}\") # keep this information somewhere\n",
+    "\n",
+    "session.launch_pipeline(pipeline_id, input_settings)\n",
+    "session.monitor_workflows()\n",
+    "# no finish for the first session"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "session = VipSession(\"session-B\")\n",
+    "# do not forget to prepend your inputs !\n",
+    "adapted_inputs_settings = {\n",
+    "    \"file\" : f\"{input_folder_on_vip}/initial.file\", # reuse the stored information !\n",
+    "    \"value\": 5\n",
+    "}\n",
+    "session.launch_pipeline(pipeline_id, adapted_inputs_settings)\n",
+    "session.monitor_workflows()\n",
+    "session.finish()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "> [!NOTE]\n",
+    "> At the very end when you won't need the dataset anymore don't forget to run `VipSession(session-A).finish()` for cleaning the data from VIP servers.\n",
+    "> You must name your session like you named it for uploading your dataset."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "upload_session = VipSession(\"upload-session\")\n",
+    "upload_session.upload_inputs(input_dir)\n",
+    "# * running the session * #\n",
+    "\n",
+    "reuse_session_a = VipSession(\"reuse-session_a\")\n",
+    "# * reunning the session on the previous dataset * #\n",
+    "\n",
+    "reuse_session_b = VipSession(\"reuse-session_b\")\n",
+    "# * reunning the session on the previous dataset * #\n",
+    "\n",
+    "# finally deleting the dataset\n",
+    "VipSession(\"upload-session\").finish()"
+   ]
+  },
   {
    "attachments": {},
    "cell_type": "markdown",

examples/tutorials/example-VipGirder.ipynb

@@ -0,0 +1,210 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e4b2f485-cb8b-4e4b-b21c-c381760fc914",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%load_ext autoreload"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9a2a0f64-107e-47a3-b994-4ba55735bfa8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%autoreload 2"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f72d24fb-f9f4-45b2-b7bc-0533216932ea",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import vip_client\n",
+    "import importlib\n",
+    "from vip_client import VipGirder\n",
+    "#importlib.reload(client)\n",
+    "vip_client.__path__"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8a60aa4a-ba39-4c9b-b749-a79b8f6d21e7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import inspect\n",
+    "inspect.getfile(VipGirder)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "fc36a278-550b-43f5-800d-3500612ba12d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "VipGirder.init()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0a7c7ee2-2656-4585-9930-895e68cc90ac",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "session.display()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "97d7cf93-2ab0-4d07-81b2-654b38480f9a",
+   "metadata": {},
+   "source": [
+    "# Output on girder (default)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cdaa53b9-eaec-44d7-a342-7b41191fcb4a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "pipeline_id=\"BasicGrepLocal/0.2\"\n",
+    "VipGirder.show_pipeline(pipeline_id)\n",
+    "session_name=\"test_girder_girder\"\n",
+    "output_dir=\"/collection/ReproVIPSpectro/test/vip_outputs\"\n",
+    "input_settings={\n",
+    "    \"file\":\"/collection/ReproVIPSpectro/test/test_for_grep.txt\",\n",
+    "    \"int\":5,\n",
+    "    \"text\":\"grep\"\n",
+    "}\n",
+    "session = VipGirder(session_name=session_name, pipeline_id=pipeline_id, input_settings=input_settings, output_dir=output_dir)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1aec7d5d-05e1-4baf-af30-2e9b42e791f1",
+   "metadata": {},
+   "source": [
+    "# Output on VIP"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "25da2f62-e044-4ffd-855f-74881ad5d770",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "pipeline_id=\"BasicGrepLocal/0.2\"\n",
+    "VipGirder.show_pipeline(pipeline_id)\n",
+    "session_name=\"test_girder_vip\"\n",
+    "input_settings={\n",
+    "    \"file\":\"/collection/ReproVIPSpectro/test/test_for_grep.txt\",\n",
+    "    \"int\":5,\n",
+    "    \"text\":\"grep\"\n",
+    "}\n",
+    "session = VipGirder(session_name=session_name, pipeline_id=pipeline_id, input_settings=input_settings, output_location=\"vip\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7c726378-93b0-443d-9531-89b42735142d",
+   "metadata": {},
+   "source": [
+    "# Output in local"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "bff17ffe-bebc-419d-b68d-ed3b0d2f4cf7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "pipeline_id=\"BasicGrepLocal/0.2\"\n",
+    "VipGirder.show_pipeline(pipeline_id)\n",
+    "session_name=\"test_girder_local\"\n",
+    "input_settings={\n",
+    "    \"file\":\"/collection/ReproVIPSpectro/test/test_for_grep.txt\",\n",
+    "    \"int\":5,\n",
+    "    \"text\":\"grep\"\n",
+    "}\n",
+    "session = VipGirder(session_name=session_name, pipeline_id=pipeline_id, input_settings=input_settings, output_location=\"local\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d47ee6da-1178-4646-b9f6-a71711dcac01",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "session.launch_pipeline()\n",
+    "session.display()\n",
+    "session.monitor_workflows()\n",
+    "session.display()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "81f8dc9e-22ab-4603-ae3b-28d83f0a2615",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "session.download_outputs()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d54e5e44-7350-4e79-a8aa-0a8a58c60fac",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "session.finish()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "39544d26-4881-4562-9d65-209c61836b33",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.13.3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

examples/tutorials/exemple_VipCI.ipynb

@@ -11,12 +11,12 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
     "# Import the class\n",
-    "from vip_client.classes import VipCI\n",
+    "from vip_client.classes import VipGirder\n",
     "import time\n",
     "\n",
     "# Pipeline identifier\n",
@@ -66,7 +66,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": null,
    "metadata": {},
    "outputs": [
     {
@@ -131,13 +131,13 @@
    ],
    "source": [
     "# Connect with Vip & Girder\n",
-    "VipCI.init(\n",
+    "VipGirder.init(\n",
     "  vip_key=\"VIP_API_KEY\", # My environment variable for the VIP API key (also works with litteral string or file name)\n",
     "  girder_key=\"GIRDER_API_KEY\" # My environment variable for the Girder API key (also works with litteral string or file name)\n",
     ")\n",
     "\n",
     "# Create a Session\n",
-    "session = VipCI(\n",
+    "session = VipGirder(\n",
     "  session_name=\"Test_Girder\", # Session Name\n",
     ")\n",
     "\n",
@@ -170,7 +170,7 @@
    "source": [
     "__N.B.__: The last cell could also be written with a single line of code:\n",
     "```python\n",
-    "VipCI.init(\n",
+    "VipGirder.init(\n",
     "    vip_key=\"VIP_API_KEY\", # My environment variable for the VIP API key \n",
     "    girder_key=\"GIRDER_API_KEY\", # My environment variable for the Girder API key\n",
     "    session_name = \"Test_Girder\", # Session Name\n",
@@ -194,7 +194,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": null,
    "metadata": {},
    "outputs": [
     {
@@ -260,7 +260,7 @@
     }
    ],
    "source": [
-    "VipCI(my_output_dir).run_session(nb_runs=2)"
+    "VipGirder(my_output_dir).run_session(nb_runs=2)"
    ]
   },
   {