Merge pull request #251 from oracle/248-mcp-langchain-export

fix LangChain MCP export

Author: corradodebari

src/client/content/config/tabs/settings.py

@@ -346,7 +346,7 @@ def display_settings():
     if spring_ai_conf == "hybrid":
         st.markdown(f"""
             The current configuration combination of embedding and language models
-            is currently **not supported** for SpringAI.
+            is currently **not supported** for Spring AI and LangChain MCP templates.
             - Language Model:  **{ll_config.get("model", "Unset")}**
             - Embedding Model: **{embed_config.get("model", "Unset")}**
         """)

src/client/mcp/rag/README.md

@@ -16,7 +16,7 @@ You need:
 - Claude Desktop free
 
 ## Setup
-With **[`uv`](https://docs.astral.sh/uv/getting-started/installation/)** installed, run the following commands in your current project directory `<PROJECT_DIR>/src/client/mcp/rag/`:
+With **[`uv`](https://docs.astral.sh/uv/getting-started/installation/)** installed, run the following commands in your current project directory `<PROJECT_DIR>`:
 
 ```bash
 uv init --python=3.11 --no-workspace
@@ -26,61 +26,52 @@ uv add mcp langchain-core==0.3.52 oracledb~=3.1 langchain-community==0.3.21 lang
 ```
 
 ## Export config
-In the **AI Optimizer & Toolkit** web interface, after tested a configuration, in `Settings/Client Settings`:
+In the **AI Optimizer & Toolkit** web interface, after tested a configuration, in `Configuration/Settings/Client Settings`: 
 
 ![Client Settings](./images/export.png)
 
-* select the checkbox `Include Sensitive Settings` 
-* press button `Download Settings` to download configuration in the project directory: `src/client/mcp/rag` as `optimizer_settings.json`.
-* in `<PROJECT_DIR>/src/client/mcp/rag/rag_base_optimizer_config_mcp.py` change filepath with the absolute path of your `optimizer_settings.json` file.
+* select the checkbox `Include Sensitive Settings`.
+* press button `Download LangchainMCP` to download an VectorSearch MCP Agent built on current configuration.
+* unzip the file in a `<PROJECT_DIR>` dir.
 
 
 ## Standalone client
-There is a client that you can run without MCP via commandline to test it:
+There is a client that you can run without MCP via command-line to test it:
 
 ```bash
 uv run rag_base_optimizer_config.py "[YOUR_QUESTION]"
 ```
+In `rag_base_optimizer_config_mcp.py`:
 
-## Quick test via MCP "inspector"
+## Claude Desktop setup
 
-* Run the inspector:
+Claude Desktop, in free version, not allows to connect remote server. You can overcome, for testing purpose only, with a proxy library called `mcp-remote`. These are the options.
+If you have already installed Node.js v20.17.0+, it should work.
 
-```bash
-npx @modelcontextprotocol/inspector uv run rag_base_optimizer_config_mcp.py
-```
+* In **Claude Desktop** application, in `Settings/Developer/Edit Config`, get the `claude_desktop_config.json` to
 
-* connect to the port `http://localhost:6274/` with your browser on the link printed, like in the following example:
-```bash
-..
-Open inspector with token pre-filled:
-http://localhost:6274/?MCP_PROXY_AUTH_TOKEN=cb2ef7521aaf2050ad9620bfb5e5df42dc958889e6e99ce4e9b18003eb93fffd
-..
-```
-
-* setup the `Inspector Proxy Address` with `http://127.0.0.1:6277` 
-* test the tool developed.
+	* Set **remote sse** execution: 
 
+	  add the references to the local MCP server for RAG in the `<PROJECT_DIR>`:
+	```json
+	{
+ 	"mcpServers": {
+		...
+		,
+		"rag":{
+			"command": "npx",
+			"args": [
+				"mcp-remote",
+				"http://127.0.0.1:9090/sse"
+				]
+			}
+   		}
+	}
+	```
 
-## Claude Desktop setup
 
-* In **Claude Desktop** application, in `Settings/Developer/Edit Config`, get the `claude_desktop_config.json` to add the references to the local MCP server for RAG in the `<PROJECT_DIR>/src/client/mcp/rag/`:
-```json
-{
- "mcpServers": {
-	...
-	,
-	"rag":{
-		"command":"bash",
-		"args":[
-			"-c",
-			"source <PROJECT_DIR>/src/client/mcp/rag/.venv/bin/activate && uv run <PROJECT_DIR>/src/client/mcp/rag/rag_base_optimizer_config_mcp.py"
-		]
-	}
-   }
-}
-```
 * In **Claude Desktop** application, in `Settings/General/Claude Settings/Configure`, under `Profile` tab, update fields like:
+
 - `Full Name`
 - `What should we call you`
 
@@ -94,8 +85,10 @@ Show the rag_tool message as-is, without modification.
 ```
 This will impose the usage of `rag_tool` in any case. 
 
-**NOTICE**: If you prefer, in this agent dashboard or any other, you could setup a message in the conversation with the same content of `Instruction` to enforce the LLM to use the rag tool as well.
-
+* Start MCP server in another shell in <PROJECT_DIR> with:
+```bash
+uv run rag_base_optimizer_config_mcp.py
+```
 * Restart **Claude Desktop**.
 
 * You will see two warnings on rag_tool configuration: they will disappear and will not cause any issue in activating the tool.
@@ -106,38 +99,45 @@ This will impose the usage of `rag_tool` in any case.
 
  If the question is related to the knowledge base content stored in the vector store, you will have an answer based on that information. Otherwise, it will try to answer considering information on which has been trained the LLM o other tools configured in the same Claude Desktop.
 
+* **Optional**: for a **local stdio** execution, without launching the MCP Server:
+
+	* Add the references to the local MCP server for RAG in the `<PROJECT_DIR>`:
+	```json
+	{
+ 	"mcpServers": {
+		...
+		,
+		"rag":{
+			"command":"bash",
+			"args":[
+				"-c",
+				"source <PROJECT_DIR>/.venv/bin/activate && uv run <PROJECT_DIR>/rag_base_optimizer_config_mcp.py"
+				]
+			}
+   		}
+	}
+	```
+	* Set `Local` with `Remote client` line in `<PROJECT_DIR>/rag_base_optimizer_config_mcp.py`:
 
-## Make a remote MCP server the RAG Tool
-
-In `rag_base_optimizer_config_mcp.py`:
-
-* Update the absolute path of your `optimizer_settings.json`. Example:
-
-```python
-rag.set_optimizer_settings_path("/Users/cdebari/Documents/GitHub/ai-optimizer-mcp-export/src/client/mcp/rag/optimizer_settings.json")
-```
+	```python
+	#mcp = FastMCP("rag", port=9090) #Remote client
+	mcp = FastMCP("rag") #Local
+	```
 
-* Substitute `Local` with `Remote client` line:
+	* Substitute `stdio` with `sse` line of code:
+	```python
+	mcp.run(transport='stdio')
+	#mcp.run(transport='sse')
+	```
 
-```python
-#mcp = FastMCP("rag", port=9001) #Remote client
-mcp = FastMCP("rag") #Local
-```
 
-* Substitute `stdio` with `sse` line of code:
-```python
-mcp.run(transport='stdio')
-#mcp.run(transport='sse')
-```
+## Alternative way for a quick test: MCP "inspector"
 
-* Start MCP server in another shell with:
+* Start MCP server in another shell in <PROJECT_DIR> with:
 ```bash
 uv run rag_base_optimizer_config_mcp.py
 ```
 
-
-## Quick test
-
 * Run the inspector:
 
 ```bash
@@ -148,38 +148,52 @@ npx @modelcontextprotocol/inspector
 
 * set the Transport Type to `SSE`
 
-* set the `URL` to `http://localhost:9001/sse`
+* set the `URL` to `http://localhost:9090/sse`
 
 * test the tool developed.
 
 
 
-## Claude Desktop setup for remote/local server
-Claude Desktop, in free version, not allows to connect remote server. You can overcome, for testing purpose only, with a proxy library called `mcp-remote`. These are the options.
-If you have already installed Node.js v20.17.0+, it should work:
+**Optional:** run with local **stdio** protocol
+* Set as shown before the protolo to run locally in `<PROJECT_DIR>/rag_base_optimizer_config_mcp.py`:
 
-* replace `rag` mcpServer, setting in `claude_desktop_config.json`:
-```json
-{
-  "mcpServers": {
-    "remote": {
-			"command": "npx",
-			"args": [
-				"mcp-remote",
-				"http://127.0.0.1:9001/sse"
-			]
-		}
-  }
-}
+	```
+	* Set `Local` with `Remote client` line:
+
+	```python
+	#mcp = FastMCP("rag", port=9090) #Remote client
+	mcp = FastMCP("rag") #Local
+	```
+
+	* Substitute `stdio` with `sse` line of code:
+	```python
+	mcp.run(transport='stdio')
+	#mcp.run(transport='sse')
+	```
+
+* Run the inspector:
+
+```bash
+npx @modelcontextprotocol/inspector uv run rag_base_optimizer_config_mcp.py
+```
+
+* connect to the port `http://localhost:6274/` with your browser on the link printed, like in the following example:
+```bash
+..
+Open inspector with token pre-filled:
+http://localhost:6274/?MCP_PROXY_AUTH_TOKEN=cb2ef7521aaf2050ad9620bfb5e5df42dc958889e6e99ce4e9b18003eb93fffd
+..
 ```
-* restart Claude Desktop. 
+
+* setup the `Inspector Proxy Address` with `http://127.0.0.1:6277` 
+* test the tool developed.
+
+
 
 **NOTICE**: If you have any problem running, check the logs if it's related to an old npx/nodejs version used with mcp-remote library. Check with:
 ```bash
 nvm -list
 ```
 if you have any other versions available than the default. It could happen that Claude Desktop uses the older one. Try to remove any other nvm versions available to force the use the only one avalable, at minimum v20.17.0+.
 
-* restart and test as remote server
-
-
+* restart and test as remote server

src/client/mcp/rag/images/export.png

@@ -16,65 +16,91 @@
 import oracledb
 
 import logging
-
-logging.basicConfig(level=logging.INFO)
-
+logger = logging.getLogger(__name__)
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(name)s - %(levelname)s - %(message)s"
+)
 
 def get_llm(data):
-    logging.info("llm data:")
-    logging.info(data["user_settings"]["ll_model"]["model"])
+    logger.info("llm data:")
+    logger.info(data["client_settings"]["ll_model"]["model"])
     llm = {}
-    llm_config = data["ll_model_config"][data["user_settings"]["ll_model"]["model"]]
+    models_by_id = {m["id"]: m for m in data.get("model_configs", [])}
+    llm_config= models_by_id.get(data["client_settings"]["ll_model"]["model"])
+    logger.info(llm_config)
     provider = llm_config["provider"]
     url = llm_config["url"]
     api_key = llm_config["api_key"]
-    model = data["user_settings"]["ll_model"]["model"]
-    logging.info(f"CHAT_MODEL: {model} {provider} {url} {api_key}")
+    model = data["client_settings"]["ll_model"]["model"]
+    logger.info(f"CHAT_MODEL: {model} {provider} {url} {api_key}")
     if provider == "ollama":
         # Initialize the LLM
         llm = OllamaLLM(model=model, base_url=url)
+        logger.info("Ollama LLM created")
     elif provider == "openai":
         llm = llm = ChatOpenAI(model=model, api_key=api_key)
+        logger.info("OpenAI LLM created")
     return llm
 
 
 def get_embeddings(data):
     embeddings = {}
-    model = data["user_settings"]["vector_search"]["model"]
-    provider = data["embed_model_config"][model]["provider"]
-    url = data["embed_model_config"][model]["url"]
-    api_key = data["embed_model_config"][model]["api_key"]
-    logging.info(f"EMBEDDINGS: {model} {provider} {url} {api_key}")
+    logger.info("getting embeddings..")
+    model = data["client_settings"]["vector_search"]["model"]
+    logger.info(f"embedding model: {model}")
+    models_by_id = {m["id"]: m for m in data.get("model_configs", [])}
+    model_params= models_by_id.get(model)
+    provider = model_params["provider"]
+    url = model_params["url"]
+    api_key = model_params["api_key"]
+
+    logger.info(f"Embeddings Model: {model} {provider} {url} {api_key}")
     embeddings = {}
     if provider == "ollama":
         embeddings = OllamaEmbeddings(model=model, base_url=url)
-    elif provider == "openai":
-        logging.info("BEFORE create embbedding")
+        logger.info("Ollama Embeddings connection successful")
+    elif (provider == "openai") or (provider == "openai_compatible"):
         embeddings = OpenAIEmbeddings(model=model, api_key=api_key)
-        logging.info("AFTER create emebdding")
+        logger.info("OpenAI embeddings connection successful")
     return embeddings
 
 
 def get_vectorstore(data, embeddings):
-    config = data["database_config"][data["user_settings"]["database"]["alias"]]
-    logging.info(config)
+    db_alias=data["client_settings"]["database"]["alias"]
+
 
-    conn23c = oracledb.connect(user=config["user"], password=config["password"], dsn=config["dsn"])
+    db_by_name = {m["name"]: m for m in data.get("database_configs", [])}
+    db_config= db_by_name.get(db_alias)
+ 
+    table_alias=data["client_settings"]["vector_search"]["alias"]
+    model=data["client_settings"]["vector_search"]["model"]
+    chunk_size=str(data["client_settings"]["vector_search"]["chunk_size"])
+    chunk_overlap=str(data["client_settings"]["vector_search"]["chunk_overlap"])
+    distance_metric=data["client_settings"]["vector_search"]["distance_metric"]
+    index_type=data["client_settings"]["vector_search"]["index_type"]
 
-    logging.info("DB Connection successful!")
-    metric = data["user_settings"]["vector_search"]["distance_metric"]
+    db_table=(table_alias+"_"+model+"_"+chunk_size+"_"+chunk_overlap+"_"+distance_metric+"_"+index_type).upper().replace("-", "_")
+    logger.info(f"db_table:{db_table}")
+
+
+    user=db_config["user"]
+    password=db_config["password"]
+    dsn=db_config["dsn"]
+   
+    logger.info(f"{db_table}: {user}/{password} - {dsn}")
+    conn23c = oracledb.connect(user=user, password=password, dsn=dsn)
+
+    logger.info("DB Connection successful!")
+    metric = data["client_settings"]["vector_search"]["distance_metric"]
 
     dist_strategy = DistanceStrategy.COSINE
     if metric == "COSINE":
         dist_strategy = DistanceStrategy.COSINE
     elif metric == "EUCLIDEAN":
         dist_strategy = DistanceStrategy.EUCLIDEAN
-
-    a = data["user_settings"]["vector_search"]["vector_store"]
-    logging.info(f"{a}")
-    logging.info(f"BEFORE KNOWLEDGE BASE")
-    logging.info(embeddings)
-    knowledge_base = OracleVS(
-        conn23c, embeddings, data["user_settings"]["vector_search"]["vector_store"], dist_strategy
-    )
+    
+    logger.info(embeddings)
+    knowledge_base = OracleVS(client=conn23c,table_name=db_table, embedding_function=embeddings, distance_strategy=dist_strategy)
+    
     return knowledge_base