Skip to content

Commit 135ef91

Browse files
rbrugaropre-commit-ci[bot]lvliang-intel
authored
Change neo4j Bolt default PORT from 7687 to $NEO4J_PORT2 (#1292)
* Change neo4j Bolt default PORT from 7687 to -configured the port in neo4j compose.yaml to use variable value -made all corresponding changes in neo4j dataprep and retriever components and test scripts to use env variable instead of default port value. Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missing positional arg in milvus dataprep Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com> * remove redundance in stop_docker Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com> * resolve retriever to neo4j connectivity issue bad URL Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * set neo4j ports to neo4j defaults and fix environment variables in READMEs Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com> --------- Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Liang Lv <liang1.lv@intel.com>
1 parent a4f6af1 commit 135ef91

File tree

8 files changed

+49
-22
lines changed

8 files changed

+49
-22
lines changed

‎comps/dataprep/src/README_neo4j_llamaindex.md‎

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -19,22 +19,30 @@ export host_ip=${your_hostname IP} # local IP
1919
export no_proxy=$no_proxy,${host_ip} # important to add {host_ip} for containers communication
2020
export http_proxy=${your_http_proxy}
2121
export https_proxy=${your_http_proxy}
22-
export NEO4J_URI=${your_neo4j_url}
2322
export NEO4J_USERNAME=${your_neo4j_username}
2423
export NEO4J_PASSWORD=${your_neo4j_password} # should match what was used in NEO4J_AUTH when running the neo4j-apoc
2524
export PYTHONPATH=${path_to_comps}
2625
export OPENAI_KEY=${your_openai_api_key} # optional, when not provided will use open models TGI/TEI
2726
export HUGGINGFACEHUB_API_TOKEN=${your_hf_token}
27+
export DATA_PATH=${host_path_to_volume_mnt}
2828

2929
# set additional environment settings
3030
export EMBEDDING_MODEL_ID="BAAI/bge-base-en-v1.5"
31+
export EMBED_MODEL=${EMBEDDING_MODEL_ID}
3132
export OPENAI_EMBEDDING_MODEL="text-embedding-3-small"
32-
export LLM_MODEL_ID="meta-llama/Meta-Llama-3-8B-Instruct"
33+
export LLM_MODEL_ID="meta-llama/Meta-Llama-3.1-8B-Instruct"
34+
export MAX_INPUT_TOKENS=4096
35+
export MAX_TOTAL_TOKENS=8192
3336
export OPENAI_LLM_MODEL="gpt-4o"
34-
export TEI_EMBEDDING_ENDPOINT="http://${host_ip}:6006"
35-
export TGI_LLM_ENDPOINT="http://${host_ip}:6005"
36-
export NEO4J_URL="bolt://${host_ip}:7687"
37-
export NEO4J_USERNAME=neo4j
37+
export TEI_EMBEDDER_PORT=11633
38+
export TEI_EMBEDDING_ENDPOINT="http://${host_ip}:${TEI_EMBEDDER_PORT}"
39+
export LLM_ENDPOINT_PORT=11634
40+
export TGI_LLM_ENDPOINT="http://${host_ip}:${LLM_ENDPOINT_PORT}"
41+
export NEO4J_AUTH="${NEO4J_USERNAME}/${NEO4J_PASSWORD}"
42+
export NEO4J_PORT1=7474 # 11631
43+
export NEO4J_PORT2=7687 # 11632
44+
export NEO4J_URI="bolt://${host_ip}:${NEO4J_PORT2}"
45+
export NEO4J_URL="bolt://${host_ip}:${NEO4J_PORT2}"
3846
export DATAPREP_SERVICE_ENDPOINT="http://${host_ip}:6004/v1/dataprep"
3947
export LOGFLAG=True
4048
```

‎comps/dataprep/src/integrations/neo4j_llamaindex.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,8 +60,9 @@
6060
from llama_index.core.schema import BaseNode, TransformComponent
6161

6262
host_ip = os.getenv("host_ip")
63+
NEO4J_PORT2 = os.getenv("NEO4J_PORT2")
6364
# Neo4J configuration
64-
NEO4J_URL = os.getenv("NEO4J_URL", f"bolt://{host_ip}:7687")
65+
NEO4J_URL = os.getenv("NEO4J_URL", f"bolt://{host_ip}:{NEO4J_PORT2}")
6566
NEO4J_USERNAME = os.getenv("NEO4J_USERNAME", "neo4j")
6667
NEO4J_PASSWORD = os.getenv("NEO4J_PASSWORD", "neo4jtest")
6768

‎comps/retrievers/deployment/docker_compose/compose.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,7 @@ services:
6767
EMBEDDING_MODEL_ID: ${EMBEDDING_MODEL_ID}
6868
LLM_MODEL_ID: ${LLM_MODEL_ID}
6969
NEO4J_URI: ${NEO4J_URI}
70+
NEO4J_URL: ${NEO4J_URI}
7071
NEO4J_USERNAME: ${NEO4J_USERNAME}
7172
NEO4J_PASSWORD: ${NEO4J_PASSWORD}
7273
VDMS_USE_CLIP: 0

‎comps/retrievers/src/README_neo4j.md‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -44,22 +44,32 @@ export host_ip=${your_hostname IP} # local IP
4444
export no_proxy=$no_proxy,${host_ip} # important to add {host_ip} for containers communication
4545
export http_proxy=${your_http_proxy}
4646
export https_proxy=${your_http_proxy}
47-
export NEO4J_URI=${your_neo4j_url}
4847
export NEO4J_USERNAME=${your_neo4j_username}
4948
export NEO4J_PASSWORD=${your_neo4j_password}
5049
export PYTHONPATH=${path_to_comps}
5150
export OPENAI_KEY=${your_openai_api_key} # optional, when not provided will use smaller models TGI/TEI
5251
export HUGGINGFACEHUB_API_TOKEN=${your_hf_token}
52+
export DATA_PATH=${host_path_to_volume_mnt}
53+
5354
# set additional environment settings
5455
export EMBEDDING_MODEL_ID="BAAI/bge-base-en-v1.5"
5556
export OPENAI_EMBEDDING_MODEL="text-embedding-3-small"
56-
export LLM_MODEL_ID="meta-llama/Meta-Llama-3-8B-Instruct"
57+
export LLM_MODEL_ID="meta-llama/Meta-Llama-3.1-8B-Instruct"
58+
export MAX_INPUT_TOKENS=4096
59+
export MAX_TOTAL_TOKENS=8192
5760
export OPENAI_LLM_MODEL="gpt-4o"
61+
export TEI_EMBEDDER_PORT=11633
62+
export TEI_EMBEDDING_ENDPOINT="http://${host_ip}:${TEI_EMBEDDER_PORT}"
63+
export LLM_ENDPOINT_PORT=11634
64+
export TGI_LLM_ENDPOINT="http://${host_ip}:${LLM_ENDPOINT_PORT}"
5865
export TEI_EMBEDDING_ENDPOINT="http://${host_ip}:6006"
5966
export TGI_LLM_ENDPOINT="http://${host_ip}:6005"
60-
export NEO4J_URL="bolt://${host_ip}:7687"
61-
export NEO4J_USERNAME=neo4j
67+
export NEO4J_PORT1=7474 # 11631
68+
export NEO4J_PORT2=7687 # 11632
69+
export NEO4J_URI="bolt://${host_ip}:${NEO4J_PORT2}"
70+
export NEO4J_URL="bolt://${host_ip}:${NEO4J_PORT2}"
6271
export DATAPREP_SERVICE_ENDPOINT="http://${host_ip}:6004/v1/dataprep"
72+
export RETRIEVER_PORT=11635
6373
export LOGFLAG=True
6474
```
6575

‎comps/retrievers/src/integrations/config.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,8 @@ def get_boolean_env_var(var_name, default_value=False):
6363
#######################################################
6464
# Neo4j #
6565
#######################################################
66-
NEO4J_URL = os.getenv("NEO4J_URI", "bolt://localhost:7687")
66+
NEO4J_PORT2 = os.getenv("NEO4J_PORT2", "11632")
67+
NEO4J_URL = os.getenv("NEO4J_URI", f"bolt://localhost:{NEO4J_PORT2}")
6768
NEO4J_USERNAME = os.getenv("NEO4J_USERNAME", "neo4j")
6869
NEO4J_PASSWORD = os.getenv("NEO4J_PASSWORD", "test")
6970
host_ip = os.getenv("host_ip")

‎comps/third_parties/neo4j/deployment/docker_compose/compose.yaml‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,10 @@ services:
99
- "${NEO4J_PORT1:-7474}:7474"
1010
- "${NEO4J_PORT2:-7687}:7687"
1111
volumes:
12-
- /$HOME/neo4j/logs:/logs
13-
- /$HOME/neo4j/config:/config
14-
- /$HOME/neo4j/data:/data
15-
- /$HOME/neo4j/plugins:/plugins
12+
- ./data/neo4j/logs:/logs
13+
- ./data/neo4j/config:/config
14+
- ./data/neo4j/data:/data
15+
- ./data/neo4j/plugins:/plugins
1616
ipc: host
1717
environment:
1818
- no_proxy=${no_proxy}
@@ -24,6 +24,7 @@ services:
2424
- NEO4J_apoc_import_file_enabled=true
2525
- NEO4J_apoc_import_file_use__neo4j__config=true
2626
- NEO4J_dbms_security_procedures_unrestricted=apoc.\*
27+
- NEO4J_server_bolt_advertised__address=localhost:${NEO4J_PORT2}
2728
restart: always
2829
healthcheck:
2930
test: wget http://localhost:7474 || exit 1

‎tests/dataprep/test_dataprep_neo4j_on_intel_hpu.sh‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,10 @@ function build_docker_images() {
2828
function start_service() {
2929
service_name="neo4j-apoc tei-embedding-serving tgi-gaudi-server dataprep-neo4j-llamaindex"
3030
export host_ip=${ip_address}
31+
export NEO4J_PORT1=7474 # 11631
32+
export NEO4J_PORT2=7687 # 11632
3133
export NEO4J_AUTH="neo4j/neo4jtest"
32-
export NEO4J_URL="bolt://${ip_address}:7687"
34+
export NEO4J_URL="bolt://${ip_address}:${NEO4J_PORT2}"
3335
export NEO4J_USERNAME="neo4j"
3436
export NEO4J_PASSWORD="neo4jtest"
3537
export NEO4J_apoc_export_file_enabled=true
@@ -95,7 +97,7 @@ function validate_service() {
9597
function validate_microservice() {
9698
# validate neo4j-apoc
9799
validate_service \
98-
"${ip_address}:7474" \
100+
"${ip_address}:${NEO4J_PORT1}" \
99101
"200 OK" \
100102
"neo4j-apoc" \
101103
"neo4j-apoc" \

‎tests/retrievers/test_retrievers_neo4j_on_intel_hpu.sh‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,10 @@ function start_service() {
5050
export LLM_MODEL_ID="meta-llama/Meta-Llama-3.1-8B-Instruct"
5151
export TGI_LLM_ENDPOINT="http://${host_ip}:${LLM_ENDPOINT_PORT}"
5252
export DATAPREP_SERVICE_ENDPOINT="http://${host_ip}:6004"
53-
export NEO4J_URI="bolt://${host_ip}:7687"
53+
export NEO4J_PORT1=7474 # 11631
54+
export NEO4J_PORT2=7687 # 11632
55+
export NEO4J_URI="bolt://${host_ip}:${NEO4J_PORT2}"
56+
export NEO4J_URL="bolt://${host_ip}:${NEO4J_PORT2}"
5457
export NEO4J_USERNAME="neo4j"
5558
export NEO4J_PASSWORD="neo4jtest"
5659
export no_proxy="localhost,127.0.0.1,"${host_ip}
@@ -64,8 +67,8 @@ function start_service() {
6467
# Not testing openai code path since not able to provide key for cicd
6568
docker run -d --name="test-comps-retrievers-neo4j-llama-index-dataprep" -p 6004:5000 -v ./data:/data --ipc=host -e TGI_LLM_ENDPOINT=$TGI_LLM_ENDPOINT \
6669
-e TEI_EMBEDDING_ENDPOINT=$TEI_EMBEDDING_ENDPOINT -e EMBEDDING_MODEL_ID=$EMBEDDING_MODEL_ID -e LLM_MODEL_ID=$LLM_MODEL_ID -e host_ip=$host_ip -e no_proxy=$no_proxy \
67-
-e http_proxy=$http_proxy -e https_proxy=$https_proxy -e NEO4J_URL="bolt://${host_ip}:7687" -e NEO4J_USERNAME="neo4j" \
68-
-e NEO4J_PASSWORD="neo4jtest" -e HF_TOKEN=$HF_TOKEN -e MAX_INPUT_TOKENS=$MAX_INPUT_TOKENS -e LOGFLAG=True \
70+
-e http_proxy=$http_proxy -e https_proxy=$https_proxy -e NEO4J_URL="bolt://${host_ip}:${NEO4J_PORT2}" -e NEO4J_USERNAME="neo4j" \
71+
-e NEO4J_PASSWORD="neo4jtest" -e NEO4J_PORT1=$NEO4J_PORT1 -e NEO4J_PORT2=$NEO4J_PORT2 -e HF_TOKEN=$HF_TOKEN -e MAX_INPUT_TOKENS=$MAX_INPUT_TOKENS -e LOGFLAG=True \
6972
-e DATAPREP_COMPONENT_NAME="OPEA_DATAPREP_NEO4J_LLAMAINDEX" opea/dataprep-neo4j-llamaindex:comps
7073

7174
sleep 1m
@@ -117,7 +120,7 @@ function validate_service() {
117120
function validate_microservice() {
118121
# validate neo4j-apoc
119122
validate_service \
120-
"${host_ip}:7474" \
123+
"${host_ip}:${NEO4J_PORT1}" \
121124
"200 OK" \
122125
"neo4j-apoc" \
123126
"neo4j-apoc" \

0 commit comments

Comments
 (0)