Merge pull request #156 from oracle-samples/112-selectai-integration

SelectAI Tool initial commit

Author: ddrechse

.gitignore

@@ -45,6 +45,7 @@ __pycache__/
 **/**.tfvars
 **/.terraform*
 **/terraform.tfstate*
+**/*.pem
 opentofu/**/generated/*.*
 opentofu/**/generated/kubeconfig
 

README.md

@@ -4,7 +4,7 @@
 
 ## Description
 
-The **Oracle AI Optimizer and Toolkit** (the **AI Optimizer**) provides a streamlined environment where developers and data scientists can explore the potential of Generative Artificial Intelligence (GenAI) combined with Retrieval-Augmented Generation (RAG) capabilities. By integrating **Oracle Database 23ai** AI Vector Search, the Sandbox enables users to enhance existing Large Language Models (LLMs) through RAG.
+The **Oracle AI Optimizer and Toolkit** (the **AI Optimizer**) provides a streamlined environment where developers and data scientists can explore the potential of Generative Artificial Intelligence (GenAI) combined with Retrieval-Augmented Generation (RAG) capabilities. By integrating **Oracle Database 23ai** AI VectorSearch and SelectAI, the Sandbox enables users to enhance existing Large Language Models (LLMs) through RAG.
 
 ## AI Optimizer Features
 

docs/content/_index.md

@@ -13,7 +13,7 @@ Licensed under the Universal Permissive License v1.0 as shown at http://oss.orac
 spell-checker:ignore streamlit, genai, venv
 -->
 
-The {{< full_app_ref >}} provides a streamlined environment where developers and data scientists can explore the potential of Generative Artificial Intelligence (**GenAI**) combined with Retrieval-Augmented Generation (**RAG**) capabilities. By integrating Oracle Database AI Vector Search, the {{< short_app_ref >}} enables users to enhance existing Large Language Models (**LLM**s) through **RAG**. This method significantly improves the performance and accuracy of AI models, helping to avoid common issues such as knowledge cutoff and hallucinations.
+The {{< full_app_ref >}} provides a streamlined environment where developers and data scientists can explore the potential of Generative Artificial Intelligence (**GenAI**) combined with Retrieval-Augmented Generation (**RAG**) capabilities. By integrating Oracle Database AI VectorSearch and SelectAI, the {{< short_app_ref >}} enables users to enhance existing Large Language Models (**LLM**s) through **RAG**. This method significantly improves the performance and accuracy of AI models, helping to avoid common issues such as knowledge cutoff and hallucinations.
 
 - **GenAI**: Powers the generation of text, images, or other data based on prompts using pre-trained **LLM**s.
 - **RAG**: Enhances **LLM**s by retrieving relevant, real-time information allowing models to provide up-to-date and accurate responses.

docs/content/client/configuration/db_config.md

@@ -80,7 +80,7 @@ For container installations, there are a couple of ways to include the contents
 
 ## Database User
 
-A database user is required to store the embeddings, used for **RAG**, into the Oracle Database. A non-privileged user with a *non-SYSTEM tablespace* should be used for this purpose.  Use the below syntax as an example of creating a new user:
+A database user is required to store the embeddings, used for **RAG**, into the Oracle Database. A non-privileged user with a *non-SYSTEM tablespace* should be used for this purpose.  Use the below syntax as an __example__ of creating a new user with least privileges:
 
 ```sql
 CREATE USER "DEMO" IDENTIFIED BY MYCOMPLEXSECRET
@@ -91,7 +91,13 @@ ALTER USER "DEMO" DEFAULT ROLE ALL;
 ALTER USER "DEMO" QUOTA UNLIMITED ON DATA;
 ```
 
-Replace "DEMO" as required.
+If running on a supported database for [SelectAI](https://docs.oracle.com/en-us/iaas/autonomous-database-serverless/doc/select-ai.html) and want to use the feature, grant the following additional privileges and open appropriate ACLs:
+
+```sql
+GRANT EXECUTE ON DBMS_CLOUD TO DEMO;
+GRANT EXECUTE ON DBMS_CLOUD_AI TO DEMO;
+GRANT EXECUTE ON DBMS_CLOUD_PIPELINE TO DEMO;
+```
 
 {{% notice style="default" title="One schema fits none..." icon="circle-info" %}}
 Creating multiple users in the same database allows developers to separate their experiments simply by changing the "Database User"

helm/charts/server/templates/deployment.yaml

@@ -164,10 +164,10 @@ spec:
       volumes:
         - name: tmp
           emptyDir: {}
-        {{- if .Values.adb.enabled }}
+        {{- if and .Values.adb .Values.adb.enabled }}
         - name: tns-admin
           secret:
-            {{- if .Values.adb.tnsAdmin.secretName }}
+            {{- if and .Values.adb.tnsAdmin .Values.adb.tnsAdmin.secretName }}
             secretName: {{ .Values.adb.tnsAdmin.secretName | quote }}
             {{ else }}
             secretName: {{ include "app.fullname" . }}-adb-tns-admin-{{ .Release.Revision }}

opentofu/iam.tf

@@ -0,0 +1,38 @@
+# Copyright (c) 2024, 2025, Oracle and/or its affiliates.
+# All rights reserved. The Universal Permissive License (UPL), Version 1.0 as shown at http://oss.oracle.com/licenses/upl
+# spell-checker: disable
+
+resource "oci_identity_tag_namespace" "tag_namespace" {
+  compartment_id = local.compartment_ocid
+  description    = format("%s Tag Namespace", local.label_prefix)
+  name           = local.label_prefix
+  provider       = oci.home_region
+}
+
+resource "oci_identity_tag" "identity_tag" {
+  description      = format("%s Infrastructure", local.label_prefix)
+  name             = "infrastructure"
+  tag_namespace_id = oci_identity_tag_namespace.tag_namespace.id
+  provider         = oci.home_region
+}
+
+resource "oci_identity_dynamic_group" "resource_dynamic_group" {
+  compartment_id = var.tenancy_ocid
+  name           = format("%s-dyngrp", local.label_prefix)
+  description    = format("%s Dynamic Group", local.label_prefix)
+  matching_rule = format(
+    "All {resource.compartment.id = '%s', tag.%s.value = '%s'}",
+    local.compartment_ocid, local.identity_tag_key, local.label_prefix
+  )
+  provider = oci.home_region
+}
+
+resource "oci_identity_policy" "adb_policies" {
+  compartment_id = var.tenancy_ocid
+  name           = format("%s-adb-policy", var.label_prefix)
+  description    = format("%s - ADB", var.label_prefix)
+  statements = [
+    format("allow dynamic-group %s to use generative-ai-family in compartment id %s", oci_identity_dynamic_group.resource_dynamic_group.name, local.compartment_ocid),
+  ]
+  provider = oci.home_region
+}

opentofu/locals.tf

@@ -6,6 +6,7 @@
 locals {
   compartment_ocid = var.compartment_ocid != "" ? var.compartment_ocid : var.tenancy_ocid
   label_prefix     = var.label_prefix != "" ? lower(var.label_prefix) : lower(random_pet.label.id)
+  identity_tag_key = format("%s.%s", oci_identity_tag_namespace.tag_namespace.name, oci_identity_tag.identity_tag.name)
 }
 
 // Autonomous Database

opentofu/main.tf

@@ -74,6 +74,7 @@ resource "oci_database_autonomous_database" "default_adb" {
   license_model                        = var.adb_license_model
   is_mtls_connection_required          = true
   whitelisted_ips                      = local.adb_whitelist_cidrs
+  defined_tags                         = { (local.identity_tag_key) = local.label_prefix }
 }
 
 // Virtual Machine
@@ -112,6 +113,7 @@ module "kubernetes" {
   compartment_id                 = local.compartment_ocid
   vcn_id                         = module.network.vcn_ocid
   region                         = var.region
+  dynamic_group                  = oci_identity_dynamic_group.resource_dynamic_group.name
   lb                             = oci_load_balancer_load_balancer.lb
   adb_id                         = oci_database_autonomous_database.default_adb.id
   adb_name                       = local.adb_name
@@ -130,6 +132,7 @@ module "kubernetes" {
   public_subnet_id               = module.network.public_subnet_ocid
   private_subnet_id              = module.network.private_subnet_ocid
   lb_nsg_id                      = oci_core_network_security_group.lb.id
+  identity_tag_key               = local.identity_tag_key
   providers = {
     oci.home_region = oci.home_region
   }

opentofu/modules/kubernetes/iam.tf

@@ -2,35 +2,10 @@
 # All rights reserved. The Universal Permissive License (UPL), Version 1.0 as shown at http://oss.oracle.com/licenses/upl
 # spell-checker: disable
 
-resource "oci_identity_tag_namespace" "tag_namespace" {
-  compartment_id = var.compartment_id
-  description    = format("%s Tag Namespace", var.label_prefix)
-  name           = var.label_prefix
-  provider       = oci.home_region
-}
-
-resource "oci_identity_tag" "identity_tag" {
-  description      = format("%s Infrastructure", var.label_prefix)
-  name             = "infrastructure"
-  tag_namespace_id = oci_identity_tag_namespace.tag_namespace.id
-  provider         = oci.home_region
-}
-
-resource "oci_identity_dynamic_group" "node_dynamic_group" {
-  compartment_id = var.tenancy_id
-  name           = format("%s-workers-dyngrp", var.label_prefix)
-  description    = format("%s Dynamic Group - K8s Workers", var.label_prefix)
-  matching_rule = format(
-    "All {instance.compartment.id = '%s', tag.%s.value = '%s'}",
-    var.compartment_id, local.identity_tag_key, local.identity_tag_val
-  )
-  provider = oci.home_region
-}
-
-resource "oci_identity_policy" "workload_node_policies" {
+resource "oci_identity_policy" "workers_policies" {
   compartment_id = var.tenancy_id
-  name           = format("%s-worker-workload-policy", var.label_prefix)
-  description    = format("%s PrincipleAuth - K8s Workers", var.label_prefix)
+  name           = format("%s-workers-policy", var.label_prefix)
+  description    = format("%s - K8s Workers", var.label_prefix)
   statements = [
     format("allow any-user to manage autonomous-database-family in compartment id %s where all {request.principal.type = 'workload', request.principal.namespace = 'oracle-database-operator-system', request.principal.service_account = 'default', request.principal.cluster_id = '%s'}", var.compartment_id, oci_containerengine_cluster.default_cluster.id),
     format("allow any-user to read objectstorage-namespaces in compartment id %s where all {request.principal.type = 'workload', request.principal.service_account = 'default', request.principal.cluster_id = '%s'}", var.compartment_id, oci_containerengine_cluster.default_cluster.id),
@@ -52,7 +27,7 @@ resource "oci_identity_policy" "workload_node_policies" {
     format("allow any-user to manage waf-family in compartment id %s where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '%s'}", var.compartment_id, oci_containerengine_cluster.default_cluster.id),
     format("allow any-user to read cluster-family in compartment id %s where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '%s'}", var.compartment_id, oci_containerengine_cluster.default_cluster.id),
     format("allow any-user to use tag-namespaces in compartment id %s where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '%s'}", var.compartment_id, oci_containerengine_cluster.default_cluster.id),
-    format("allow dynamic-group %s to manage repos in compartment id %s", oci_identity_dynamic_group.node_dynamic_group.name, var.compartment_id),
+    format("allow dynamic-group %s to manage repos in compartment id %s", var.dynamic_group, var.compartment_id),
   ]
   provider = oci.home_region
 }

opentofu/modules/kubernetes/locals.tf

@@ -4,8 +4,6 @@
 
 // Region Mapping
 locals {
-  identity_tag_key = format("%s.%s", oci_identity_tag_namespace.tag_namespace.name, oci_identity_tag.identity_tag.name)
-  identity_tag_val = var.label_prefix
   region_map = {
     for r in data.oci_identity_regions.identity_regions.regions : r.name => r.key
   }