Skip to content

docs: updating readme content #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
connor-leech wants to merge 2 commits into
base: main
Choose a base branch
from
Open

docs: updating readme content #12

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
95fcd53
docs: updating readme content
connor-leech Mar 26, 2025
15737c1
clarity and formatting improvements
connor-leech Apr 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
191 changes: 72 additions & 119 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,170 +1,123 @@
# CISO Agent

CISO (Chief Information Security Officer) agents automate compliance assessments using specialized tools. They generate policies (e.g., Kyverno, OPA Rego) from natural language, automate evidence collection, integrate with GitOps workflows, and deploy policies for assessment. Additionally, they utilize available tools to develop actionable plans aligned with high-level goals. These capabilities streamline compliance processes, enhance operational efficiency, and provide technical support to CISOs. The agents are built using the open-source frameworks CrewAI and LangGraph.

CISO (Chief Information Security Officer) agents automate compliance assessments. They generate policies (e.g., Kyverno, OPA Rego) from natural language prompts, automate evidence collection, integrate with GitOps workflows, and deploy policies to validate compliance. These capabilities streamline compliance processes and enhance operational efficiency. The agents are built using the open-source frameworks [CrewAI](https://github.com/crewAIInc/crewAI) and [LangGraph](https://github.com/langchain-ai/langgraph).

## Prerequisites

- Access to an OpenAI-Compatible LLM service
- tested with `IBM watsonx.ai`, `OpenAI` and `Azure OpenAI Service`.
- [Sample Task Scenario](https://github.com/IBM/ITBench-Scenarios) setup
- This requires 1 Kubernetes cluster and/or 1 RHEL host. For more details, please refer to its [README](https://github.com/IBM/ITBench-Scenarios/tree/main/ciso#readme).
- `python` command (tested with `3.11` )
- `docker` or `podman` command (tested with docker `26.1.0` and podman `5.1.2` )
- Access to an OpenAI-compatible LLM service
(tested with IBM watsonx.ai, OpenAI, and Azure OpenAI)
- A deployed [ITBench scenario](https://github.com/IBM/ITBench-Scenarios)
(requires 1 Kubernetes cluster and/or 1 RHEL host)
- `python` (tested with 3.11)
- `docker` or `podman` (tested with Docker 26.1.0, Podman 5.1.2)

## Getting started

### 1. Setup a Sample Task Scenario on your Kubernetes cluster / RHEL host

Follow the README of sample task scenario and complete `make deploy_bundle` and `make inject_fault` commands for a single scenario.
### 1. Deploy a scenario

Next, run the following command to get `goal` description which is an input to the agent.
Follow the [CISO scenario guide](https://github.com/IBM/ITBench-Scenarios/tree/main/ciso#readme):

```bash
$ make get 2>/dev/null | jq -r .goal_template
make deploy_bundle
make inject_fault
```

Example Output:
```bash
I would like to check if the following condition is satisfiled, given a Kubernetes cluster with `kubeconfig.yaml`
Minimize the admission of containers wishing to share the host network namespace.

To check the condition, do the following steps.
- deploy a Kyverno policy to the cluster
- chcek if the policy is correctly deployed.
Then get the scenario goal for the agent:

If deploying the policy failed and if you can fix the issue, you will do it and try deploying again.
Once you get a final answer, you can quit the work.

The cluster's kubeconfig is at `{{ kubeconfig }}`.
```bash
make get 2>/dev/null | jq -r .goal_template
```

The above text is an example of ciso task scenarios.
Example goal output:

Please keep this goal text for running the agent later.
```text
I would like to check if the following condition is satisfied, given a Kubernetes cluster with `kubeconfig.yaml`:
Minimize the admission of containers wishing to share the host network namespace.

### 2. Clone this repository
To check the condition, do the following steps:
- Deploy a Kyverno policy to the cluster
- Check if the policy is correctly deployed

```bash
$ git clone https://github.com/IBM/it-bench-ciso-caa-agent.git
$ cd ciso-agent
If deploying the policy fails and you can fix the issue, do so and try again.

The cluster's kubeconfig is at `{{ kubeconfig }}`.
```

### 3. Build agent container image
Save this goal text for later.

CISO Agent is designed to run as a container. Please build the container image with this command.
### 2. Clone this repository

```bash
$ docker build . -f Dockerfile -t ciso-agent:latest
git clone https://github.com/IBM/it-bench-ciso-caa-agent.git
cd ciso-agent
```

If you are using `podman` runtime instead of `docker`, this is the command.
### 3. Build the agent container

```bash
$ podman build . -f Dockerfile -t ciso-agent:latest
docker build -f Dockerfile -t ciso-agent:latest .
# or, if using podman:
podman build -f Dockerfile -t ciso-agent:latest .
```

This command could take a several minutes normally.

**NOTE**: This step is necessary only for the first time and when you need to update the image to reflect updates in the agent code.
### 4. Configure `.env` with LLM credentials

### 3. Create `.env` file and set LLM API credentials
Create a `.env` file using your LLM service credentials:

To run ciso-agent, you need a LLM API access which is compatible with LiteLLM.

Many LLM services support it, including IBM watsonx.ai, OpenAI and Azure OpenAI Service.

To configure access, create a .env file with the following details.

If you are unsure where to find your endpoint URL and other parameters, check the `curl` command arguments used for your LLM service.

i. For **IBM watsonx.ai**

```bash
# .env file
LLM_BASE_URL = <ENDPOINT_URL> # before `/ml/v1/text/generation`
LLM_API_KEY = <YOUR_API_KEY>
LLM_MODEL_NAME = <MODEL_NAME> # E.g. "ibm/granite-3-8b-instruct"
WATSONX_PROJECT_ID = <YOUR_WATSONX_PROJECT_ID>
- **IBM watsonx.ai**
```env
LLM_BASE_URL=<ENDPOINT_URL>
LLM_API_KEY=<YOUR_API_KEY>
LLM_MODEL_NAME=<MODEL_NAME>
WATSONX_PROJECT_ID=<YOUR_PROJECT_ID>
```

ii. For **OpenAI**
```bash
# .env file
LLM_API_KEY = <YOUR_API_KEY>
LLM_MODEL_NAME = <MODEL_NAME> # E.g. "gpt-4o-mini"
# NOTE: The endpoint URL an be omitted for OpenAI
- **OpenAI**
```env
LLM_API_KEY=<YOUR_API_KEY>
LLM_MODEL_NAME=<MODEL_NAME>
```

iii. For **Azure OpenAI Service**
```bash
# .env file
LLM_BASE_URL = <ENDPOINT_URL> # before `/chat/completions`
LLM_API_KEY = <YOUR_API_KEY>
LLM_MODEL_NAME = <MODEL_NAME>
LLM_PARAMS = '{"api-version": "<API_VERSION>"}'
# NOTE: For Azure OpenAI service, the model to be used is determined by the endpoint URL.
# Thus, <MODEL_NAME> here is ignored during LLM calls.
- **Azure OpenAI**
```env
LLM_BASE_URL=<ENDPOINT_URL>
LLM_API_KEY=<YOUR_API_KEY>
LLM_MODEL_NAME=<MODEL_NAME>
LLM_PARAMS='{"api-version": "<API_VERSION>"}'
```

Any location is fine for `.env` file, but the file needs to be mountable by the agent container later.

### 4. Start the agent

Now, ready to run the agent.
### 5. Run the agent

Inputs to the agent is the following 3 things.
- Workdir
- The directory for scenario. Before agent starts, `kubeconfig.yaml` and/or `inventory.ansible.ini` must be planced here.
- `.env` file
- Goal text
Prepare:

Please run the following command with these inputs.
- `<WORKDIR>`: Contains scenario files (`kubeconfig.yaml`, etc.)
- `<DOT_ENV>`: Path to your `.env`
- `<GOAL_TEXT>`: Scenario goal (replace `{{ kubeconfig }}` with `/tmp/agent/kubeconfig.yaml`)

Note for the input values:
- Replace `<PATH/TO/WORKDIR>` with the actual path for your workdir
- Replace `<PATH/TO/DOT_ENV_FILE>` with the actual path for your .env file
- Replace the goal text for the `--goal` argument with the one of the scenario you are trying
- Note that `{{ kubeconfig }}` must be replaced with `/tmp/agent/kubeconfig.yaml`.
- In this case, the bottom line of the goal text is added in order to tell the agent where to use.

If you use `podman` engine, please replace `docker` with `podman` below.
Then run:

```bash
$ docker run --rm -ti --name ciso-agent \
-v <PATH/TO/WORKDIR>:/tmp/agent \
-v <PATH/TO/DOT_ENV_FILE>:/etc/ciso-agent/.env \
ciso-agent:latest \
python src/ciso_agent/main.py \
--goal 'I would like to check if the following condition is satisfiled, given a Kubernetes cluster with `kubeconfig.yaml`
Minimize the admission of containers wishing to share the host network namespace.

To check the condition, do the following steps.
- deploy a Kyverno policy to the cluster
- chcek if the policy is correctly deployed.

If deploying the policy failed and if you can fix the issue, you will do it and try deploying again.
Once you get a final answer, you can quit the work.

The cluster's kubeconfig is at `/tmp/agent/kubeconfig.yaml`.
You can use `/tmp/agent` as your workdir.' \
--auto-approve
docker run --rm -ti \
-v <WORKDIR>:/tmp/agent \
-v <DOT_ENV>:/etc/ciso-agent/.env \
ciso-agent:latest \
python src/ciso_agent/main.py \
--goal '<GOAL_TEXT>' \
--auto-approve
```

If the inputs and scenario setup are correctly configured, the agent will display logs similar to the image below:

<img src="img/agent_log_example_beginning.png" alt="Example agent log at the beginning">

The agent will then continue its work until it achieves the goal.
Replace `docker` with `podman` if needed.

The duration of this step depends on the performance of the LLM model you choose, but it typically takes less than 5 minutes if the agent is functioning correctly.
The agent logs will appear similar to:

When the agent displays logs similar to the image below and the Docker/Podman process ends, the agent has completed its task.
![Start](img/agent_log_example_beginning.png)
(Agent starting)

<img src="img/agent_log_example_result.png" alt="Example agent log for results">
![Result](img/agent_log_example_result.png)
(Agent finished)

### 5. Evaluation
The run typically takes under 5 minutes.

Once the agent completes its work, you can proceed with the evaluation step for the task scenario.
### 6. Evaluation

Please refer to the README of the task scenario for further details.
After the agent finishes, evaluate results using the scenario’s instructions [here](https://github.com/IBM/ITBench-Scenarios/tree/main/ciso#readme).