From 2095b9644351636cfa24f18d29b29ad73aa336f2 Mon Sep 17 00:00:00 2001 From: vangeliq Date: Sat, 15 Mar 2025 12:37:16 -0700 Subject: [PATCH] added uml diagram --- README.md | 56 ++++++++++-- assets/node_diagram.drawio | 180 +++++++++++++++++++++++++++++++++++++ assets/node_diagram.svg | 4 + ros2_ws/launch.sh | 3 + 4 files changed, 238 insertions(+), 5 deletions(-) create mode 100644 assets/node_diagram.drawio create mode 100644 assets/node_diagram.svg create mode 100644 ros2_ws/launch.sh diff --git a/README.md b/README.md index 97a4f30..715b2ae 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,54 @@ -# ROS +# ROS Computer Vision System -Asset folder holds all image sources +Developed by UBC AgroBot Systems Architecture Team --> video setting should be named video.mp4 -images can be named w/e but must end with .jpg or .png +## Background Information -maize model dimensions: 1024x448 (448 is width?) \ No newline at end of file +This repository contains a ROS2-based computer vision system for agricultural robotics. The system processes visual data to enable autonomous navigation and crop identification for the AgroBot. It includes both hardware integration components and software algorithms for real-time image processing and decision-making. + +## Overview +![Node Diagram](assets/node_diagram.svg) + +The system consists of four main components: +1. **Camera Input Processing Unit**: Captures data from the Zed Camera for inference. +2. **Inference Unit**: This is the main processing hub, running the core ROS nodes and computer vision algorithms. +3. **Inference Result Handler**: Manages inference data, performs preprocessing, and handles object tracking. +4. **Signal Combining**: Combines signals from both the left and right cameras to send to the designated module. + +On the Jetson, the ROS `proxy_node` sends commands to the Arduino for physical actuation. + +## Quickstart + +To quickly start the system, use a ROS launch file to launch the entire stack. + +First, ensure you have a ROS2 environment. Open the command palette in VS Code and select `Dev Containers: Rebuild and Reopen in Container`. + +Our project is built and run inside the `ros2_ws` directory. Navigate to it by running: +``` +cd ros2_ws +``` + +To build and run the stack, execute the following shell script: +``` +./launch.sh +``` + +Congratulations 🎉! The entire stack is now running locally on your computer. + +## Other Available Utilities + +### Assets + +The `assets` folder contains all image and video sources for testing: +- Video files should be named `video.mp4`. +- Images can have any name but must end with `.jpg` or `.png`. + +Maize model dimensions: 1024x448 (448 is the width). + +### `action_scripts` + +The `scripts` folder contains various action scripts used in the project. + +### `python_wip` + +This folder contains experimental Python code related to computer vision that is still under development and may be pushed in the future. \ No newline at end of file diff --git a/assets/node_diagram.drawio b/assets/node_diagram.drawio new file mode 100644 index 0000000..1f3a556 --- /dev/null +++ b/assets/node_diagram.drawio @@ -0,0 +1,180 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/assets/node_diagram.svg b/assets/node_diagram.svg new file mode 100644 index 0000000..81a96e1 --- /dev/null +++ b/assets/node_diagram.svg @@ -0,0 +1,4 @@ + + + +
left_camera_node
left_camera_node
right_camera_node
right_camera_node
left_inference_node
left_inference_node
left_post_node
left_post_node
proxy_node
proxy_node
right_inference_node
right_inference_node
right_post_node
right_post_node
/left_image_input
/left_imag...
/left_inference_out
/left_infe...
/right_image_input
/right_ima...
/right_inference_out
/right_inf...
/extermination_out
/extermina...
/extermination_out
/extermina...
Send info to arduino
Send info...
/right_get_row_count service
/right_get...
/left_get_row_count service
/left_get_...
Camera Node
Camera Node
Connects to Zed Camera
Connects to Zed Camera
Image preprocessing
Image preprocessing
Inference Node
Inference Node
Instantiates model
Instantiates model
Runs inference on image
Runs inference on image
Post-Inference Node
Post-Inference Node
Connects to Zed Camera
Connects to Zed Camera
Image preprocessing%3CmxGraphModel%3E%3Croot%3E%3CmxCell%20id%3D%220%22%2F%3E%3CmxCell%20id%3D%221%22%20parent%3D%220%22%2F%3E%3CmxCell%20id%3D%222%22%20value%3D%22Post-Inference%20Node%22%20style%3D%22swimlane%3BfontStyle%3D0%3BchildLayout%3DstackLayout%3Bhorizontal%3D1%3BstartSize%3D30%3BhorizontalStack%3D0%3BresizeParent%3D1%3BresizeParentMax%3D0%3BresizeLast%3D0%3Bcollapsible%3D1%3BmarginBottom%3D0%3BwhiteSpace%3Dwrap%3Bhtml%3D1%3B%22%20vertex%3D%221%22%20parent%3D%221%22%3E%3CmxGeometry%20x%3D%22480%22%20y%3D%22420%22%20width%3D%22140%22%20height%3D%2290%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%223%22%20value%3D%22Connects%20to%20Zed%20Camera%22%20style%3D%22text%3BstrokeColor%3Dnone%3BfillColor%3Dnone%3Balign%3Dleft%3BverticalAlign%3Dmiddle%3BspacingLeft%3D4%3BspacingRight%3D4%3Boverflow%3Dhidden%3Bpoints%3D%5B%5B0%2C0.5%5D%2C%5B1%2C0.5%5D%5D%3BportConstraint%3Deastwest%3Brotatable%3D0%3BwhiteSpace%3Dwrap%3Bhtml%3D1%3B%22%20vertex%3D%221%22%20parent%3D%222%22%3E%3CmxGeometry%20y%3D%2230%22%20width%3D%22140%22%20height%3D%2230%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%224%22%20value%3D%22Image%20preprocessing%22%20style%3D%22text%3BstrokeColor%3Dnone%3BfillColor%3Dnone%3Balign%3Dleft%3BverticalAlign%3Dmiddle%3BspacingLeft%3D4%3BspacingRight%3D4%3Boverflow%3Dhidden%3Bpoints%3D%5B%5B0%2C0.5%5D%2C%5B1%2C0.5%5D%5D%3BportConstraint%3Deastwest%3Brotatable%3D0%3BwhiteSpace%3Dwrap%3Bhtml%3D1%3B%22%20vertex%3D%221%22%20parent%3D%222%22%3E%3CmxGeometry%20y%3D%2260%22%20width%3D%22140%22%20height%3D%2230%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3C%2Froot%3E%3C%2FmxGraphModel%3E
Image preprocessing%3CmxGraphMod...
provides get_row_count service that can be called by another process to return a count of the number of plants 
provides get_row_count service t...
Proxy Node
Proxy Node
Combines processed data from both left and right nodes
Combines processed dat...
Sends a single boolean sprayer signal to the arduino 
Sends a single boolean...
/image_input topic
/image_input topic
 sensor_msgs/Image preprocessed_image
 sensor_msgs/Image preprocessed_image
sensor_msgs/Image raw_image
sensor_msgs/Image raw_image
/inference_out topic
/inference_out topic
std_msgs/Float32MultiArray bounding_boxes 
std_msgs/Float32MultiArray bounding_boxes 
 std_msgs/Float32MultiArray confidence 
 std_msgs/Float32MultiArray confidence 
sensor_msgs/Image raw_image
sensor_msgs/Image raw_image
 sensor_msgs/Image preprocessed_image
 sensor_msgs/Image preprocessed_image
/extermination_out topic
/extermination_out topic
std_msgs/Float32MultiArray bounding_boxes 
std_msgs/Float32MultiArray bounding_boxes 
/right_get_row_count service
/right_get_row_count s...
---
int16 plant_count
---...Text is not SVG - cannot display \ No newline at end of file diff --git a/ros2_ws/launch.sh b/ros2_ws/launch.sh new file mode 100644 index 0000000..a0c29e6 --- /dev/null +++ b/ros2_ws/launch.sh @@ -0,0 +1,3 @@ +colcon build --symlink-install --packages-select +source install/setup.bash +ros2 launch python_package/launch/launch.py use_display_node:=False \ No newline at end of file