Jerome's Checklist

[Ishaan-Datta]

• Converting .onnx file to .trt in colab
• Implementing the TensorRT engine inference using NVIDIA examples and other tutorials in C++
• Optimizing the inference with batching, CUDA contexts, multithreading/asynchronous streams
• Include best practices: NVIDIA TensorRT optimizations (include creating interfaces or abstract classes that define the common functionality required for neural network processing)
• Comparison w/ industry examples (Nvidia, Zed, Github examples) to adapt our case to the Jetson hardware
• Metric logging for benchmarking using Nvidia tutorials
• Test different batch sizes to maximize the GPU utlilization
• Investigate any other potential areas for multiprocessing or parallel execution to speed up task processing

Links Checklist:

• TensorRT C++ API Repository
• Youtube Tutorial
• Nvidia Support Matrix
• Nvidia Archived Best Practices
• Nvidia Object Detection Examples
• Nvidia Guide for Benchmarking/Profiling
• Nvidia Guide for Optimization
• Nvidia Example ONNX Deployment
• Nvidia Basic Workflow (for making docs)

Links Checklist (some of these may overlap w/ Jerome's so maybe meet with him to discuss progress and what needs to be revisited or reviewed)

• YOLOV8
• ZedBox Custom Input
• ONNX TensorRT
• Python Tutorial Script
• Nvidia Tutorial Example
• Updated Nvidia Tutorial
• TRTexec Tool Script