WIP: feat(AWML): introduce new deployment and onnx/tensorRT evaluation pipeline

deployment/README.md

@@ -0,0 +1,74 @@
+# AWML Deployment Framework
+
+AWML ships a unified, task-agnostic deployment stack that turns trained PyTorch checkpoints into production-ready ONNX and TensorRT artifacts. The verification and evaluation toolchain runs across every backend, ensuring numerical parity and consistent metrics across different projects.
+
+At the center is a shared runner/pipeline/exporter architecture that teams can extend with lightweight wrappers or workflows. CenterPoint, YOLOX, CalibrationStatusClassification, and future models plug into the same export and verification flow while still layering in task-specific logic where needed.
+
+
+## Quick Start
+
+```bash
+# Deployment entrypoint
+python -m deployment.cli.main <project> <deploy_cfg.py> <model_cfg.py> [project-specific args]
+
+# Example: CenterPoint deployment
+python -m deployment.cli.main centerpoint <deploy_cfg.py> <model_cfg.py> --rot-y-axis-reference
+```
+
+## Documentation Map
+
+| Topic | Description |
+| --- | --- |
+| [`docs/overview.md`](docs/overview.md) | Design principles, key features, precision policies. |
+| [`docs/architecture.md`](docs/architecture.md) | Workflow diagram, core components, file layout. |
+| [`docs/usage.md`](docs/usage.md) | CLI usage, runner patterns, typed contexts, export modes. |
+| [`docs/configuration.md`](docs/configuration.md) | Config structure, typed schemas, backend enums. |
+| [`docs/projects.md`](docs/projects.md) | CenterPoint, YOLOX, and Calibration deployment specifics. |
+| [`docs/export_pipeline.md`](docs/export_pipeline.md) | ONNX/TRT export steps and pipeline patterns. |
+| [`docs/verification_evaluation.md`](docs/verification_evaluation.md) | Verification scenarios, evaluation metrics, core contract. |
+| [`docs/best_practices.md`](docs/best_practices.md) | Best practices, troubleshooting, roadmap. |
+| [`docs/contributing.md`](docs/contributing.md) | How to add new deployment projects end-to-end. |
+
+Refer to `deployment/docs/README.md` for the same index.
+
+## Architecture Snapshot
+
+- **Entry point** (`deployment/cli/main.py`) loads a project bundle from `deployment/projects/<project>/`.
+- **Runtime** (`deployment/runtime/*`) coordinates load → export → verify → evaluate via shared orchestrators.
+- **Exporters** live under `exporters/common/` with typed config classes; project wrappers/pipelines compose the base exporters as needed.
+- **Pipelines** are registered by each project bundle and resolved via `PipelineFactory`.
+- **Core package** (`core/`) supplies typed configs, runtime contexts, task definitions, and shared verification utilities.
+
+See [`docs/architecture.md`](docs/architecture.md) for diagrams and component details.
+
+## Export & Verification Flow
+
+1. Load the PyTorch checkpoint and run ONNX export (single or multi-file) using the injected wrappers/pipelines.
+2. Optionally build TensorRT engines with precision policies such as `auto`, `fp16`, `fp32_tf32`, or `strongly_typed`.
+3. Register artifacts via `ArtifactManager` for downstream verification and evaluation.
+4. Run verification scenarios defined in config—pipelines are resolved by backend and device, and outputs are recursively compared with typed tolerances.
+5. Execute evaluation across enabled backends and emit typed metrics.
+
+Implementation details live in [`docs/export_pipeline.md`](docs/export_pipeline.md) and [`docs/verification_evaluation.md`](docs/verification_evaluation.md).
+
+## Project Coverage
+
+- **CenterPoint** – multi-file export orchestrated by dedicated ONNX/TRT pipelines; see [`docs/projects.md`](docs/projects.md).
+- **YOLOX** – single-file export with output reshaping via `YOLOXOptElanONNXWrapper`.
+- **CalibrationStatusClassification** – binary classification deployment with identity wrappers and simplified pipelines.
+
+Each project ships its own deployment bundle under `deployment/projects/<project>/`.
+
+## Core Contract
+
+[`core_contract.md`](docs/core_contract.md) defines the boundaries between runners, orchestrators, evaluators, pipelines, and metrics interfaces. Follow the contract when introducing new logic to keep refactors safe and dependencies explicit.
+
+## Contributing & Best Practices
+
+- Start with [`docs/contributing.md`](docs/contributing.md) for the required files and patterns when adding a new deployment project.
+- Consult [`docs/best_practices.md`](docs/best_practices.md) for export patterns, troubleshooting tips, and roadmap items.
+- Keep documentation for project-specific quirks in the appropriate file under `deployment/docs/`.
+
+## License
+
+See LICENSE at the repository root.

deployment/__init__.py

@@ -0,0 +1,22 @@
+"""
+Autoware ML Unified Deployment Framework
+
+This package provides a unified, task-agnostic deployment framework for
+exporting, verifying, and evaluating machine learning models across different
+tasks (classification, detection, segmentation, etc.) and backends (ONNX,
+TensorRT).
+"""
+
+from deployment.configs import BaseDeploymentConfig
+from deployment.core.evaluation.base_evaluator import BaseEvaluator
+from deployment.core.io.base_data_loader import BaseDataLoader
+from deployment.runtime.runner import BaseDeploymentRunner
+
+__all__ = [
+    "BaseDeploymentConfig",
+    "BaseDataLoader",
+    "BaseEvaluator",
+    "BaseDeploymentRunner",
+]
+
+__version__ = "1.0.0"

deployment/cli/__init__.py

@@ -0,0 +1 @@
+"""Deployment CLI package."""

deployment/cli/args.py

@@ -0,0 +1,56 @@
+"""
+CLI helpers: argument parsing and logging setup.
+
+Config schema does not depend on argparse/logging; this module is the single place for CLI concerns.
+"""
+
+from __future__ import annotations
+
+import argparse
+import logging
+from typing import Optional
+
+
+def setup_logging(level: str = "INFO") -> logging.Logger:
+    """
+    Setup logging configuration.
+
+    Args:
+        level: Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
+
+    Returns:
+        Configured logger instance
+    """
+    logging.basicConfig(level=getattr(logging, level), format="%(levelname)s:%(name)s:%(message)s")
+    return logging.getLogger("deployment")
+
+
+def parse_base_args(
+    parser: Optional[argparse.ArgumentParser] = None,
+) -> argparse.ArgumentParser:
+    """
+    Create argument parser with common deployment arguments.
+
+    Args:
+        parser: Optional existing ArgumentParser to add arguments to
+
+    Returns:
+        ArgumentParser with deployment arguments
+    """
+    if parser is None:
+        parser = argparse.ArgumentParser(
+            description="Deploy model to ONNX/TensorRT",
+            formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+        )
+
+    parser.add_argument("deploy_cfg", help="Deploy config path")
+    parser.add_argument("model_cfg", help="Model config path")
+    # Optional overrides
+    parser.add_argument(
+        "--log-level",
+        default="INFO",
+        choices=["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"],
+        help="Logging level",
+    )
+
+    return parser

deployment/cli/main.py

@@ -0,0 +1,101 @@
+"""
+Single deployment entrypoint.
+
+Usage:
+    python -m deployment.cli.main <project> <deploy_cfg.py> <model_cfg.py> [project-specific args]
+"""
+
+from __future__ import annotations
+
+import argparse
+import importlib
+import pkgutil
+import sys
+import traceback
+from typing import List
+
+import deployment.projects as projects_pkg
+from deployment.cli.args import parse_base_args
+from deployment.projects import project_registry
+
+
+def _discover_project_packages() -> List[str]:
+    """Discover project package names under deployment.projects (without importing them)."""
+
+    names: List[str] = []
+    for mod in pkgutil.iter_modules(projects_pkg.__path__):
+        if not mod.ispkg:
+            continue
+        if mod.name.startswith("_"):
+            continue
+        names.append(mod.name)
+    return sorted(names)
+
+
+def _import_and_register_project(project_name: str) -> None:
+    """Import project package, which should register itself into project_registry."""
+    importlib.import_module(f"deployment.projects.{project_name}")
+
+
+def build_parser() -> argparse.ArgumentParser:
+    """Build the unified deployment CLI parser.
+
+    This discovers `deployment.projects.<name>` bundles, imports them to trigger
+    registration into `deployment.projects.project_registry`, then creates a
+    subcommand per registered project.
+    """
+    parser = argparse.ArgumentParser(
+        description="AWML Deployment CLI",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+
+    subparsers = parser.add_subparsers(dest="project", required=True)
+
+    # Discover projects and import them so they can contribute args.
+    failed_projects: List[str] = []
+    for project_name in _discover_project_packages():
+        try:
+            _import_and_register_project(project_name)
+        except Exception as e:
+            tb = traceback.format_exc()
+            failed_projects.append(f"- {project_name}: {e}\n{tb}")
+            continue
+
+        try:
+            adapter = project_registry.get(project_name)
+        except KeyError:
+            continue
+
+        sub = subparsers.add_parser(project_name, help=f"{project_name} deployment")
+        parse_base_args(sub)  # adds deploy_cfg, model_cfg, --log-level
+        adapter.add_args(sub)
+        sub.set_defaults(_adapter_name=project_name)
+
+    if not project_registry.list_projects():
+        details = "\n".join(failed_projects) if failed_projects else "(no project packages discovered)"
+        raise RuntimeError(
+            "No deployment projects were registered. This usually means project imports failed.\n" f"{details}"
+        )
+
+    return parser
+
+
+def main(argv: List[str] | None = None) -> int:
+    """CLI entrypoint.
+
+    Args:
+        argv: Optional argv list (without program name). If None, uses `sys.argv[1:]`.
+
+    Returns:
+        Process exit code (0 for success).
+    """
+    argv = sys.argv[1:] if argv is None else argv
+    parser = build_parser()
+    args = parser.parse_args(argv)
+
+    adapter = project_registry.get(args._adapter_name)
+    return int(adapter.run(args) or 0)
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

deployment/configs/__init__.py

@@ -0,0 +1,28 @@
+"""
+Deployment config package: enums, schema, and base container.
+
+Re-export commonly used types for clean imports:
+
+    from deployment.configs import BaseDeploymentConfig, ExportMode, TensorRTConfig
+"""
+
+from deployment.configs.base import BaseDeploymentConfig
+from deployment.configs.enums import ExportMode, PrecisionPolicy
+from deployment.configs.schema import (
+    ComponentsConfig,
+    DeviceConfig,
+    ExportConfig,
+    OnnxConfig,
+    TensorRTConfig,
+)
+
+__all__ = [
+    "BaseDeploymentConfig",
+    "ComponentsConfig",
+    "DeviceConfig",
+    "ExportConfig",
+    "ExportMode",
+    "OnnxConfig",
+    "PrecisionPolicy",
+    "TensorRTConfig",
+]