BoT-SORT tracking, but code implements a custom IoU tracker (no KF/Hungarian/ReID/GMC)

[ArghyaChatterjee]

Hello,

Thanks for your work.

The README mentions BoT-SORT for multi-object tracking, but the current implementation appears to be a custom IoU-based tracker with heuristic scoring. I couldn’t find BoT-SORT components (Kalman filter, Hungarian/global assignment, two-stage association like ByteTrack, camera motion compensation, or ReID embeddings).

What I expected

• A BoT-SORT style tracker: constant-velocity Kalman filter state prediction, global (Hungarian) association on an IoU/appearance distance matrix, two-stage association (high-score then low-score detections), optional ReID (e.g., FastReID) and camera motion compensation (global 2×3 warp).

What I observed

• Greedy IoU matching (best IoU > threshold) per detection.
• No Kalman filter prediction/update, no linear/Hungarian assignment.
• No two-stage association using low-score detections.
• No camera motion compensation (GMC) and no ReID feature extraction/fusion.
• A heuristic track “score” that blends probability/temporal/border/size/texture factors to start/stop publishing tracks.

How to reproduce / verify

1. Search the repo for symbols like BoTSORT, bot_sort, Kalman, Hungarian, linear_assignment, FastReID, or GMC — none appear.
2. Inspect the tracker module: matching is greedy IoU, track state is updated without a motion model, and tracks are started/stopped via heuristic thresholds.

Why this matters
Calling the current implementation “BoT-SORT” may mislead users about expected robustness (occlusions, camera motion, crowded scenes). BoT-SORT’s motion model + global assignment + two-stage matching + (optional) ReID/GMC are key to its performance.

Suggestions (any of these would resolve the mismatch)

• Doc fix: Update the README to describe the tracker as a “lightweight IoU-based tracker with heuristic scoring,” and note BoT-SORT as a planned enhancement.

• Feature addition: Integrate a true BoT-SORT pipeline:

  • Per-track constant-velocity Kalman filter.
  • Hungarian assignment over an IoU/appearance distance matrix (with gating).
  • Two-stage association (high-score then low-score detections).
  • Optional ReID (e.g., FastReID) and GMC (ECC/optical-flow affine).
  • Duplicate-track suppression (keep longer-lived track when IoU is high).

If I’m missing the BoT-SORT path or a compile-time flag that enables it, happy to be corrected—please point me to the relevant files/params. Thanks!

[alexlin2]

Hi, this repository is temporarily moved to another project, I have updated the perception system significantly and includes a lot more features such as grasp generation, 3D object detection and tracking etc. The new repository is called dimensional/dimos

[alexlin2]

I just emailed you

[ArghyaChatterjee]

Hey Alex,

Can you briefly describe why you used your own tracker in stead of BoTSORT Tracker ? I have tested your repo and the detection, segmentation looks pretty stable and fine to me.

Is there a particular reason why you stayed with custom IoU based tracker with heuristic scoring over the original BoTSORT Tracker ?

[alexlin2]

Hey Alex,

Can you briefly describe why you used your own tracker in stead of BoTSORT Tracker ? I have tested your repo and the detection, segmentation looks pretty stable and fine to me.

Is there a particular reason why you stayed with custom IoU based tracker with heuristic scoring over the original BoTSORT Tracker ?

Actually both are used. The FastSAM implementation I'm using is ultralytics, and when you have the model.track(persist=true) it uses the default tracker which is BoT-SORT (https://docs.ultralytics.com/datasets/track/#dataset-format-coming-soon). The reason why I implemented heuristics and scoring over it is because their implementation did not yield the results I wanted so I just added my own on top of it which is kind of hacky tbh.