> For the complete documentation index, see [llms.txt](https://jgoodman8.gitbook.io/iron-data-science-notebook/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://jgoodman8.gitbook.io/iron-data-science-notebook/ml-datascience/computer-vision/mot/deep-sort.md).

# Deep SORT

The improvements w\.r.t. the [SORT](/iron-data-science-notebook/ml-datascience/computer-vision/mot/sort.md) algorithm:

* Incorporates association metrics based on appearance features.
* Incorporates ID assignment to track individual objects across frames.

<figure><img src="https://569842953-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmjpNbCRLUyGiAxD8kn%2Fuploads%2FOJIZp1aVslR7nC0p6uLP%2Fimage.png?alt=media&amp;token=e987ce7a-9e32-41df-95a2-15f8425e5646" alt=""><figcaption></figcaption></figure>

## Steps

1. Detection and feature extraction
   * Using any detector
2. Apply Kalman filter
   * For the state prediction
   * Given the current position, velocity, and acceleration
   * Predicts the state of each object given the motion dynamics
3. Association:
   * Hungarian algorithm performs matching based on a cost matrix that considers:
     * **Mahalanobis** distance between bboxes (current and Kalman filters' prediction)
     * **Cosine distance** for appearance similarity
4. Track management (algorithm heuristics)
   * It confirms a track when is detected after several consecutive frames.
   * Age parameter to remove old tracks no longer existing.