Introduction

Following are my notes on BYTE, simple and effective association method. Proposed in paper ByteTrack: Multi-Object Tracking by Associating Every Detection Box.

BYTE, The Algo?

• For each frame in the video, we predict the detection boxes and scores using the detector Det.
• We separate all the detection boxes D_boxes into two parts D_high and D_low according to the detection score threshold τ.
• detection boxes, D_boxes > τ stored into D_high
• detection boxes, D_boxes < τ stored into D_low
• Use Kalman Filter to predict new location in current frame for each track in T;
• First association is performed between D_high and all the tracks T (including the lost tracks T_lost ) using Similarity #1.
• Similarity #1 can be computed either by the IoU or the Re-ID feature distances between the D_high and the predicted box of tracks T.
• Next, we adopt Hungarian Algorithm to finish the matching based on the similarity.
• We keep the unmatched detections in D_remain and the unmatched tracks in T_remain.
• Second association is performed between the low score detection boxes D_low and the remaining tracks T_remain after the first association.
• We keep the unmatched tracks in T_re-remain and just delete all the unmatched low score detection boxes, since we view them as background.
• We find it important to use IoU alone as the Similarity #2 in the second association because the low score detection boxes usually contains severe occlusion or motion blur and appearance features are not reliable.
• For the unmatched tracks T_re-remain after the second association, we put them into T_lost.
• For each track in T_lost , only when it exists for more than a certain number of frames, i.e. 30, we delete it from the tracks T . Otherwise, we remain the lost tracks T_lost in T.
• Finally, we initialize new tracks from the unmatched high score detection boxes D_remain after the first association.
• The output of each individual frame is the bounding boxes and identities of the tracks T in the current frame.