This repo is the official implementation of "UAWTrack: Universal 3D Single Object Tracking in Adverse Weather".
AdverseWeather.mp4
3D single object tracking (3D SOT) in LiDAR point clouds is essential for autonomous driving. Most existing 3D SOT methods focus on clear weather, where point clouds are more defined. However, adverse weather conditions lead to sparser and noisier point clouds, significantly degrading tracking performance and posing safety risks. In this study, we introduce UAWTrack, a universal 3D SOT model designed to perform effectively across diverse real-world weather conditions. UAWTrack comprises three key modules: 1) Voxel Feature Extraction, which mitigates the perturbations in point clouds caused by adverse weather; 2) Motion-centric Spatial-temporal Aggregation and Motion-guided Feature Fusion, capturing motion clues and sampling dense BEV motion features to address the issue of sparsity; and 3) Weather-Specific Tracker, which efficiently handles tracking in various weather conditions. To fill the gap of lacking benchmarks for 3D SOT in adverse weather, we simulate physically valid adverse weather conditions on the KITTI and NuScenes datasets, creating two benchmarks: KITTI-A and NuScenes-A. Extensive experiments demonstrate that UAWTrack achieves state-of-the-art performance under all weather conditions.
To address the lack of datasets for 3D SOT under adverse weather conditions, we simulate physically valid weather on the KITTI and NuScenes datasets, generating two new datasets, KITTI-A and NuScenes-A. These datasets include three types of adverse weather — rain, snow, and fog — each with three intensity levels, as well as clear weather conditions. Below, we provide technical details on the simulated weather types, with visualization examples.
Rain: We simulate rain using the LISA technique on both KITTI and NuScenes datasets. Rainfall is categorized into three levels with rates set at {0.20, 1.5625, and 7.29}.
Snow: Snow simulation is also implemented using the LISA technique, with snowfall rates mirroring the rain intensities at {0.20, 1.5625, and 7.29}.
Fog: Fog datasets are generated by setting the attenuation coefficient to {0.005, 0.02, and 0.06} across three levels, based on the clear KITTI and NuScenes datasets.
| Weather Type | Tracker | Source | Car | Pedestrian | Van | Cyclist | Mean | Mean by Category |
|---|---|---|---|---|---|---|---|---|
| Clear | P2B | CVPR'20 | 54.4/68.7 | 36.9/59.6 | 42.2/50.8 | 27.9/37.7 | 45.2/62.5 | 40.4/54.2 |
| M2Track | CVPR'22 | 62.7/76.0 | 49.9/80.4 | 61.4/77.2 | 73.1/93.4 | 57.3/78.4 | 61.8/81.8 | |
| GLT-T | AAAI'23 | 66.3/79.3 | 44.8/71.5 | 44.8/52.7 | 58.3/87.8 | 54.9/73.8 | 53.5/72.9 | |
| MBPTrack | ICCV'23 | 70.6/81.3 | 58.3/85.0 | 67.0/78.4 | 69.6/93.0 | 64.9/82.9 | 66.4/84.4 | |
| UAWTrack | AAAI'25 | 71.8/84.0 | 62.2/88.9 | 64.5/77.7 | 74.8/94.1 | 67.1/85.8 | 68.3/86.1 | |
| Rain | P2B | CVPR'20 | 35.0/42.1 | 36.7/58.8 | 20.6/23.2 | 27.5/38.0 | 34.3/47.6 | 30.0/40.5 |
| M2Track | CVPR'22 | 42.9/52.1 | 50.5/81.8 | 26.3/32.5 | 73.4/93.5 | 45.4/64.1 | 45.4/64.1 | |
| GLT-T | AAAI'23 | 38.5/45.8 | 44.9/71.0 | 21.7/23.3 | 55.9/84.2 | 40.2/55.6 | 40.2/55.6 | |
| MBPTrack | ICCV'23 | 55.0/66.9 | 58.5/86.1 | 25.6/28.5 | 68.9/92.8 | 54.2/72.4 | 52.0/68.6 | |
| UAWTrack | AAAI'25 | 58.3/70.2 | 58.0/85.8 | 41.0/49.5 | 74.5/94.1 | 57.0/75.6 | 60.3/74.9 | |
| Snow | P2B | CVPR'20 | 34.3/41.4 | 36.3/58.5 | 20.6/23.2 | 28.7/39.4 | 33.8/47.1 | 30.0/40.6 |
| M2Track | CVPR'22 | 41.1/49.6 | 51.2/82.6 | 27.3/33.7 | 72.5/93.2 | 44.9/63.4 | 48.0/64.8 | |
| GLT-T | AAAI'23 | 34.8/41.1 | 42.3/67.8 | 21.1/22.6 | 57.4/87.4 | 37.3/52.0 | 38.9/54.7 | |
| MBPTrack | ICCV'23 | 53.7/65.5 | 56.7/82.7 | 23.1/24.8 | 69.1/92.9 | 52.6/69.6 | 50.7/66.5 | |
| UAWTrack | AAAI'25 | 57.2/68.9 | 56.9/84.6 | 39.6/46.9 | 74.3/94.0 | 55.9/74.3 | 57.0/73.6 | |
| Fog | P2B | CVPR'20 | 34.3/41.4 | 36.3/58.5 | 20.6/23.2 | 28.7/39.4 | 33.8/47.1 | 30.0/40.6 |
| M2Track | CVPR'22 | 46.7/57.7 | 15.0/25.5 | 40.2/47.7 | 19.9/24.8 | 31.8/42.2 | 30.5/38.9 | |
| GLT-T | AAAI'23 | 58.4/69.1 | 21.5/38.0 | 47.0/56.8 | 32.7/50.4 | 40.9/54.1 | 53.7/53.6 | |
| MBPTrack | ICCV'23 | 66.5/77.6 | 23.6/39.4 | 59.6/69.1 | 59.7/84.0 | 47.2/60.5 | 52.3/67.5 | |
| UAWTrack | AAAI'25 | 72.8/85.6 | 29.1/51.0 | 62.6/74.7 | 66.4/90.1 | 52.8/69.8 | 57.7/75.3 |
| Weather Type | Tracker | Car | Pedestrian | Truck | Trailer | Bus | Mean | Mean by Category |
|---|---|---|---|---|---|---|---|---|
| Clear | P2B | 34.43/36.70 | 16.18/29.52 | 39.61/35.24 | 48.60/35.62 | 36.54/29.02 | 30.32/34.27 | 35.07/33.22 |
| M2Track | 47.59/55.65 | 24.77/50.16 | 42.78/41.93 | 44.42/34.40 | 42.78/41.93 | 40.21/51.27 | 39.38/42.60 | |
| GLT-T | 36.93/40.40 | 23.05/44.71 | 44.68/42.92 | 46.43/37.92 | 39.99/33.46 | 34.24/41.67 | 38.22/39.88 | |
| MBPTrack | 54.60/62.85 | 40.85/69.67 | 54.98/54.65 | 59.52/47.88 | 55.32/50.43 | 50.91/63.09 | 53.05/57.10 | |
| UAWTrack | 62.82/69.21 | 44.77/76.06 | 65.33/64.71 | 74.07/70.60 | 59.40/52.30 | 58.23/70.24 | 61.26/66.58 | |
| Rain | P2B | 34.34/36.51 | 16.32/29.84 | 39.49/35.25 | 49.63/37.18 | 36.74/28.84 | 30.33/34.30 | 35.30/33.52 |
| M2Track | 47.35/55.48 | 24.82/50.26 | 42.63/41.84 | 44.53/34.76 | 42.63/41.84 | 40.09/51.22 | 39.38/42.78 | |
| GLT-T | 36.40/39.80 | 22.95/44.34 | 44.55/42.81 | 46.73/38.82 | 39.58/32.95 | 33.91/41.23 | 38.04/39.74 | |
| MBPTrack | 53.91/60.89 | 40.71/69.42 | 54.84/54.62 | 59.81/48.04 | 55.16/50.08 | 55.16/50.08 | 52.89/56.61 | |
| UAWTrack | 62.80/69.19 | 44.72/76.06 | 65.32/64.62 | 73.90/70.41 | 59.14/52.10 | 58.19/70.21 | 61.18/66.48 | |
| Snow | P2B | 34.32/36.49 | 16.39/30.11 | 39.25/35.01 | 49.34/36.64 | 35.75/28.01 | 30.28/34.30 | 35.01/33.25 |
| M2Track | 47.33/55.45 | 24.76/50.22 | 42.58/41.72 | 44.59/34.84 | 42.58/41.72 | 40.05/51.17 | 39.30/42.62 | |
| GLT-T | 36.37/39.74 | 22.88/44.17 | 44.27/42.44 | 46.84/38.55 | 39.26/32.74 | 33.84/41.10 | 37.92/39.53 | |
| MBPTrack | 54.45/62.73 | 40.84/69.62 | 55.13/54.92 | 59.51/47.81 | 55.17/50.23 | 50.84/63.04 | 53.02/57.06 | |
| UAWTrack | 62.74/69.06 | 44.68/76.09 | 65.26/64.59 | 74.07/70.62 | 59.00/51.98 | 58.14/70.15 | 61.15/66.47 | |
| Fog | P2B | 31.51/32.48 | 15.88/29.41 | 39.24/35.99 | 45.70/35.79 | 35.03/27.53 | 28.47/31.99 | 33.47/32.24 |
| M2Track | 42.56/50.28 | 22.77/45.98 | 36.97/36.21 | 42.10/32.61 | 36.97/36.21 | 36.08/46.36 | 35.76/38.56 | |
| GLT-T | 34.29/37.63 | 21.34/41.15 | 40.68/37.82 | 41.81/32.25 | 35.17/28.92 | 31.60/38.28 | 34.66/35.55 | |
| MBPTrack | 54.21/61.51 | 37.21/63.37 | 54.36/53.79 | 60.33/50.22 | 52.67/47.59 | 49.55/60.47 | 51.76/55.30 | |
| UAWTrack | 62.03/68.20 | 40.78/70.04 | 63.90/63.71 | 73.87/70.49 | 56.03/49.65 | 55.94/67.80 | 59.32/64.42 |
This repo is heavily built upon MMDetection3D and SiamMo.
This repository is released under MIT License (see LICENSE file for details).