More general "Custom" solution + PoseTracker to also work without detection model + Fixed RTMO NMS + Fixed wholebody solution #45
Conversation
davidpagnon
commented
Jan 8, 2025
•
davidpagnon
commented
- Added the "Custom" class, a more general solution which works with any detection and/or pose estimation class
- Made PoseTracker work when no detection model is used
Readme and tiny edits
Co-authored-by: Tau <taujiang@outlook.com>
Co-authored-by: Tau <taujiang@outlook.com>
Co-authored-by: Tau <taujiang@outlook.com>
I don't think tracking was taken into account. Reordered keypoints, scores, according to track_id
…ydetection and/or pose estimation class - Made PoseTracker work with the Custom class and even when no detection mode is used
|
Unless I am mistaken, non maximum suppression did not work with RTMO. I edited postprocess in rtmo.py to make it work. I also had to slightly edit multiclass_nms in post_processing.py to output the indices of the bboxes to keep. |
davidpagnon
changed the title
|
Illustration of the previous issue with Whole body solution in balanced mode. rtmw-x_simcc-cocktail13_pt-ucoco_270e-256x192-fbef0d61_20230925 predicts toes on top of the head:
Replaced with rtmw-dw-x-l_simcc-cocktail14_270e-256x192_20231122 |
davidpagnon
changed the title
rtmlib/tools/pose_estimation/rtmo.py
Outdated
| # apply nms | ||
| dets, keep = multiclass_nms(final_boxes, | ||
| final_scores[:, np.newaxis], | ||
| nms_thr=0.45, | ||
| score_thr=0.7) | ||
| if keep is not None: | ||
| keypoints = keypoints[keep] | ||
| scores = scores[keep] | ||
| else: | ||
| keypoints = np.expand_dims(np.zeros_like(keypoints[0]), axis=0) | ||
| scores = np.expand_dims(np.zeros_like(scores[0]), axis=0) |
There was a problem hiding this comment.
I think nms_thr and score_thr can be used as parameters for __call__() and postprocess(), allowing users to conveniently set these two parameters when calling rtmo. What do you think?
There was a problem hiding this comment.
Certainly! Does my last commit correspond to your idea?
There was a problem hiding this comment.
Sorry one minute, I need to edit my commit
There was a problem hiding this comment.
It should be fine now. You can call it two different ways:
- keypoints, scores = rtmo(image)
- keypoints, scores = rtmo(image, nms_thr=0.5, score_thr=0.8)
setup.py
Outdated
| setup(name='rtmlib_pose2sim', | ||
| version=get_version(), | ||
| description='A library for real-time pose estimation.', | ||
| author='Tau-J', | ||
| author_email='taujiang@outlook.com', | ||
| author='Tau-J, David Pagnon', | ||
| author_email='taujiang@outlook.com, contact@david-pagnon.com', | ||
| keywords='pose estimation', | ||
| long_description=readme(), | ||
| long_description_content_type='text/markdown', | ||
| packages=find_packages(), | ||
| include_package_data=True, | ||
| url='https://github.com/Tau-J/rtmlib', | ||
| url='https://github.com/davidpagnon/rtmlib_Pose2Sim', #'https://github.com/Tau-J/rtmlib', | ||
| license='Apache License 2.0', |
There was a problem hiding this comment.
These information seem refer to your repo, would you mind modifying?
There was a problem hiding this comment.
Of course! I needed to push it to pypi so I gave it another temporary name. Back to the origins now!
