FineMotion: A Dataset and Benchmark with both Spatial and Temporal Annotation for Fine-grained Motion Generation and Editing

FineMotion provides strictly aligned, fine-grained descriptions of human body part movements (BPM) for both motion snippets (short segments of motion sequences) and entire motion sequences, with motions sourced from HumanML3D).

If this dataset is useful in your projects, we will apprecite your star on this codebase. 😆😆

Checkout Works on FineMotion

📌 (CVPR 2025) MG-MotionLLM: A Unified Framework for Motion Comprehension and Generation across Multiple Granularities [paper][code]

How to Obtain the Data

For motion sequences, you could find the details as well as download link here).

For our fine-grained textual descriptions, you could directly download them from this repository under the folder "./FineMotion".

Data Structure

sh <FineMotion> ./BPMP_auto.zip // Automatically-Generated Body Part Movement Paragraph. ./BPMSD_auto.zip // Automatically-Generated Body Part Snippet Descriptions. ./BPMP_human.zip // Human-Annotated Body Part Movement Paragraph. Coming Soon! ./BPMSD_human.zip // Human-Annotated Body Part Snippet Description. Coming Soon!

Unzip all files under the folder "./FineMotion" to obtain four JSON files: 'BPMPauto.json', 'BPMSDauto.json', 'BPMPhuman.json', and 'BPMSDhuman.json'. All JSON files use a dictionary format. The keys follow the same naming protocols as HumanML3D for consistency, while the values contain fine-grained textual descriptions of the corresponding motion sequences in HumanML3D. Specifically, the description in the value of the key "XXXXXX" (e.g., '000000') precisely describes the human motion stored in "./newjoints (or newjoint_vecs)/XXXXXX.npy" (e.g., '000000.npy') in HumanML3D.

An example of all the detailed human body part snippet descriptions (BPMSDs) in a whole human motion sequence is shown below: sh { "000314": # name of motion sequence [ "", # 0.0s-0.5s’ BPMSD "Raise your hands up to your head. Turn your head to the left.”, "", # 1.0s-1.5s’ BPMSD "", # 1.5s-2.0s’ BPMSD "Lean to the right. Move your hands closer together. Turn your head to the left.", "", # 2.5s-3.0s’ BPMSD "Lower your arms to your sides. Turn your head to the left.", "Lean your upper body to the right. Move your right hand to your right thigh. Move your left hand to your left thigh." ], } Each item in the list (i.e. the value of the key "000314") represents a fine-grained human body movement snippet description for a 0.5s interval.

An example for three different detailed human body part paragraphs (BPMPs) for the same human motion sequence is shown below: sh { "000314": # name of motion sequence [ "Beginning with his hands raised overhead and his head turned left, the individual leans to the right while bringing his hands closer and maintaining his head position. Continuing the movement, he lowers his arms to his sides while keeping his head turned left. Subsequently, he leans his upper body to the right and simultaneously moves his right hand to his right thigh and his left hand to his left thigh.", "Initially, the individual raises both hands above his head and turns his head to the left. He then leans to the right while bringing his hands closer together. Simultaneously, he maintains his head turned to the left. Subsequently, he lowers his arms to his sides and continues turning his head to the left. Finally, he leans his upper body to the right, placing his right hand on his right thigh and his left hand on his left thigh.", ”Initially, the individual raises his hands to his head and turns his head to the left. Subsequently, he leans to the right while continuing to move his hands closer together and turning his head to the left. Next, he lowers his arms to his sides and keeps turning his head to the left. Finally, he leans his upper body to the right, placing his right hand on his right thigh and his left hand on his left thigh." ], } Each item in the list (i.e. the value of key "000314") represents a fine-grained human body movement paragraph for the entire motion sequence.

Citation

If you are using FineMotion dataset, please consider citing this repository: @misc{Wu_FineMotion_Dataset, author = {Wu, Bizhu and Xie, Jinheng and Ding, Meidan and Kong, Zhe and Ren, Jianfeng and Bai, Ruibin and Qu, Rong and Shen, Linlin}, title = {{FineMotion Dataset}}, url = {https://github.com/BizhuWu/FineMotion} }

Acknowledgments

Thanks to HumanML3D, our repository is partially borrowing from them.