cv

M.Sc.

Sun Yat-sen University, Guangdong, China

B.Eng.

Sun Yat-sen University, Guangdong, China

Skeleton-based Interaction Recognition

This project aimed to recognize and understand interactions involving multiple entities—such as human bodies, hands, objects, or other elements within a scene. Our goal was to unify the recognition tasks of person-peron, hand-hand, hand-object interactions, and even group activities.

• Led and played a key role in every aspect of the project, including method design, coding, conducting experiments, result analysis and manuscript writing.
• Proposed a model based on entity rearrangement and token attention, achieving SOTA performance across 4 benchmarks.
• Enabled the single-entity backbones (e.g., CTR-GCN) to achieve SOTA performance on 6 multi-entity action benchmarks in subsequent research.
• Published this work in IROS 2023, with subsequent research published in NeurIPS 2024.

Progressive Reasoning with World Models for Robotic Manipulation

This project built a robot manipulation simulator based on UE5, which can automatically generate robot manipulation demonstration data containing complex language instructions. With this simulator, we created a robot manipulation benchmark with progressive inference tasks, and proposed a world model-based robot manipulation method.

• Reviewed recent works related to robotic manipulation and deployed 3+ multimodal models (e.g., BC-Z, RT-1) using PyTorch.
• Trained multimodal models using reinforcement learning in the simulation environment to enable accurate comprehension of human language instructions and object grasping.

Facial Prior Guided Micro-Expression Generation

IEEE Transactions on Image Processing

CHASE: Learning Convex Hull Adaptive Shift for Skeleton-based Multi-Entity Action Recognition

Thirty-eighth Conference on Neural Information Processing Systems (NeurIPS)

Interactive Spatiotemporal Token Attention Network for Skeleton-Based General Interactive Action Recognition

2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Facial Prior Based First Order Motion Model for Micro-Expression Generation

Proceedings of the 29th ACM International Conference on Multimedia

Deep Learning Intern (Sony R&D Center)

Sony (China) Ltd.

• Assisted in developing deep learning models on human action analysis, based on skeletal data and RGB-D videos.

Algorithm Development Intern (Team Leader)

Hangzhou Lingxi Robot Intelligent Technology Co., Ltd. (LINX ROBOT)

• Completed a defect detection project utilizing few-shot learning, involving the review of 40+ research papers, archiving 16 open-source datasets, and successfully reproducing 4 key methods.
• Completed an image matching and retrieval project, collecting and annotating over 400 images, and fine-tuning DELF and CGD methods, resulting in a 10% increase in top-1 accuracy.

CVPR 2022 5th UG2+ Challenge Second Runner-up

CVPR 2022 5th UG2+ Workshop Organization Committee

It is awarded to our team (Titan5-HPL) in recognition of our solution submission to Challenge 2: Semi-supervied Action Recognition in the Dark.

ACMMM 2021 Facial Micro-Expression (FME) Challenge Winner

ACMMM 2021 Facial Micro-Expression (FME) Challenge Organization Committee

It is awarded to our team (Titan5-HPL) for ranking 1st in Facial Micro-Expression Generation Task.