Skip to yearly menu bar Skip to main content


Poster

Pose and Interaction Aware Human Object Interaction Image Generation

zhu xu · Qingchao Chen · Yuxin Peng · Yang Liu


Abstract:

Recent text-to-image generative models have demonstrated remarkable abilities in generating realistic images.Despite their great success, these models struggle to generate high-fidelity images with prompts oriented toward human-object interaction (HOI). The difficulty in HOI generation arises from two aspects. Firstly, the complexity and diversity of human poses challenge plausible human generation. Furthermore, untrustworthy generation of interaction boundary regions may lead to deficiency in HOI semantics. To tackle the problems, we propose a Pose and Interaction Aware HOI generation framework PIA-HOI. It utilizes human Pose quality and Interaction boundary region information as Guidance for denoising process, thereby encouraging refinement in these regions to produce more reasonable HOI images. Based on it, we establish an Iterative inversion and Image Refinement pipeline to continually enhance generation quality. Further, we introduce a comprehensive benchmark for HOI generation, which comprises a dataset involving diverse and fine-grained HOI categories, along with multiple custom-tailored evaluation metrics for HOI generation. Experiments demonstrate that our method significantly improves generation quality under both HOI-specific and conventional image evaluation metrics.

Live content is unavailable. Log in and register to view live content