DreamCraft3D++: Efficient Hierarchical 3D Generation with Multi-Plane Reconstruction Model

Jingxiang Sun 1,   Cheng Peng 1,   Ruizhi Shao 1,   Yuan-Chen Guo 2,   Xiaochen Zhao 1,   Yangguang Li 2,   Yanpei Cao 2,   Bo Zhang 3,   Yebin Liu 1

1 Tsinghua University

,

2 VAST

,

3 Zhejiang University

Abstract

We present DreamCraft3D++, an enhanced iteration of DreamCraft3D, a multi-stage 3D generation framework that enables efficient, high-quality production of complex 3D assets. DreamCraft3D++ significantly improves upon its predecessor in both generation speed and quality, reducing creation time to 10 minutes — a 20-fold acceleration. While retaining the multi-stage generation process, DreamCraft3D++ introduces two key innovations: (1) A feed-forward multi-plane based reconstruction model replaces the time-consuming geometry sculpting optimization, achieving a 1000x speedup in this stage. (2) For texture refinement, we propose a novel training-free IP-Adapter module that dynamically selects embeddings based on camera position, enhancing texture and geometry consistency. This approach provides a 4x faster alternative to DreamCraft3D's DreamBooth fine-tuning. Compared to current LRM-based methods, DreamCraft3D++ achieves a substantial improvement in both texture and geometric quality. Extensive experiments across diverse datasets demonstrate DreamCraft3D++'s superior capability in generating creative 3D assets with intricate geometry and realistic 360° textures, outperforming state-of-the-art image-to-3D methods. To foster further advancements in 3D content creation, we will open-source the complete implementation of DreamCraft3D++.

Method Overview

Given a single input image, Dreamcraft3D++ processes it with multi-view diffusion models to generate orthogonal, consistent views and normal maps. A feed-forward sparse-view 3D reconstruction model then infers textured meshes from these multi-view images, using a convolutional U-Net to map the input to non-orthogonal planes, which are subsequently decoded into Flexicubes. Finally, a training-free object-aware diffusion prior enhances high-frequency geometry and texture details through score distillation.

Generated Textured Meshes

More Results

Exported Meshes

Mesh Animations

The following results are animated by Mixamo.

BibTeX

@article{sun2024dreamcraft3d++,
  title={DreamCraft3D++: Efficient Hierarchical 3D Generation with Multi-Plane Reconstruction Model}, 
  author={Sun, Jingxiang and Peng, Cheng and Shao, Ruizhi and Guo, Yuan-Chen and Zhao, Xiaochen and Li, Yangguang and Cao, Yanpei and Zhang, Bo and Liu, Yebin},
  journal={arXiv preprint arXiv:2410.12928},
  year={2024},
}
@article{sun2023dreamcraft3d,
  title={Dreamcraft3d: Hierarchical 3d generation with bootstrapped diffusion prior},
  author={Sun, Jingxiang and Zhang, Bo and Shao, Ruizhi and Wang, Lizhen and Liu, Wen and Xie, Zhenda and Liu, Yebin},
  journal={arXiv preprint arXiv:2310.16818},
  year={2023}
}