MeshAnything:
Artist-Created Mesh Generation
with Autoregressive Transformers

Yiwen Chen^1,2*, Tong He^2†, Di Huang², Weicai Ye², Sijin Chen³, Jiaxiang Tang⁴
Xin Chen⁵, Zhongang Cai⁶, Lei Yang⁶, Gang Yu⁷, Guosheng Lin^1†, Chi Zhang^8†
*Work done during a research internship at Shanghai AI Lab.
^†Corresponding authors.
¹S-Lab, Nanyang Technological University, ²Shanghai AI Lab,
³Fudan University, ⁴Peking University, ⁵University of Chinese Academy of Sciences,
⁶SenseTime Research, ⁷Stepfun, ⁸Westlake University

                  ;

Release

• [6/17] 🔥🔥 We released the 350m version of MeshAnything.

Contents

• Release
• Contents
• Installation
• Usage
• Important Notes
• TODO

Installation

Our environment has been tested on Ubuntu 22, CUDA 11.8 with A100, A800 and A6000.

1. Clone our repo and create conda environment

git clone https://github.com/buaacyw/MeshAnything.git && cd MeshAnything
conda create -n MeshAnything python==3.10.13
conda activate MeshAnything
pip install torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt
pip install flash-attn --no-build-isolation

Usage

# Gradio Demo
python app.py

# Command line inference

# For mesh

# inference for folder
python main.py --input_dir examples --out_dir mesh_output --input_type mesh

# inference for single file
python main.py --input_dir examples/wand.ply --out_dir mesh_output --input_type mesh

# Preprocess with Marching Cubes first
python main.py --input_dir examples --out_dir mesh_output --input_type mesh --mc

# For point cloud

# Note: if you want to use your own point cloud, please make sure the normal is included.
# The file format should be a .npy file with shape (N, 6), where N is the number of points. The first 3 columns are the coordinates, and the last 3 columns are the normal.

# inference for folder
python main.py --input_dir pc_examples --out_dir pc_output --input_type pc_normal

# inference for single file
python main.py --input_dir pc_examples/mouse.npy --out_dir pc_output --input_type pc_normal

Important Notes

• The input mesh will be normalized to a unit bounding box. The up vector of the input mesh should be +Y for better results.
• Limited by computational resources, MeshAnything is trained on meshes with fewer than 800 faces and cannot generate meshes with more than 800 faces. The shape of the input mesh should be sharp enough; otherwise, it will be challenging to represent it with only 800 faces. Thus, feed-forward image-to-3D methods may often produce bad results due to insufficient shape quality.
• It takes about 7GB and 30s to generate a mesh on an A6000 GPU.
• Please refer to https://huggingface.co/spaces/Yiwen-ntu/MeshAnything/tree/main/examples for more examples.

TODO

The repo is still being under construction, thanks for your patience.

• Release of training code.
• Release of larger model.

Acknowledgement

Our code is based on these wonderful repos:

• MeshGPT
• meshgpt-pytorch
• Michelangelo
• transformers
• vector-quantize-pytorch