+
+
+
+
+https://github.com/user-attachments/assets/5ee056e4-73a9-4fd8-bf60-59cae90d3dfc
+
+*(Compressed version due to GitHub size limits. See the full-quality video on our project page!)*
+
+**TRELLIS.2** is a state-of-the-art large 3D generative model (4B parameters) designed for high-fidelity **image-to-3D** generation. It leverages a novel "field-free" sparse voxel structure termed **O-Voxel** to reconstruct and generate arbitrary 3D assets with complex topologies, sharp features, and full PBR materials.
+
+
+## ✨ Features
+
+### 1. High Quality, Resolution & Efficiency
+Our 4B-parameter model generates high-resolution fully textured assets with exceptional fidelity and efficiency using vanilla DiTs. It utilizes a Sparse 3D VAE with 16× spatial downsampling to encode assets into a compact latent space.
+
+| Resolution | Total Time* | Breakdown (Shape + Mat) |
+| :--- | :--- | :--- |
+| **512³** | **~3s** | 2s + 1s |
+| **1024³** | **~17s** | 10s + 7s |
+| **1536³** | **~60s** | 35s + 25s |
+
+*Tested on NVIDIA H100 GPU.
+
+### 2. Arbitrary Topology Handling
+The **O-Voxel** representation breaks the limits of iso-surface fields. It robustly handles complex structures without lossy conversion:
+* ✅ **Open Surfaces** (e.g., clothing, leaves)
+* ✅ **Non-manifold Geometry**
+* ✅ **Internal Enclosed Structures**
+
+### 3. Rich Texture Modeling
+Beyond basic colors, TRELLIS.2 models arbitrary surface attributes including **Base Color, Roughness, Metallic, and Opacity**, enabling photorealistic rendering and transparency support.
+
+### 4. Minimalist Processing
+Data processing is streamlined for instant conversions that are fully **rendering-free** and **optimization-free**.
+* **< 10s** (Single CPU): Textured Mesh → O-Voxel
+* **< 100ms** (CUDA): O-Voxel → Textured Mesh
+
+
+## 🗺️ Roadmap
+
+- [x] Paper release
+- [x] Release image-to-3D inference code
+- [x] Release pretrained checkpoints (4B)
+- [x] Hugging Face Spaces demo
+- [ ] Release shape-conditioned texture generation inference code (Current schdule: before 12/24/2025)
+- [ ] Release training code (Current schdule: before 12/31/2025)
+
+
+## 🛠️ Installation
+
+### Prerequisites
+- **System**: The code is currently tested only on **Linux**.
+- **Hardware**: An NVIDIA GPU with at least 24GB of memory is necessary. The code has been verified on NVIDIA A100 and H100 GPUs.
+- **Software**:
+ - The [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit-archive) is needed to compile certain packages. Recommended version is 12.4.
+ - [Conda](https://docs.anaconda.com/miniconda/install/#quick-command-line-install) is recommended for managing dependencies.
+ - Python version 3.8 or higher is required.
+
+### Installation Steps
+1. Clone the repo:
+ ```sh
+ git clone -b main https://github.com/microsoft/TRELLIS.2.git --recursive
+ cd TRELLIS.2
+ ```
+
+2. Install the dependencies:
+
+ **Before running the following command there are somethings to note:**
+ - By adding `--new-env`, a new conda environment named `trellis2` will be created. If you want to use an existing conda environment, please remove this flag.
+ - By default the `trellis2` environment will use pytorch 2.6.0 with CUDA 12.4. If you want to use a different version of CUDA, you can remove the `--new-env` flag and manually install the required dependencies. Refer to [PyTorch](https://pytorch.org/get-started/previous-versions/) for the installation command.
+ - If you have multiple CUDA Toolkit versions installed, `CUDA_HOME` should be set to the correct version before running the command. For example, if you have CUDA Toolkit 12.4 and 13.0 installed, you can run `export CUDA_HOME=/usr/local/cuda-12.4` before running the command.
+ - By default, the code uses the `flash-attn` backend for attention. For GPUs do not support `flash-attn` (e.g., NVIDIA V100), you can install `xformers` manually and set the `ATTN_BACKEND` environment variable to `xformers` before running the code. See the [Minimal Example](#minimal-example) for more details.
+ - The installation may take a while due to the large number of dependencies. Please be patient. If you encounter any issues, you can try to install the dependencies one by one, specifying one flag at a time.
+ - If you encounter any issues during the installation, feel free to open an issue or contact us.
+
+ Create a new conda environment named `trellis2` and install the dependencies:
+ ```sh
+ . ./setup.sh --new-env --basic --flash-attn --nvdiffrast --nvdiffrec --cumesh --o-voxel --flexgemm
+ ```
+ The detailed usage of `setup.sh` can be found by running `. ./setup.sh --help`.
+ ```sh
+ Usage: setup.sh [OPTIONS]
+ Options:
+ -h, --help Display this help message
+ --new-env Create a new conda environment
+ --basic Install basic dependencies
+ --flash-attn Install flash-attention
+ --cumesh Install cumesh
+ --o-voxel Install o-voxel
+ --flexgemm Install flexgemm
+ --nvdiffrast Install nvdiffrast
+ --nvdiffrec Install nvdiffrec
+ ```
+
+## 📦 Pretrained Weights
+
+The pretrained model **TRELLIS.2-4B** is available on Hugging Face. Please refer to the model card there for more details.
+
+| Model | Parameters | Resolution | Link |
+| :--- | :--- | :--- | :--- |
+| **TRELLIS.2-4B** | 4 Billion | 512³ - 1536³ | [Hugging Face](https://huggingface.co/microsoft/TRELLIS.2-4B) |
+
+
+## 🚀 Usage
+
+### 1. Image to 3D Generation
+
+#### Minimal Example
+
+Here is an [example](example.py) of how to use the pretrained models for 3D asset generation.
+
+```python
+import os
+os.environ['OPENCV_IO_ENABLE_OPENEXR'] = '1'
+os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True" # Can save GPU memory
+import cv2
+import imageio
+from PIL import Image
+import torch
+from trellis2.pipelines import Trellis2ImageTo3DPipeline
+from trellis2.utils import render_utils
+from trellis2.renderers import EnvMap
+import o_voxel
+
+# 1. Setup Environment Map
+envmap = EnvMap(torch.tensor(
+ cv2.cvtColor(cv2.imread('assets/hdri/forest.exr', cv2.IMREAD_UNCHANGED), cv2.COLOR_BGR2RGB),
+ dtype=torch.float32, device='cuda'
+))
+
+# 2. Load Pipeline
+pipeline = Trellis2ImageTo3DPipeline.from_pretrained("microsoft/TRELLIS.2-4B")
+pipeline.cuda()
+
+# 3. Load Image & Run
+image = Image.open("assets/example_image/T.png")
+mesh = pipeline.run(image)[0]
+mesh.simplify(16777216) # nvdiffrast limit
+
+# 4. Render Video
+video = render_utils.make_pbr_vis_frames(render_utils.render_video(mesh, envmap=envmap))
+imageio.mimsave("sample.mp4", video, fps=15)
+
+# 5. Export to GLB
+glb = o_voxel.postprocess.to_glb(
+ vertices = mesh.vertices,
+ faces = mesh.faces,
+ attr_volume = mesh.attrs,
+ coords = mesh.coords,
+ attr_layout = mesh.layout,
+ voxel_size = mesh.voxel_size,
+ aabb = [[-0.5, -0.5, -0.5], [0.5, 0.5, 0.5]],
+ decimation_target = 1000000,
+ texture_size = 4096,
+ remesh = True,
+ remesh_band = 1,
+ remesh_project = 0,
+ verbose = True
+)
+glb.export("sample.glb", extension_webp=True)
+```
+
+Upon execution, the script generates the following files:
+ - `sample.mp4`: A video visualizing the generated 3D asset with PBR materials and environmental lighting.
+ - `sample.glb`: The extracted PBR-ready 3D asset in GLB format.
+
+**Note:** The `.glb` file is exported in `OPAQUE` mode by default. Although the alpha channel is preserved within the texture map, it is not active initially. To enable transparency, import the asset into your 3D software and manually connect the texture's alpha channel to the material's opacity or alpha input.
+
+#### Web Demo
+
+[app.py](app.py) provides a simple web demo for image to 3D asset generation. you can run the demo with the following command:
+```sh
+python app.py
+```
+
+Then, you can access the demo at the address shown in the terminal.
+
+### 2. PBR Texture Generation
+
+Will be released soon. Please stay tuned!
+
+## 🧩 Related Packages
+
+TRELLIS.2 is built upon several specialized high-performance packages developed by our team:
+
+* **[O-Voxel](o-voxel):**
+ Core library handling the logic for converting between textured meshes and the O-Voxel representation, ensuring instant bidirectional transformation.
+* **[FlexGEMM](https://github.com/JeffreyXiang/FlexGEMM):**
+ Efficient sparse convolution implementation based on Triton, enabling rapid processing of sparse voxel structures.
+* **[CuMesh](https://github.com/JeffreyXiang/CuMesh):**
+ CUDA-accelerated mesh utilities used for high-speed post-processing, remeshing, decimation, and UV-unwrapping.
+
+
+## ⚖️ License
+
+This model and code are released under the **[MIT License](LICENSE)**.
+
+Please note that certain dependencies operate under separate license terms:
+
+- [**nvdiffrast**](https://github.com/NVlabs/nvdiffrast): Utilized for rendering generated 3D assets. This package is governed by its own [License](https://github.com/NVlabs/nvdiffrast/blob/main/LICENSE.txt).
+
+- [**nvdiffrec**](https://github.com/NVlabs/nvdiffrec): Implements the split-sum renderer for PBR materials. This package is governed by its own [License](https://github.com/NVlabs/nvdiffrec/blob/main/LICENSE.txt).
+
+## 📚 Citation
+
+If you find this model useful for your research, please cite our work:
+
+```bibtex
+@article{
+ xiang2025trellis2,
+ title={Native and Compact Structured Latents for 3D Generation},
+ author={Xiang, Jianfeng and Chen, Xiaoxue and Xu, Sicheng and Wang, Ruicheng and Lv, Zelong and Deng, Yu and Zhu, Hongyuan and Dong, Yue and Zhao, Hao and Yuan, Nicholas Jing and Yang, Jiaolong},
+ journal={Tech report},
+ year={2025}
+}
+```
diff --git a/SECURITY.md b/SECURITY.md
new file mode 100644
index 0000000..e751608
--- /dev/null
+++ b/SECURITY.md
@@ -0,0 +1,14 @@
+
+
+## Security
+
+Microsoft takes the security of our software products and services seriously, which
+includes all source code repositories in our GitHub organizations.
+
+**Please do not report security vulnerabilities through public GitHub issues.**
+
+For security reporting information, locations, contact information, and policies,
+please review the latest guidance for Microsoft repositories at
+[https://aka.ms/SECURITY.md](https://aka.ms/SECURITY.md).
+
+
\ No newline at end of file
diff --git a/app.py b/app.py
new file mode 100644
index 0000000..b6dea25
--- /dev/null
+++ b/app.py
@@ -0,0 +1,645 @@
+import gradio as gr
+
+import os
+os.environ['OPENCV_IO_ENABLE_OPENEXR'] = '1'
+os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"
+from datetime import datetime
+import shutil
+import cv2
+from typing import *
+import torch
+import numpy as np
+from PIL import Image
+import base64
+import io
+from trellis2.modules.sparse import SparseTensor
+from trellis2.pipelines import Trellis2ImageTo3DPipeline
+from trellis2.renderers import EnvMap
+from trellis2.utils import render_utils
+import o_voxel
+
+
+MAX_SEED = np.iinfo(np.int32).max
+TMP_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'tmp')
+MODES = [
+ {"name": "Normal", "icon": "assets/app/normal.png", "render_key": "normal"},
+ {"name": "Clay render", "icon": "assets/app/clay.png", "render_key": "clay"},
+ {"name": "Base color", "icon": "assets/app/basecolor.png", "render_key": "base_color"},
+ {"name": "HDRI forest", "icon": "assets/app/hdri_forest.png", "render_key": "shaded_forest"},
+ {"name": "HDRI sunset", "icon": "assets/app/hdri_sunset.png", "render_key": "shaded_sunset"},
+ {"name": "HDRI courtyard", "icon": "assets/app/hdri_courtyard.png", "render_key": "shaded_courtyard"},
+]
+STEPS = 8
+DEFAULT_MODE = 3
+DEFAULT_STEP = 3
+
+
+css = """
+/* Overwrite Gradio Default Style */
+.stepper-wrapper {
+ padding: 0;
+}
+
+.stepper-container {
+ padding: 0;
+ align-items: center;
+}
+
+.step-button {
+ flex-direction: row;
+}
+
+.step-connector {
+ transform: none;
+}
+
+.step-number {
+ width: 16px;
+ height: 16px;
+}
+
+.step-label {
+ position: relative;
+ bottom: 0;
+}
+
+.wrap.center.full {
+ inset: 0;
+ height: 100%;
+}
+
+.wrap.center.full.translucent {
+ background: var(--block-background-fill);
+}
+
+.meta-text-center {
+ display: block !important;
+ position: absolute !important;
+ top: unset !important;
+ bottom: 0 !important;
+ right: 0 !important;
+ transform: unset !important;
+}
+
+/* Previewer */
+.previewer-container {
+ position: relative;
+ font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
+ width: 100%;
+ height: 722px;
+ margin: 0 auto;
+ padding: 20px;
+ display: flex;
+ flex-direction: column;
+ align-items: center;
+ justify-content: center;
+}
+
+.previewer-container .tips-icon {
+ position: absolute;
+ right: 10px;
+ top: 10px;
+ z-index: 10;
+ border-radius: 10px;
+ color: #fff;
+ background-color: var(--color-accent);
+ padding: 3px 6px;
+ user-select: none;
+}
+
+.previewer-container .tips-text {
+ position: absolute;
+ right: 10px;
+ top: 50px;
+ color: #fff;
+ background-color: var(--color-accent);
+ border-radius: 10px;
+ padding: 6px;
+ text-align: left;
+ max-width: 300px;
+ z-index: 10;
+ transition: all 0.3s;
+ opacity: 0%;
+ user-select: none;
+}
+
+.previewer-container .tips-text p {
+ font-size: 14px;
+ line-height: 1.2;
+}
+
+.tips-icon:hover + .tips-text {
+ display: block;
+ opacity: 100%;
+}
+
+/* Row 1: Display Modes */
+.previewer-container .mode-row {
+ width: 100%;
+ display: flex;
+ gap: 8px;
+ justify-content: center;
+ margin-bottom: 20px;
+ flex-wrap: wrap;
+}
+.previewer-container .mode-btn {
+ width: 24px;
+ height: 24px;
+ border-radius: 50%;
+ cursor: pointer;
+ opacity: 0.5;
+ transition: all 0.2s;
+ border: 2px solid #ddd;
+ object-fit: cover;
+}
+.previewer-container .mode-btn:hover { opacity: 0.9; transform: scale(1.1); }
+.previewer-container .mode-btn.active {
+ opacity: 1;
+ border-color: var(--color-accent);
+ transform: scale(1.1);
+}
+
+/* Row 2: Display Image */
+.previewer-container .display-row {
+ margin-bottom: 20px;
+ min-height: 400px;
+ width: 100%;
+ flex-grow: 1;
+ display: flex;
+ justify-content: center;
+ align-items: center;
+}
+.previewer-container .previewer-main-image {
+ max-width: 100%;
+ max-height: 100%;
+ flex-grow: 1;
+ object-fit: contain;
+ display: none;
+}
+.previewer-container .previewer-main-image.visible {
+ display: block;
+}
+
+/* Row 3: Custom HTML Slider */
+.previewer-container .slider-row {
+ width: 100%;
+ display: flex;
+ flex-direction: column;
+ align-items: center;
+ gap: 10px;
+ padding: 0 10px;
+}
+
+.previewer-container input[type=range] {
+ -webkit-appearance: none;
+ width: 100%;
+ max-width: 400px;
+ background: transparent;
+}
+.previewer-container input[type=range]::-webkit-slider-runnable-track {
+ width: 100%;
+ height: 8px;
+ cursor: pointer;
+ background: #ddd;
+ border-radius: 5px;
+}
+.previewer-container input[type=range]::-webkit-slider-thumb {
+ height: 20px;
+ width: 20px;
+ border-radius: 50%;
+ background: var(--color-accent);
+ cursor: pointer;
+ -webkit-appearance: none;
+ margin-top: -6px;
+ box-shadow: 0 2px 5px rgba(0,0,0,0.2);
+ transition: transform 0.1s;
+}
+.previewer-container input[type=range]::-webkit-slider-thumb:hover {
+ transform: scale(1.2);
+}
+
+/* Overwrite Previewer Block Style */
+.gradio-container .padded:has(.previewer-container) {
+ padding: 0 !important;
+}
+
+.gradio-container:has(.previewer-container) [data-testid="block-label"] {
+ position: absolute;
+ top: 0;
+ left: 0;
+}
+"""
+
+
+head = """
+
+"""
+
+
+empty_html = f"""
+● Render Mode - Click on the circular buttons to switch between different render modes.
+● View Angle - Drag the slider to change the view angle.
+