詹小雨 Xiaoyu Zhan

2025

1. 
   

   

   Make PBR Materials Tileable with Latent Diffusion Inpainting

   Xiaoyu Zhan, Jianxin Yang, Jun Wang, Yuanqi Li, Jie Guo, and Yanwen Guo*

   IEEE Transactions on Visualization and Computer Graphics (TVCG), 2025

   Abs Bib PDF

                                                       
   @ARTICLE{10982120,
       author={Zhan, Xiaoyu and Yang, Jianxin and Wang, Jun and Li, Yuanqi and Guo, Jie and Guo, Yanwen},
       journal={IEEE Transactions on Visualization and Computer Graphics}, 
       title={Make PBR Materials Tileable with Latent Diffusion Inpainting}, 
       year={2025},
       volume={},
       number={},
       pages={1-13},
       keywords={Diffusion models;Image resolution;Pipelines;Image color analysis;Data mining;Neural networks;Image reconstruction;Training;Optimization;Lighting;Physically-based-rendering;Tileable Material Synthesis;Diffusion Model;Image Inpainting},
       doi={10.1109/TVCG.2025.3566315}
   }
       
   
   

   Physically-based-rendering (PBR) materials are crucial in modern rendering pipelines, and many studies have focused on acquiring these materials from reality or images. However, existing methods may result in non-tileable results, since the realistic inputs usually have seams. Compared to non-tileable materials, tileable PBR materials have more universal application scenarios. To address this issue, we introduce MaTi, a novel pipeline that converts non-tileable PBR materials into tileable ones with minimal distortion. MaTi rearranges material patches to align boundaries at the center of the image, and then uses a diffusion model to inpaint the seams. We use scaled gamma correction to reduce the occurrence of collapse when processing special material maps. The color correction and triangular blending are adopt to preserve the original material information. Additionally, we design a division and blending strategy to efficiently handle high resolution materials. Our experiments demonstrate that MaTi can seamlessly modify PBR materials while preserving the original information, outperforming existing synthesis methods.

2024

1. 
   

   

   LiDAR-Net: A Real-scanned 3D Point Cloud Dataset for Indoor Scenes

   Yanwen Guo*, Yuanqi Li, Dayong Ren, Xiaohong Zhang, Jiawei Li, Liang Pu, Changfeng Ma, Xiaoyu Zhan, Jie Guo, Mingqiang Wei, Yan Zhang, Piaopiao Yu, Shuangyu Yang, Donghao Ji, Huisheng Ye, Hao Sun, Yansong Liu, Yinuo Chen, Jiaqi Zhu, and Hongyu Liu

   Conference on Computer Vision and Pattern Recognition (CVPR), 2024

   Abs Bib HTML

                                                       
   @inproceedings{guo2024lidarnet,
       title     = {LiDAR-Net: A Real-scanned 3D Point Cloud Dataset for Indoor Scenes},
       author    = {Guo, Yanwen and Li, Yuanqi and Ren, Dayong and Zhang, Xiaohong and Li, Jiawei and Pu, Liang and Ma, Changfeng and Zhan, Xiaoyu and Guo, Jie and Wei, Mingqiang and Zhang, Yan and Yu, Piaopiao and Yang, Shuangyu and Ji, Donghao and Ye, Huisheng and Sun, Hao and Liu, Yansong and Chen, Yinuo and Zhu, Jiaqi and Liu, Hongyu},
       booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
       year      = {2024},
   }
   
   

   In this paper, we present LiDAR-Net, a new real-scanned indoor point cloud dataset, containing nearly 3.6 billion precisely point-level annotated points, covering an expansive area of 30,000m2. It encompasses three prevalent daily environments, including learning scenes, working scenes, and living scenes. LiDAR-Net is characterized by its non-uniform point distribution, e.g., scanning holes and scanning lines. Additionally, it meticulously records and annotates scanning anomalies, including reflection noise and ghost. These anomalies stem from specular reflections on glass or metal, as well as distortions due to moving persons. LiDAR-Net's realistic representation of non-uniform distribution and anomalies significantly enhances the training of deep learning models, leading to improved generalization in practical applications. We thoroughly evaluate the performance of state-of-the-art algorithms on LiDAR-Net and provide a detailed analysis of the results. Crucially, our research identifies several fundamental challenges in understanding indoor point clouds, contributing essential insights to future explorations in this field.

1. 
   

   

   Semantic Human Mesh Reconstruction with Textures

   Xiaoyu Zhan, Jianxin Yang, Yuanqi Li, Jie Guo, Yanwen Guo*, and Wenping Wang

   Oral

   Conference on Computer Vision and Pattern Recognition (CVPR), 2024

   Abs Bib HTML PDF Code

                                                       
   @inproceedings{zhan2024shert,
       title     = {Semantic Human Mesh Reconstruction with Textures},
       author    = {Zhan, Xiaoyu and Yang, Jianxin and Li, Yuanqi and Guo, Jie and Guo, Yanwen and Wang, Wenping},
       booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
       year      = {2024},
   }
   
   

   The field of 3D detailed human mesh reconstruction has made significant progress in recent years. However, current methods still face challenges when used in industrial applications due to unstable results, low-quality meshes, and a lack of UV unwrapping and skinning weights. In this paper, we present SHERT, a novel pipeline that can reconstruct semantic human meshes with textures and high-precision details. SHERT applies semantic and normal-based sampling between the detailed surface (e.g. mesh and SDF) and the corresponding SMPL-X model to obtain a partially sampled semantic mesh and then generates the complete semantic mesh by our specifically designed self-supervised completion and refinement networks. Using the complete semantic mesh as a basis, we employ a texture diffusion model to create human textures that are driven by both images and texts. Our reconstructed meshes have stable UV unwrapping, high-quality triangle meshes, and consistent semantic information. The given SMPL-X model provides semantic information and shape priors, allowing SHERT to perform well even with incorrect and incomplete inputs. The semantic information also makes it easy to substitute and animate different body parts such as the face, body, and hands. Quantitative and qualitative experiments demonstrate that SHERT is capable of producing high-fidelity and robust semantic meshes that outperform state-of-the-art methods.