Recent Advances in Generative AI Allow Users to Create 3D Models From Text or Images. However, These Models Prioritize Visual Plausibility Over Geometric Accuracy, Often Generating Results with Flaws That Compromise Their Intended Use Post-fabrication. We Present InstructMesh, an Interactive Post-generation Refinement Tool That Enables Selective Repair of Generative 3D Models Through Region Selection and Targeted Operations, Such As Opening or Sealing Voids, or Adjusting Local Thickness. Users Can Invoke Edit Operations Via Natural Language Prompts or Slider Controls. By Operating Directly on the Intermediate Latent Representation, InstructMesh Allows Users to Apply Robust Geometric Corrections Without Requiring Expert Modeling Skills. To Inform Our Design, We First Analyze Common Fabrication-related Failure Modes in Outputs From State-of-the-art Generative Tools. We Then Conduct Two User Studies, Demonstrating That Novices Can Identify and Perform Fabrication-relevant Repairs on Generative Outputs Using InstructMesh, and Revealing User Preference for Hybrid Interfaces That Combine Slider Controls with Natural Language Input

Recent advances in generative AI allow users to create 3D models from text or images. However, these models prioritize visual plausibility over geometric accuracy, often generating results with flaws that compromise their intended use post-fabrication. We present InstructMesh, an interactive post-generation refinement tool that enables selective repair of generative 3D models through region selection and targeted operations, such as opening or sealing voids, or adjusting local thickness. Users can invoke edit operations via natural language prompts or slider controls. By operating directly on the intermediate latent representation, InstructMesh allows users to apply robust geometric corrections without requiring expert modeling skills. To inform our design, we first analyze common fabrication-related failure modes in outputs from state-of-the-art generative tools. We then conduct two user studies, demonstrating that novices can identify and perform fabrication-relevant repairs on generative outputs using InstructMesh, and revealing user preference for hybrid interfaces that combine slider controls with natural language input.

Publications

teaser image of InstructMesh: Selective Refinement of Generative 3D Models for Fabrication

InstructMesh: Selective Refinement of Generative 3D Models for Fabrication

Proceedings of the 39th Annual ACM Symposium on User Interface Software and Technology (UIST), 2026.
Keywords: Personal Fabrication; Generative AI; 3D modeling



Videos

Talks

Cited By

Stay In Touch