| 305 | 0 | 735 |
| 下载次数 | 被引频次 | 阅读次数 |
3D Gaussian Splatting作为一种新兴的3D表示方法，在复杂场景的重建和渲染中展现了卓越的性能。在3D Gaussian Splatting目标分割中，大多数方法需要重新训练3D Gaussian Splatting的内置分割属性，不仅耗时较长，还可能导致边界区域的分割结果模糊。为了解决这些问题，提出了一种以2D特征为引导的高效三维分割方法。该分割方法将2D视觉信息与3D空间信息相关联，通过2D提示分割出3D高斯函数表示的目标物体，整个流程无需通过第二次迭代梯度下降为每个高斯函数添加新的分割属性。具体来说，首先根据2D掩码为3D高斯函数分配初始分割标签，随后通过邻界细化算法，细化目标物体的边界，成功克服了边界模糊的挑战，显著提升了分割精度。实验结果表明，该方法能够实现高质量的三维分割，处理时间减少了80%以上，同时分割精度显著提升，mIoU提高了1百分点以上，mACC提高了0.3百分点。
Abstract：3D Gaussian Splatting, as an emerging 3D representation method, has demonstrated remarkable performance in complex scene reconstruction and rendering. In the context of 3D Gaussian Splatting object segmentation, most existing methods require retraining a dedicated segmentation property within the 3D Gaussian Splatting representation. This process is not only time-consuming but can also lead to ambiguous segmentation results, particularly in boundary regions. To address these issues, an efficient 3D segmentation method guided by 2D features is proposed. This method correlates 2D visual information with 3D spatial information to segment target objects represented by 3D Gaussians using 2D prompts. The entire process eliminates the need for a second iterative gradient descent to add new segmentation properties to each Gaussian. Specifically, it first assigns initial segmentation labels to the 3D Gaussians based on 2D masks. Subsequently, a neighboring boundary refinement algorithm is employed to refine the boundaries of the target objects, successfully overcoming the challenge of boundary blurring and significantly improving segmentation accuracy. Experimental results show that the proposed method achieves high-quality 3D segmentation, reduces processing time by over 80%,and significantly enhances segmentation accuracy, with mIoU increasing by more than 1 percentage point and mACC increasing by 0.3 percentage points.
[1] JATAVALLABHULA K M,KUWAJERWALA A,GU Q,et al.Conceptfusion:open-set multimodal 3d mapping[J].arXiv.2302.07241,2023.
[2] WANG J,FANG J,ZHANG X,et al.Gaussianeditor:editing 3d gaussians delicately with text instructions[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.Seattle:IEEE,2024:20902-20911.
[3] CHEN Y,CHEN Z,ZHANG C,et al.Gaussianeditor:Swift and controllable 3d editing with Gaussian splatting[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.Seattle:IEEE,2024:21476-21485.
[4] LIU K,ZHAN F,ZHANG J,et al.Weakly supervised 3d open-vocabulary segmentation[J].Advances in Neural Information Processing Systems,2023,36:53433-53456.
[5] WANG G,YE J,CHENG J,et al.SAM-Med3D-MoE:towards a non-forgetting segment anything model via mixture of experts for 3D medical image segmentation[C]//International conference on medical image computing and computer-assisted intervention.Marralesh:Springer,2024:552-561.
[6] MILDENHALL B,SRINIVASAN P P,TANCIK M,et al.Nerf:representing scenes as neural radiance fields for view synthesis[J].Communications of the ACM,2021,65(1):99-106.
[7] BARRON J T,MILDENHALL B,VERBIN D,et al.Mip-nerf 360:unbounded anti-aliased neural radiance fields[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.New Orleans:IEEE,2022:5470-5479.
[8] CHEN A,XU Z,GEIGER A,et al.Tensorf:Tensorial radiance fields[C]//European conference on computer vision.Tel Aviv:Springer,2022:333-350.
[9] SUN C,SUN M,CHEN H T.Direct voxel grid optimization:super-fast convergence for radiance fields reconstruction[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.New Orleans:IEEE,2022:5459-5469.
[10] KERBL B,KOPANAS G,LEIMKÜHLER T,et al.3D Gaussian splatting for real-time radiance field rendering[J].ACM Trans.Graph.,2023,42(4):139:1-139:14.
[11] CEN J,FANG J,YANG C,et al.Segment any 3d gaussians[C]//Proceedings of the AAAI conference on artificial intelligence.Philadelphia:AAAI,2025:1971-1979.
[12] YE M,DANELLJAN M,YU F,et al.Gaussian grouping:segment and edit anything in 3d scenes[C]//European conference on computer vision.Milan:Springer,2024:162-179.
[13] QIN M,LI W,ZHOU J,et al.Langsplat:3d language gaussian splatting[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.Seattle:IEEE,2024:20051-20060.
[14] HU X,WANG Y,FAN L,et al.SAGD:boundary-enhanced segment anything in 3D Gaussian via Gaussian decomposition[J].arXiv:2401.17857,2024.
[15] KIRILLOV A,MINTUN E,RAVI N,et al.Segment anything[C]//Proceedings of the IEEE/CVF international conference on computer vision.Paris:IEEE,2023:4015-4026.
[16] 王淼，黄智忠，何晖光，等.分割一切模型SAM的潜力与展望：综述[J].中国图象图形学报，2024,29(6):1479-1509.
[17] ZOU X,YANG J,ZHANG H,et al.Segment everything everywhere all at once[J].Advances in Neural Information Processing Systems,2023,36:19769-19782.
[18] CHEN X,TANG J,WAN D,et al.Interactive segment anything nerf with feature imitation[J].arXiv:2305.16233,2023.
[19] CEN J,ZHOU Z,FANG J,et al.Segment anything in 3d with nerfs[J].Advances in Neural Information Processing Systems,2023,36:25971-25990.
[20] KIM C M,WU M,KERR J,et al.Garfield:group anything with radiance fields[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.Seattle:IEEE,2024:21530-21539.
[21] FAN Z,WANG P,JIANG Y,et al.Nerf-sos:any-view self-supervised object segmentation on complex scenes[J].arXiv:2209.08776,2022.
[22] LIU Y,HU B,TANG C K,et al.Sanerf-hq:segment anything for nerf in high quality[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.Seattle:IEEE,2024:3216-3226.
[23] YING H,YIN Y,ZHANG J,et al.Omniseg3d:omniversal 3d segmentation via hierarchical contrastive learning[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.Seattle:IEEE,2024:20612-20622.
[24] 陈涛，杨启亮，陈寅.神经辐射场技术及应用综述[J].计算机辅助设计与图形学学报，2025,37(1):51-74.
[25] ZHI S,LAIDLOW T,LEUTENEGGER S,et al.In-place scene labelling and understanding with implicit scene representation[C]//Proceedings of the IEEE/CVF international conference on computer vision.Virtual:IEEE,2021:15838-15847.
[26] REN Z,AGARWALA A,RUSSELL B,et al.Neural volumetric object selection[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.New Orleans:IEEE,2022:6133-6142.
[27] TSCHERNEZKI V,LAINA I,LARLUS D,et al.Neural feature fusion fields:3d distillation of self-supervised 2d image representations[C]//2022 international conference on 3D vision (3DV).Prague:IEEE,2022:443-453.
[28] KOBAYASHI S,MATSUMOTO E,SITZMANN V.Decomposing nerf for editing via feature field distillation[J].Advances in Neural Information Processing Systems,2022,35:23311-23330.
[29] GOEL R,SIRIKONDA D,SAINI S,et al.Interactive segmentation of radiance fields[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.Vancouver:IEEE,2023:4201-4211.
[30] RADFORD A,KIM J W,HALLACY C,et al.Learning transferable visual models from natural language supervision[C]//International conference on machine learning.Virtual:ACM,2021:8748-8763.
[31] KERR J,KIM C M,GOLDBERG K,et al.Lerf:language embedded radiance fields[C]//Proceedings of the IEEE/CVF international conference on computer vision.Paris:IEEE,2023:19729-19739.
[32] WANG B,CHEN L,YANG B.Dm-nerf:3d scene geometry decomposition and manipulation from 2d images[J].arXiv:2208.07227,2022.
[33] FU X,ZHANG S,CHEN T,et al.Panoptic nerf:3d-to-2d label transfer for panoptic urban scene segmentation[C]//2022 international conference on 3D vision (3DV).Prague:IEEE,2022:1-11.
[34] STELZNER K,KERSTING K,KOSIOREK A R.Decomposing 3d scenes into objects via unsupervised volume segmentation[J].arXiv:2104.01148,2021.
[35] LIU X,CHEN J,YU H,et al.Unsupervised multi-view object segmentation using radiance field propagation[J].Advances in Neural Information Processing Systems,2022,35:17730-17743.
[36] LIU Y,HU B,HUANG J,et al.Instance neural radiance field[C]//Proceedings of the IEEE/CVF international conference on computer vision.Paris:IEEE,2023:787-796.
[37] VORA S,RADWAN N,GREFF K,et al.Nesf:neural semantic fields for generalizable semantic segmentation of 3d scenes[J].arXiv:2111.13260,2021.
[38] YU H X,GUIBAS L J,WU J.Unsupervised discovery of object radiance fields[J].arXiv:2107.07905,2021.
[39] LAN K,LI H,SHI H,et al.2d-guided 3d gaussian segmentation[C]//2024 Asian conference on communication and networks (ASIANComNet).Bangkok:IEEE,2024:1-5.
[40] ZHOU S,CHANG H,JIANG S,et al.Feature 3dgs:supercharging 3d Gaussian splatting to enable distilled feature fields[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.Seattle:IEEE,2024:21676-21685.
[41] SHEN Q,YANG X,WANG X.Flashsplat:2d to 3d gaussian splatting segmentation solved optimally[C]//European conference on computer vision.Cham:Springer,2024:456-472.
[42] 刘高屹，胡瑞珍，刘利刚.基于2D特征蒸馏的3D高斯泼溅语义分割与编辑[J].图学学报，2025,46(2):312-321.
[43] YIFAN W,SERENA F,WU S,et al.Differentiable surface splatting for point-based geometry processing[J].ACM Transactions On Graphics,2019,38(6):1-14.
[44] ZWICKER M,PFISTER H,VAN BAAR J,et al.Surface splatting[C]//Proceedings of the 28th annual conference on computer graphics and interactive techniques.Los Angeles:IEEE,2001:371-378.
[45] MILDENHALL B,SRINIVASAN P P,ORTIZ-CAYON R,et al.Local light field fusion:practical view synthesis with prescriptive sampling guidelines[J].ACM Transactions on Graphics,2019,38(4):1-14.
[46] 卢丽华，张晓辉，魏辉，等.以神经辐射场和三维高斯泼溅为基础的文本指导三维编辑综述[J].中国图象图形学报，2025,30(5):1238-1256.
[47] TANG S,PEI W,TAO X,et al.Scene-generalizable interactive segmentation of radiance fields[C]//Proceedings of the 31st ACM international conference on multimedia.Ottawa:ACM,2023:6744-6755.
基本信息:
DOI：10.20165/j.cnki.ISSN1673-629X.2025.0255
中图分类号:TP391.41
引用信息:
[1]朱雨馨,朱烨,魏敏,等.基于2D分割引导的3D高斯泼溅分割[J].计算机技术与发展,2026,36(03):68-76.DOI:10.20165/j.cnki.ISSN1673-629X.2025.0255.
基金信息:
四川省科技计划项目(2023YFQ0072)
2025-09-22
2025-09-22
2025-09-22