| 144 | 0 | 55 |
| 下载次数 | 被引频次 | 阅读次数 |
点云补全是从部分形状中补全完整的三维形状。已有多模态点云补全方法的模态融合过程都是以点云模态为主，缺乏对图像模态的充分利用。为了最大化利用图像提供的几何信息，生成具有足够几何细节的三维点云，该文提出了一种图像引导的双通道跨模态点云补全网络IgDccmNet。首先，通过编码器对不完整点云和完整点云的图像分别进行特征提取。其中，点云路径采用PointNet作为点云特征提取的骨干网络，图像路径采用ResNet18作为图像编码器。然后，为了实现点云与图像特征的深度交互学习，提出了双通道跨模态融合模块。通过自注意力机制强化各模态内部的特征关联性，通过交叉注意力机制建立跨模态间的语义关联，使图像和点云两种模态的特征能够在互补性信息指导下有效融合。最后，设计基于风格的点云解码器对全局特征解码生成预测点云，并与输入的不完整点云进行拼接，采用最远点采样得到完整点云。在ShapeNet-ViPC数据集上的实验结果表明，该方法相比其他主流方法，倒角距离降低21%～66%,F分数增加4%～78%。
Abstract：Point cloud completion refers to the process of reconstructing a complete 3D shape from a partial shape. Existing multimodal point cloud completion methods primarily focus on the point cloud modality, often underutilizing the information provided by the image modality. To maximize the geometric information provided by images and generate 3D point clouds with sufficient geometric details, we propose an image-guided dual-channel cross-modal point cloud completion network, IgDccmNet. Firstly, an encoder is employed to extract features from the incomplete point cloud and the corresponding image of the complete point cloud. In the point cloud path, PointNet is used as the backbone network for point cloud feature extraction, while ResNet18 is used as the image encoder. Then, to achieve deep interaction and learning between point cloud and image features, a dual-channel cross-modal fusion module is proposed. This module strengthens the internal feature correlation of each modality via a self-attention mechanism, and establishes semantic correlations between modalities using a cross-attention mechanism, enabling effective fusion of point cloud and image features under the guidance of complementary information. Finally, a style-based point cloud decoder is designed to decode the global features and generate the predicted point cloud, which is then fused with the input incomplete point cloud. Farthest Point Sampling(FPS) is applied to obtain the completed point cloud. Experiments conducted on the ShapeNet-ViPC dataset show that the proposed method outperformsother state-of-the-art methods, achieving a 21% to 66% reduction in chamfer distance and a 4% to 78% increase in F-score.
[1] GUO Y,WANG H,HU Q,et al.Deep learning for 3d point clouds:a survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,43(12):4338-4364.
[2] 柴玉晶，梁坤豪，杨历省，等.用于点云语义分割的局部特征增强网络[J].计算机技术与发展，2025,35(3):49-55.
[3] 雷根华，王蕾，张志勇.基于Feature-RNet的三维大场景点云分类框架[J].计算机技术与发展，2022,32(6):85-91.
[4] 高焱，熊风光.基于深度学习的点云目标检测方法[J].计算机技术与发展，2022,32(3):76-83.
[5] BOUD A C,HANIFF D J,BABER C,et al.Virtual reality and augmented reality as a training tool for assembly tasks[C]//1999 IEEE international conference on information visualization (Cat.No.PR00210).London:IEEE,1999:32-36.
[6] LIU Y,WU Xiaoqun.3D shape completion via deep learning:a method survey[J].Journal of Graphics,2023,44(2):201-215.
[7] YU X M,RAO Y M,WANG Z Y,et al.PoinTr:diverse point cloud completion with geometry-aware transformers[C]//2021 IEEE/CVF international conference on computer vision (ICCV).Montreal:IEEE,2022:12478-12487.
[8] NAN L,SHARF A,CHEN B.2D-D lifting for shape reconstruction[C]//Computer graphics forum.[s.l.]:Wiley-Blackwell Ltd,2014:249-258.
[9] LIU Z,LIN Y,CAO Y,et al.Swin transformer:hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF international conference on computer vision.[s.l.]:IEEE,2021:10012-10022.
[10] ZHANG X,FENG Y,LI S,et al.View-guided point cloud completion[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.Nashville:IEEE,2021:15890-1589.
[11] ZHU Z,NAN L,XIE H,et al.CSDN:cross-modal shape-transfer dual-refinement network for point cloud completion[J].IEEE Transactions on Visualization and Computer Graphics,2023,30(7):3545-3563.
[12] 何星，朱哲，燕雪峰，等.融合图像信息的跨模态Transformer点云补全算法[J].计算机辅助设计与图形学学报，2024,36(7):1026-1033.
[13] CHARLES R Q,HAO S,MO K C,et al.PointNet:deeplearning on point sets for 3D classification and segmentation[C]//2017 IEEE conference on computer vision and pattern recognition (CVPR).Honolulu:IEEE,2017:77-85.
[14] YUAN W T,KHOT T,HELD D,et al.PCN:point completion network[C]//2018 international conference on 3D vision (3DV).Verona:IEEE,2018:728-737.
[15] YANG Y,FENG C,SHEN Y,et al.Foldingnet:point cloud auto-encoder via deep grid deformation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.Washington:IEEE,2018:206-215.
[16] GROUEIX T,FISHER M,KIM V G,et al.A papier-m??ché approach to learning 3d surface generation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition(CVPR).Salt Lake City:IEEE,2018:216-224.
[17] LIU M,SHENG L,YANG S,et al.Morphing and sampling network for dense point cloud completion[C]//Proceedings of the AAAI conference on artificial intelligence.Menlo Park:AAAI,2020:11596-11603.
[18] TCHAPMI L P,KOSARAJU V,REZATOFIGHI H,et al.TopNet:structural point cloud decoder[C]//2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR).Long Beach:IEEE,2019:383-392.
[19] XIE H,YAO H,ZHOU S,et al.Grnet:gridding residual network for dense point cloud completion[C]//Proceedings of the European conference on computer vision (ECCV).Glasgow:Springer,2020:365-381.
[20] PAN L.ECG:edge-aware point cloud completion with graph convolution[J].IEEE Robotics and Automation Letters,2020,5(3):4392-4398.
[21] HUANG Z T,YU Y K,XU J W,et al.PF-net:point fractal network for 3D point cloud completion[C]//2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR).Seattle:IEEE,2020:7659-7667.
[22] XIANG P,WEN X,LIU Y S,et al.Snowflakenet:point cloud completion by snowflake point deconvolution with skip-transformer[C]//Proceedings of the IEEE/CVF international conference on computer vision.Montreal:IEEE,2021:5499-5509.
[23] ZHOU H,CAO Y,CHU W,et al.Seedformer:patch seeds based point cloud completion with upsample transformer[C]//European conference on computer vision.Cham:Springer,2022:416-432.
[24] CHEN Z,LONG F,QIU Z,et al.Anchorformer:point cloud completion from discriminative nodes[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.Vancouver:IEEE,2023:13581-13590.
[25] GU A,DAO T.Mamba:linear-time sequence modeling with selective state spaces[J].arXiv:2312.00752,2023.
[26] LI Y,YANG W,FEI B.3dmambacomplete:exploring structured state space model for point cloud completion[J].arXiv:2404.07106,2024.
[27] HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.Las Vegas:IEEE,2016:770-778.
[28] MA J,TANG L,FAN F,et al.SwinFusion:cross-domain long-range learning for general image fusion via swin transformer[J].IEEE/CAA Journal of Automatica Sinica,2022,9(7):1200-1217.
[29] CHOY C B,XU D,GWAK J Y,et al.3d-r2n2:a unified approach for single and multi-view 3d object reconstruction[C]//Computer vision–ECCV 2016:14th European conference(ECCV).Amsterdam:Springer,2016:628-644.
[30] WANG J,CUI Y,GUO D,et al.Pointattn:you only need attention for point cloud completion[C]//Proceedings of the AAAI conference on artificial intelligence(AAAI).Menlo Park:AAAI,2024:5472-5480.
基本信息:
DOI：10.20165/j.cnki.ISSN1673-629X.2025.0237
中图分类号:TP391.41
引用信息:
[1]杜晓飞,高宏娟,王帅杰.IgDccmNet:图像引导的双通道跨模态点云补全网络[J].计算机技术与发展,2026,36(02):38-45.DOI:10.20165/j.cnki.ISSN1673-629X.2025.0237.
基金信息:
宁夏回族自治区重点研发项目(2023BDE03006)
2025-09-23
2025-09-23
2025-09-23