3D SCENEFLOWNET: SELF-SUPERVISED 3D SCENE FLOW ESTIMATION BASED ON GRAPH CNN

Yawen Lu; Yuhao Zhu; Guoyu Lu

doi:10.1109/ICIP42928.2021.9506286

3D SCENEFLOWNET: SELF-SUPERVISED 3D SCENE FLOW ESTIMATION BASED ON GRAPH CNN

Yawen Lu
, Yuhao Zhu
, Guoyu Lu

School of Computing

Binghamton University

Rochester Institute of Technology
University of Rochester

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

12 Scopus citations

Abstract

Despite deep learning approaches have achieved promising successes in 2D optical flow estimation, it is a challenge to accurately estimate scene flow in 3D space as point clouds are inherently lacking topological information. In this paper, we aim at handling the problem of self-supervised 3D scene flow estimation based on dynamic graph convolutional neural networks (GCNNs), namely 3D SceneFlowNet. To better learn geometric relationships among points, we introduce EdgeConv to learn multiple-level features in a pyramid from point clouds and a self-attention mechanism to apply the multi-level features to predict the final scene flow. Our trained model can efficiently process a pair of adjacent point clouds as input and predict a 3D scene flow accurately without any supervision. The proposed approach achieves superior performance on both synthetic ModelNet40 dataset and real LiDAR scans from KITTI Scene Flow 2015 datasets.

Original language	English
Title of host publication	2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings
Publisher	IEEE Computer Society
Pages	3647-3651
Number of pages	5
ISBN (Electronic)	9781665441155
DOIs	https://doi.org/10.1109/ICIP42928.2021.9506286
State	Published - 2021
Event	28th IEEE International Conference on Image Processing, ICIP 2021 - Anchorage, United States Duration: Sep 19 2021 → Sep 22 2021

Publication series

Name	Proceedings - International Conference on Image Processing, ICIP
Volume	2021-September

Conference

Conference	28th IEEE International Conference on Image Processing, ICIP 2021
Country/Territory	United States
City	Anchorage
Period	09/19/21 → 09/22/21

Keywords

3D scene flow estimation
3D scene understanding
Graph CNN
Self-supervised learning

Access to Document

10.1109/ICIP42928.2021.9506286

Cite this

Lu, Y., Zhu, Y., & Lu, G. (2021). 3D SCENEFLOWNET: SELF-SUPERVISED 3D SCENE FLOW ESTIMATION BASED ON GRAPH CNN. In 2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings (pp. 3647-3651). (Proceedings - International Conference on Image Processing, ICIP; Vol. 2021-September). IEEE Computer Society. https://doi.org/10.1109/ICIP42928.2021.9506286

@inproceedings{07187f1c6fd34cdb8cb2ca1acb2c994f,

title = "3D SCENEFLOWNET: SELF-SUPERVISED 3D SCENE FLOW ESTIMATION BASED ON GRAPH CNN",

abstract = "Despite deep learning approaches have achieved promising successes in 2D optical flow estimation, it is a challenge to accurately estimate scene flow in 3D space as point clouds are inherently lacking topological information. In this paper, we aim at handling the problem of self-supervised 3D scene flow estimation based on dynamic graph convolutional neural networks (GCNNs), namely 3D SceneFlowNet. To better learn geometric relationships among points, we introduce EdgeConv to learn multiple-level features in a pyramid from point clouds and a self-attention mechanism to apply the multi-level features to predict the final scene flow. Our trained model can efficiently process a pair of adjacent point clouds as input and predict a 3D scene flow accurately without any supervision. The proposed approach achieves superior performance on both synthetic ModelNet40 dataset and real LiDAR scans from KITTI Scene Flow 2015 datasets.",

keywords = "3D scene flow estimation, 3D scene understanding, Graph CNN, Self-supervised learning",

author = "Yawen Lu and Yuhao Zhu and Guoyu Lu",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE.; 28th IEEE International Conference on Image Processing, ICIP 2021 ; Conference date: 19-09-2021 Through 22-09-2021",

year = "2021",

doi = "10.1109/ICIP42928.2021.9506286",

language = "English",

series = "Proceedings - International Conference on Image Processing, ICIP",

publisher = "IEEE Computer Society",

pages = "3647--3651",

booktitle = "2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings",

}

Lu, Y, Zhu, Y & Lu, G 2021, 3D SCENEFLOWNET: SELF-SUPERVISED 3D SCENE FLOW ESTIMATION BASED ON GRAPH CNN. in 2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings. Proceedings - International Conference on Image Processing, ICIP, vol. 2021-September, IEEE Computer Society, pp. 3647-3651, 28th IEEE International Conference on Image Processing, ICIP 2021, Anchorage, United States, 09/19/21. https://doi.org/10.1109/ICIP42928.2021.9506286

3D SCENEFLOWNET: SELF-SUPERVISED 3D SCENE FLOW ESTIMATION BASED ON GRAPH CNN. / Lu, Yawen; Zhu, Yuhao; Lu, Guoyu.
2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings. IEEE Computer Society, 2021. p. 3647-3651 (Proceedings - International Conference on Image Processing, ICIP; Vol. 2021-September).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - 3D SCENEFLOWNET

T2 - 28th IEEE International Conference on Image Processing, ICIP 2021

AU - Lu, Yawen

AU - Zhu, Yuhao

AU - Lu, Guoyu

PY - 2021

Y1 - 2021

N2 - Despite deep learning approaches have achieved promising successes in 2D optical flow estimation, it is a challenge to accurately estimate scene flow in 3D space as point clouds are inherently lacking topological information. In this paper, we aim at handling the problem of self-supervised 3D scene flow estimation based on dynamic graph convolutional neural networks (GCNNs), namely 3D SceneFlowNet. To better learn geometric relationships among points, we introduce EdgeConv to learn multiple-level features in a pyramid from point clouds and a self-attention mechanism to apply the multi-level features to predict the final scene flow. Our trained model can efficiently process a pair of adjacent point clouds as input and predict a 3D scene flow accurately without any supervision. The proposed approach achieves superior performance on both synthetic ModelNet40 dataset and real LiDAR scans from KITTI Scene Flow 2015 datasets.

AB - Despite deep learning approaches have achieved promising successes in 2D optical flow estimation, it is a challenge to accurately estimate scene flow in 3D space as point clouds are inherently lacking topological information. In this paper, we aim at handling the problem of self-supervised 3D scene flow estimation based on dynamic graph convolutional neural networks (GCNNs), namely 3D SceneFlowNet. To better learn geometric relationships among points, we introduce EdgeConv to learn multiple-level features in a pyramid from point clouds and a self-attention mechanism to apply the multi-level features to predict the final scene flow. Our trained model can efficiently process a pair of adjacent point clouds as input and predict a 3D scene flow accurately without any supervision. The proposed approach achieves superior performance on both synthetic ModelNet40 dataset and real LiDAR scans from KITTI Scene Flow 2015 datasets.

KW - 3D scene flow estimation

KW - 3D scene understanding

KW - Graph CNN

KW - Self-supervised learning

UR - https://www.scopus.com/pages/publications/85125574675

U2 - 10.1109/ICIP42928.2021.9506286

DO - 10.1109/ICIP42928.2021.9506286

M3 - Conference contribution

T3 - Proceedings - International Conference on Image Processing, ICIP

SP - 3647

EP - 3651

BT - 2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings

PB - IEEE Computer Society

Y2 - 19 September 2021 through 22 September 2021

ER -

3D SCENEFLOWNET: SELF-SUPERVISED 3D SCENE FLOW ESTIMATION BASED ON GRAPH CNN

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this