-
Notifications
You must be signed in to change notification settings - Fork 2
/
E2NeRF.html
174 lines (161 loc) · 7.77 KB
/
E2NeRF.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>E2NeRF</title>
<link rel="stylesheet" type="text/css" href="assets/scripts/bulma.min.css">
<link rel="stylesheet" type="text/css" href="assets/scripts/theme.css">
<link rel="stylesheet" type="text/css" href="https://cdn.bootcdn.net/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
</head>
<body>
<section class="hero is-light" style="">
<div class="hero-body" style="padding-top: 50px;">
<div class="container" style="text-align: center;margin-bottom:5px;">
<h1 class="title">
E<sup>2</sup>NeRF: Event Enhanced Neural Radiance Fields from Blurry Images
</h1>
<div class="author"><p style="font-size: 22px">Yunshan Qi<sup>1</sup>       Lin Zhu<sup>2 *</sup>       Yu Zhang<sup>3</sup>       Jia Li<sup>1,4 *</sup></p></div>
<div class="aff">
<p style="font-size: 20px"><sup>1</sup>Beihang University    <sup>2</sup>Beijing Institute of Technology    <sup>3</sup>SenseTime and Tetras.AI    <sup>4</sup>Peng Cheng Laboratory</p>
<p> </p>
</div>
<div class="group"><p style="font-size: 22px"><a href="http://cvteam.net/">CVTEAM</a></p></div>
<div class="con">
<p style="font-size: 24px; margin-top:5px; margin-bottom: 15px;">
<b>Accepted by International Conference on Computer Vision (ICCV) 2023</b>
</p>
</div>
<div class="columns">
<div class="column"></div>
<div class="column"></div>
<div class="column">
<a href="assets/E2NeRF/Papers.pdf" target="_blank">
<p class="link" style="font-size: 24px; margin-top:5px; margin-bottom: 15px;">[Paper]</p>
</a>
</div>
<div class="column">
<a href="https://github.com/iCVTEAM/E2NeRF" target="_blank">
<p class="link" style="font-size: 24px; margin-top:5px; margin-bottom: 15px;">[Code]</p>
</a>
</div>
<div class="column">
<a href="https://github.com/iCVTEAM/E2NeRF" target="_blank">
<p class="link" style="font-size: 24px; margin-top:5px; margin-bottom: 15px;">[Dataset]</p>
</a>
</div>
<div class="column">
<a href="https://cvteam.buaa.edu.cn/papers/2023-ICCV-Sup.pdf" target="_blank">
<p class="link" style="font-size: 24px; margin-top:5px; margin-bottom: 15px;">[Sup]</p>
</a>
</div>
<div class="column"></div>
<div class="column"></div>
</div>
</div>
</div>
</section>
<div style="text-align: center;">
<div class="head_cap">
<p style="font-size: 20px">
<b>The Framework of E<sup>2</sup>NeRF</b>
</p>
</div>
<div class="container" style="max-width:850px">
<div style="text-align: center;">
<img src="./assets/E2NeRF/method.png" class="centerImage">
</div>
</div>
</div>
<div style="text-align: center;">
<div class="head_cap">
<p style="font-size: 20px">
<b>The Results of E<sup>2</sup>NeRF</b>
</p>
</div>
<div class="container" style="max-width:850px">
<div style="text-align: center;">
<img src="./assets/E2NeRF/gif4.gif" class="centerImage">
</div>
</div>
</div>
<section class="hero">
<div class="hero-body">
<div class="container" style="max-width: 800px" >
<h1 style="">Abstract</h1>
<p style="text-align: justify; font-size: 16px;">
Neural Radiance Fields (NeRF) achieves impressive rendering performance by learning volumetric 3D representation from several images of different views.
However, it is difficult to reconstruct a sharp NeRF from blurry input as often occurred in the wild.
To solve this problem, we propose a novel Event-Enhanced NeRF (E<sup>2</sup>NeRF) by utilizing the combination data of a bio-inspired event camera and a standard RGB camera.
To effectively introduce event stream into the learning process of neural volumetric representation, we propose a blur rendering loss and an event rendering loss, which guide the network via modelling real blur process and event generation process, respectively.
Moreover, a camera pose estimation framework for real-world data is built with the guidance of event stream to generalize the method to practical applications.
In contrast to previous image-based or event-based NeRF, our framework effectively utilizes the internal relationship between events and images.
As a result, E<sup>2</sup>NeRF not only achieves image deblurring but also achieves high-quality novel view image generation.
Extensive experiments on both synthetic data and real-world data demonstrate that E<sup>2</sup>NeRF can effectively learn a sharp NeRF from blurry images, especially in complex and low-light scenes.
</p>
</div>
</div>
</section>
<section class="hero">
<div class="hero-body">
<div class="container" style="max-width: 800px" >
<h1 style="">Supplementary Video</h1>
<iframe width="800" height="450" src="https://www.youtube.com/embed/EvTHcLFX8yY" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
</div>
</div>
</section>
<section class="hero is-light" style="background-color:#FFFFFF;">
<div class="hero-body">
<div class="container" style="max-width:800px;margin-bottom:20px;">
<h1>
Qualitative Comparison on Synthetic Data
</h1>
</div>
<div class="container" style="max-width:800px">
<div style="text-align: center;">
<img src="./assets/E2NeRF/qualitative1.png" class="centerImage">
</div>
</div>
<div class="container" style="max-width:800px;margin-bottom:20px;">
<h1>
Qualitative Comparison on Real Data
</h1>
</div>
<div class="container" style="max-width:800px">
<div style="text-align: center;">
<img src="./assets/E2NeRF/qualitative2.png" class="centerImage">
</div>
</div>
</div>
</section>
<section class="hero" style="padding-top:0px;">
<div class="hero-body">
<div class="container" style="max-width:800px;">
<div class="card">
<!--header class="card-header">
<p class="card-header-title">
BibTex Citation
</p>
<a class="card-header-icon button-clipboard" style="border:0px; background: inherit;" data-clipboard-target="#bibtex-info" >
<i class="fa fa-copy" height="20px"></i>
</a>
</header>
<div class="card-content">
<pre style="background-color:inherit;padding: 0px;" id="bibtex-info">@article{Ma_Xia_Li_2021,
title={Pyramidal Feature Shrinking for Salient Object Detection},
volume={35},
url={https://ojs.aaai.org/index.php/AAAI/article/view/16331},
number={3},
journal={Proceedings of the AAAI Conference on Artificial Intelligence},
author={Ma, Mingcan and Xia, Changqun and Li, Jia},
year={2021},
month={May},
pages={2311-2318}
}</pre-->
</div>
</section>
<script type="text/javascript" src="assets/scripts/clipboard.min.js"></script>
<script>
new ClipboardJS('.button-clipboard');
</script>
</body>
</html>