image registration related books, papers, videos, and toolboxes
Many thanks to yzhao062 Anomaly Detection Learning Resources. I followed his style to make this depository.
Image registration is the process of transforming different sets of data into one coordinate system. Data may be multiple photographs, and from different sensors, times, depths, or viewpoints.
It is used in computer vision, medical imaging, military automatic target recognition, compiling and analyzing images and data from satellites. Registration is necessary in order to be able to compare or integrate the data obtained from different measurements.
This repository collects:
- Books & Academic Papers
- On-line Courses and Videos
- Datasets
- Open-source and Commercial Libraries/Toolkits
- Key Conferences & Journals
[toc]
More items will be added to the repository. Please feel free to suggest other key resources by opening an issue report, submitting a pull request, or dropping me an email @ ([email protected]). Enjoy reading!
Multiple view geometry in computer vision by Richard Hartley and Andrew Zisserman, 2004: Mathematic and geometric basis for 2D-2D and 2D-3D registration. A must-read for people in the field of registration. E-book
Computer Vision: A Modern Approach by David A. Forsyth, Jean Ponce: for upper-division undergraduate- and graduate-level courses in computer vision found in departments of Computer Science, Computer Engineering and Electrical Engineering.
Algebra, Topology, Differential Calculus, and Optimization Theory For Computer Science and Engineering by Jean Gallier and Jocelyn Quaintance. The latest book from upenn about the algebra and optimization theory.
Three-Dimensional Computer vision-A Geometric Viewpoint Classical 3D computer vision textbook.
An invitation to 3D vision a self-contained introduction to the geometry of three-dimensional (3-D) vision.
Zhenhuan Zhou, et.al: A software guide for medical image segmentation and registration algorithm. 医学图像分割与配准(ITK实现分册) Part Ⅱ introduces the most basic network and architecture of medical registration algorithms (Chinese Version).
2-D and 3-D Image Registration for Medical, Remote Sensing, and Industrial Applications by A. Ardeshir Goshtasby
医学图像配准技术与应用 by 吕晓琪
Intensity-based 2D-3D Medical Image Registration by Russakoff, Daniel
Biomedical Image Registration by Fischer, Dawant, Lorenz
Medical Image Registration by Hajnal, Joseph V.
Deep Learning for Medical Image Analysis (part IV)
14 lectures on visual SLAM By Xiang Gao and Tao Zhang and Yi Liu and Qinrui Yan. 视觉SLAM十四讲 视觉配准方向较易懂的入门教材。通俗讲述视觉匹配的物理模型， 数学几何基础，优化过程等。 新手必读。 [github] [Videos]
点云数据配准及曲面细分技术 by 薛耀红, 赵建平, 蒋振刚, 等 书籍内容比较过时，仅适合零基础读者阅读。推荐自行查找相关博客学习。
2-D and 3-D Image Registration: For Medical, Remote Sensing, and Industrial Applications by A. A. Goshtasby, 2005.
Big thanks to Yipeng Hu organizing the excellent tutorial.
Description:
Medical image registration has been a cornerstone in the research fields of medical image computing and computer assisted intervention, responsible for many clinical applications. Whilst machine learning methods have long been important in developing pairwise algorithms, recently proposed deep-learning-based frameworks directly infer displacement fields without iterative optimization for unseen image pairs, using neural networks trained from large population data. These novel approaches promise to tackle several most challenging aspects previously faced by classical pairwise methods, such as high computational cost, robustness for generalization and lack of inter-modality similarity measures.
Output from several international research groups working in this area include award-winning conference presentations, high-impact journal publications, well-received open-source implementations and industrial-partnered translational projects, generating significant interests to all levels of world-wide researchers. Accessing to the experience and expertise in this inherently multidisciplinary topic can be beneficial to many in our community, especially for the next generation of young scientists, engineers and clinicians who often have only been exposed to a subset of these methodologies and applications.
We organize a tutorial including both theoretical and practical sessions, inviting expert lectures and tutoring coding for real-world examples. Three hands-on sessions guiding participants to understand and implement published algorithms using clinical imaging data. This aims to provide an opportunity for the participants to bridge the gap between expertises in medical image registration and deep learning, as well as to start a forum to discuss know-hows, challenges and future opportunities in this area.
[kaggle:2016] Image registration, the R way, (almost) from scratch
There are some packages in R for image manipulation and after some test I select “imager” , based on the CImg C++, fast and providing several image processing tools.
- [kaggle:2018] X-Ray Patient Scan Registration
SimpleITK, ITK, scipy, OpenCV, Tensorflow and PyTorch all offer tools for registering images, we explore a few here to see how well they work when applied to the fairly tricky problem of registering from the same person at different time and disease points.
-
[MICCAI2019] Autograd Image Registration Laboratory
图像配准会议介绍@ MICCAI2019 / CVPR2019 / ICCV2019 / NeurIPS2019
Image Registration: From SIFT to Deep Learning
16-822: Geometry-based Methods in Vision
[VALSE 2018] Talk: 2017以来的2D to 3D by 吴毅红
WBIR - International Workshop on Biomedical Image Registration
WBIR2018, Leiden, Netherlands WBIR2016, Las Vegas NV WBIR2014, London, UK
[C++] [Python] OpenCV: OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.
[C++] PCL: Point Cloud Library. The Point Cloud Library (PCL) is a standalone, large scale, open project for 2D/3D image and point cloud processing.
[C++] Ceres Solver: Ceres Solver is an open source C++ library for modeling and solving large, complicated optimization problems. It can be used to solve Non-linear Least Squares problems with bounds constraints and general unconstrained optimization problems.
[C++] Open3D: Open3D is an open-source library that supports rapid development of software that deals with 3D data. The Open3D frontend exposes a set of carefully selected data structures and algorithms in both C++ and Python. The backend is highly optimized and is set up for parallelization.
[c++] ITK: Segmentation & Registration Toolkit
An open-source, cross-platform system that provides developers with an extensive suite of software tools for image analysis. Developed through extreme programming methodologies. ITK employs leading-edge algorithms for registering and segmenting multidimensional data.
[c++] [Python] [Java] SimpleITK: a simplified layer built on top of ITK.
[c++] ANTs: Advanced Normalization Tools.
Image registration with variable transformations (elastic, diffeomorphic, diffeomorphisms, unbiased) and similarity metrics (landmarks, cross-correlation, mutual information, etc.). Image segmentation with priors & nonparametric, multivariate models.
[c++] Elastix: open source software, based on the well-known ITK .
The software consists of a collection of algorithms that are commonly used to solve (medical) image registration problems. [manual]
[C++] [Python] [Java] [R] [Ruby] [Lua] [Tcl] [C#] SimpleElastix: a medical image registration library that makes state-of-the-art image registration really easy to do in languages like Python, Java and R.
3D slicer : an open source software platform for medical image informatics, image processing, and three-dimensional visualization. Built over two decades through support from the National Institutes of Health and a worldwide developer community, Slicer brings free, powerful cross-platform processing tools to physicians, researchers, and the general public.
Github repository for deep learning medical image registration:
[Keras] VoxelMorph
[Keras] FAIM
[Tensorflow] Weakly-supervised CNN
[Tensorflow] RegNet3D
[Tensorflow] Recursive-Cascaded-Networks
[Pytorch] Probabilistic Dense Displacement Network
[Pytorch] Linear and Deformable Image Registration
[Pytorch] Inverse-Consistent Deep Networks
[Pytorch] Non-parametric image registration
[Pytorch] One Shot Deformable Medical Image Registration
[Pytorch] Image-and-Spatial Transformer Networks
简介：是一款开源、可移植和可扩展的三维几何处理系统。主要用于处理和编辑3D三角网格，它提供了一组用于编辑、清理、修复、检查、渲染、纹理化和转换网格的工具。提供了处理由3D数字化工具/设备生成的原始数据以及3D打印功能，功能全面而且丰富。MeshLab支持多数市面上常见的操作系统，包括Windows、Linux及Mac OS X，支持输入/输出的文件格式有：STL 、OBJ 、 VRML2.0、U3D、X3D、COLLADA MeshLab可用于各种学术和研究环境，如微生物学、文化遗产及表面重建等。
Indoor LiDAR-RGBD Scan Dataset
ETH3D SLAM & Stereo Benchmarks
ViViD : Vision for Visibility Dataset
| Dataset | Number | Modality | Region | Format |
|---|---|---|---|---|
| DIRLAB | 10 | 4D CT | Lung | .img |
| LPBA40 | 40 | 3D MRI | T1 Brain | .img+.hdr .nii |
| IBSR18 | 18 | 3D MRI | T1 Brain | .img+.hdr |
| EMPIRE | 30 | 4D CT | Lung | .mhd+.raw |
| LiTS | 131 | 3D CT | Liver | .nii |
| CT-scans-before-and-after | ||||
| Openi | X-ray | |||
| POPI | 6 | 4D CT | ||
| NLST | CT | Lung | ||
| ADNI | 3D MRI | Brain | ||
| OASIS | 3D MRI | Brain | ||
| ABIDE | 3D MRI | Brain | ||
| ADHD200 | ||||
| CUMC12 | 12 | 3D MRI | Brain | .img+.hdr |
| MGH10 | 10 | 3D MRI | Brain | .img+.hdr |
The Stanford 3D Scanning Repository（斯坦福大学的3d扫描存储库）
http://graphics.stanford.edu/data/3Dscanrep/
这应该是做点云数据最初大家用最多的数据集，其中包含最开始做配准的Bunny、Happy Buddha、Dragon等模型。
Shapenet
ShapeNet是一个丰富标注的大规模点云数据集，其中包含了55中常见的物品类别和513000个三维模型。
The KITTI Vision Benchmark Suite
链接：http://www.cvlibs.net/datasets/kitti/
这个数据集来自德国卡尔斯鲁厄理工学院的一个项目，其中包含了利用KIT的无人车平台采集的大量城市环境的点云数据集（KITTI），这个数据集不仅有雷达、图像、GPS、INS的数据，而且有经过人工标记的分割跟踪结果，可以用来客观的评价大范围三维建模和精细分类的效果和性能。
Robotic 3D Scan Repository
链接：http://kos.informatik.uni-osnabrueck.de/3Dscans/
这个数据集比较适合做SLAM研究，包含了大量的Riegl和Velodyne雷达数据
佐治亚理工大型几何模型数据集
链接：https://www.cc.gatech.edu/projects/large_models/
PASCAL3D+
链接：http://cvgl.stanford.edu/projects/pascal3d.html
包含了12中刚体分类，每一类超过了3000个实例。并且包含了对应的imageNet中每一类的图像。
其他总结
链接：https://github.com/timzhang642/3D-Machine-Learning
CuRIOUS:2019 | Official solution
1 Register pre-operative MRI to iUS before tumor resection
2 Register iUS after tumor resection to iUS before tumor resection
ANHIR:2019 | Official solution
IEEE International Symposium on Biomedical Imaging (ISBI) 2019
High-resolution (up to 40x magnification) whole-slide images of tissues (lesions, lung-lobes, mammary-glands) were acquired - the original size of our images is up to 100k x 200k pixels. The acquired images are organized in sets of consecutive sections where each slice was stained with a different dye and any two images within a set can be meaningfully registered.
Continuous Registration Challenge
Multi-shell Diffusion MRI Harmonisation Challenge 2018 (MUSHAC)
[1] A. Sotiras, et.al., “Deformable medical image registration: A survey,” 2013.
[2] N. J. Tustison, et.al., “Learning image-based spatial transformations via convolutional neural networks : A review,” 2019.
[3] G. Haskins,et.al. “Deep Learning in Medical Image Registration: A Survey,” 2019.
[4] M. Blendowski, et.al., “Combining MRF-based deformable registration and deep binary 3D-CNN descriptors for large lung motion estimation in COPD patients,” 2019.
CVPR: IEEE International Conference on Computer Vision and Pattern Recognition
ICCV: IEEE International Conference on Computer Vision
ECCV: European Conference on Computer Vision
NeurIPS: Conference on Neural Information Processing Systems
AAAI: Association for the Advancement of Artificial Intelligence
ICML: International Conference on Machine Learning
ICPR: International Conference on Pattern Recognition
IJCNN: International Joint Conference on Neural Networks
ICIP: IEEE International Conference on Image Processing
IJCAI: International Joint Conferences on Artificial Intelligence
ICRA: IEEE International Conference on Robotics and Automation
International Conference on 3D Vision
WACV: Winter Conference on Applications of Computer Vision
MICCAI: International Conference on Medical Image Computing and Computer Assisted Intervention
IPMI: Information Processing in Medical Imaging
ISBI: International Symposium on Biomedical Imaging
REMOTE SENSING OF ENVIRONMENT( IF 6.457)，注重全球尺度或长时间尺度的遥感数据处理，极难
ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING( IF5.994)，注重对摄影测量与遥感领域的贡献和算法的创新性，极难
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING( IF4.662)，注重算法的创新和论文结构的严谨性，很难
International Journal of Applied Earth Observation and Geoinformation( IF4.003)，偏向于遥感数据在地学的大范围的应用
IEEE Geoscience and Remote Sensing Letters ( IF2.892)，注重算法的创新，且全文要控制在双栏5页之内，较难
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing( IF2.777 )，偏向于创新方法在地学方面的应用
Remote sensing ( IF3.406)，审稿周期短，发表相对容易，但是有降区的风险
GIScience & Remote Sensing
Photogrammetric engineering and remote sensing
International journal of remote sensing
Remote Sensing Letters
Journal of Applied Remote Sensing
点云配准主要应用于工业制造业的逆向工程、古文物修复、医学三维图像构建等领域。研究内容是属于计算机视觉领域的研究范畴。国际上的会议如计算机视觉三大顶会ICCV、CVPR、ECCV等都会有相关技术，除此之外，还有ACCV、BMVC、SSVM等认可度也比较高。
IEEE Transactions on Pattern Analysis and Machine Intelligence
International Journal of Computer Vision
ISPRS Journal of Photogrammetry and Remote Sensing
TMI: IEEE Transactions on Medical Imaging
MIA: Medical Image Analysis
TIP: IEEE Transactions on Image Processing
TBME: IEEE Transactions on Biomedical Engineering
BOE: Biomedical Optics Express
JHBHI: Journal of Biomedical and Health Informatics
IEEE旗下的TPAMI，TIP等，还有SIAM Journal Image Sciences，Springer那边有IJCV