Martin Danelljan

Martin Danelljan

Researcher

Apple

Biography

I am senior computer vision research engineer at the 3D Vision group at Apple, where I am building a team working on 3D computer vision at a global scale. Previously, I was a research group leader at ETH Zurich and founder of Elevate3D. I received my Ph.D. degree from Linköping University, Sweden in 2018. My thesis “Visual Tracking” was awarded the biennial Best Nordic Thesis Prize.

Hiring: I am looking for very talented researchers and engineers in computer vision and machine learning to join our group at Apple. Contact me if you are interested!

News:

  • Joined Apple in Nov. 2023.
  • Three NeurIPS 2023 papers accepted.
  • Three ICCV 2023 papers accepted.

Selected Publications

Segment Anything in High Quality
NeurIPS 2023
Extending Segment Anything (SAM) to achieve pixel-accurate segmentations of any object.
Lei Ke, Mingqiao Ye, Martin Danelljan, Yifan Liu, Yu-Wing Tai, Chi-Keung Tang
Cite Code arXiv
Segment Anything in High Quality
BiMatting: Efficient Video Matting via Binarization
NeurIPS 2023
A binary network for highly efficient video matting.
Haotong Qin, Lei Ke, Xudong Ma, Martin Danelljan, Yu-Wing Tai, Chi-Keung Tang, Xianglong Liu
Cite Code
BiMatting: Efficient Video Matting via Binarization
QuantSR: Accurate Low-bit Quantization for Efficient Image Super-Resolution
NeurIPS 2023
Quantization method and architecture for super-resolution.
Haotong Qin, Yulun Zhang, Yifu Ding, Yifan Liu, Xianglong Liu, Martin Danelljan
Cite Code
QuantSR: Accurate Low-bit Quantization for Efficient Image Super-Resolution
Cascade-DETR: Delving into High-Quality Universal Object Detection
ICCV 2023
Cascade attention and IoU-based scoring for object detection.
Mingqiao Ye, Lei Ke, Siyuan Li, Yu-Wing Tai, Chi-Keung Tang, Martin Danelljan
Cite Code arXiv
Cascade-DETR: Delving into High-Quality Universal Object Detection
R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras
ICCV 2023
A method for 3d reconstruction of dynamic scenes with multi-camera setups.
Aron Schmied, Tobias Fischer, Martin Danelljan, Marc Pollefeys
Cite Code arXiv
R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras
MolGrapher: Graph-based Visual Recognition of Chemical Structures
ICCV 2023
A graph-based method for chemical structure recognition in documents.
Lucas Morin, Martin Danelljan, Maria Isabel Agea, Ahmed Nassar, Valery Weber, Ingmar Meijer, Peter Staar
Cite Code arXiv
MolGrapher: Graph-based Visual Recognition of Chemical Structures
Mask-Free Video Instance Segmentation
CVPR 2023
State-of-the-art Video Instance Segmentation without any ground-truth masks for training.
Lei Ke, Martin Danelljan, Henghui Ding, Yu-Wing Tai, Chi-Keung Tang
Cite Code Project arXiv
Mask-Free Video Instance Segmentation
OVTrack: Open-Vocabulary Multiple Object Tracking
CVPR 2023
First method and benchmark for open vocabulary tracking.
Siyuan Li, Tobias Fischer, Lei Ke, Henghui Ding, Martin Danelljan
Cite Project
OVTrack: Open-Vocabulary Multiple Object Tracking
Continuous Pseudo-Label Rectified Domain Adaptive Semantic Segmentation with Implicit Neural Representations
CVPR 2023
Leveraging implicit models to estimate the output confidence for unsupervised domain adaptation.
Rui Gong, Qin Wang, Martin Danelljan, Dengxin Dai, Luc Van Gool
Cite Code Project
Continuous Pseudo-Label Rectified Domain Adaptive Semantic Segmentation with Implicit Neural Representations
PDC-Net+: Enhanced Probabilistic Dense Correspondence Network
TPAMI 2023
A method that gives you accurate dense optical flow and correspondences with robust uncertainty.
Prune Truong, Martin Danelljan, Luc Van Gool, Radu Timofte
Cite Code Project arXiv
PDC-Net+: Enhanced Probabilistic Dense Correspondence Network
Visual Object Tracking with Discriminative Filters and Siamese Networks: A Survey and Outlook
TPAMI 2023
Comprehensive survey over Visual Object Tracking.
Sajid Javed, Martin Danelljan, Fahad Shahbaz Khan, Muhammad Haris Khan, Michael Felsberg, Jiri Matas
Cite arXiv
Visual Object Tracking with Discriminative Filters and Siamese Networks: A Survey and Outlook
Dense Gaussian Processes for Few-Shot Segmentation
ECCV 2022
A few-shot learner based on Gaussian Processes for few-shot semantic segmentation.
Joakim Johnander, Johan Edstedt, Michael Felsberg, Fahad Shahbaz Khan, Martin Danelljan
Cite Code arXiv
Dense Gaussian Processes for Few-Shot Segmentation
Robust Visual Tracking by Segmentation
ECCV 2022
Bringing the robustness in tracking to video segmentation.
Matthieu Paul, Martin Danelljan, Christoph Mayer, Luc Van Gool
Cite Code arXiv
Robust Visual Tracking by Segmentation
Transform your Smartphone into a DSLR Camera: Learning the ISP in the Wild
ECCV 2022
A method and dataset for learning the camera ISP in the wild.
Ardhendu Tripathi, Martin Danelljan, Samarth Shukla, Radu Timofte, Luc Van Gool
Cite arXiv
Transform your Smartphone into a DSLR Camera: Learning the ISP in the Wild
Tracking Every Thing in the Wild
ECCV 2022
A new method and metric for large-scale and open world multi-object tracking.
Siyuan Li, Martin Danelljan, Henghui Ding, Thomas Huang
Cite Code Project arXiv
Tracking Every Thing in the Wild
Video Mask Transfiner for High-Quality Video Instance Segmentation
ECCV 2022
An efficient transformer-based method for highly accurate video instance segmentation.
Lei Ke, Henghui Ding, Martin Danelljan, Yu-Wing Tai, Chi-Keung Tang
Cite Code Dataset Project arXiv
Video Mask Transfiner for High-Quality Video Instance Segmentation
TACS: Taxonomy Adaptive Cross-Domain Semantic Segmentation
ECCV 2022
Bringing the robustness in tracking to video segmentation.
Rui Gong, Martin Danelljan, Dengxin Dai, Danda Pani Paudel, Ajad Chhatkuli, Luc Van Gool
Cite Code arXiv
TACS: Taxonomy Adaptive Cross-Domain Semantic Segmentation
RePaint: Inpainting using Denoising Diffusion Probabilistic Models
CVPR 2022
A Probabilistic Denoising Diffusion Model for image inpainting.
Andreas Lugmayr, Martin Danelljan, Andres Romero, Radu Timofte, Luc Van Gool
Cite Code arXiv
RePaint: Inpainting using Denoising Diffusion Probabilistic Models
Mask Transfiner for High-Quality Instance Segmentation
CVPR 2022
An efficient transformer-based method for highly accurate instance segmentation.
Lei Ke, Martin Danelljan, Xia Li, Yu-Wing Tai, Chi-Keung Tang
Cite Code Project arXiv
Mask Transfiner for High-Quality Instance Segmentation
Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondences
CVPR 2022
A weakly-supervised method for learning dense semantic correspondences.
Prune Truong, Martin Danelljan, Luc Van Gool
Cite Code Project arXiv
Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondences
Adiabatic Quantum Computing for Multi Object Tracking
CVPR 2022
A Multi-Object Tracking algorithm that can be solved with Adiabatic Quantum Computing
Jan-Nico Zäch, Alexander Liniger, Martin Danelljan, Dengxin Dai, Luc Van Gool
Cite arXiv
Adiabatic Quantum Computing for Multi Object Tracking
Transforming Model Prediction for Tracking
CVPR 2022
A transformer-based tracker inspired by discriminative correlation filters.
Christoph Mayer, Martin Danelljan, Goutam Bhat, Matthieu Paul, Danda Pani Paudel, Luc Van Gool
Cite Code arXiv
Transforming Model Prediction for Tracking
Arbitrary-Scale Image Synthesis
CVPR 2022
A GAN that generates consistent images at arbitrary scales and resolutions.
Evangelos Ntavelis, Mohamad Shahbazi, Iason Kastanis, Radu Timofte, Martin Danelljan, Luc Van Gool
Cite
Arbitrary-Scale Image Synthesis
Collapse by Conditioning: Training Class-conditional GANs with Limited Data
ICLR 2022
Analyzing and addressing mode collapse in conditional GANs caused by the conditioning itself.
Mohamad Shahbazi, Martin Danelljan, Danda Pani Paudel, Luc Van Gool
Cite Code arXiv
Collapse by Conditioning: Training Class-conditional GANs with Limited Data
Learnable Online Graph Representations for 3D Multi-Object Tracking
ICRA 2022
An online 3D Multi-Object Tracking method based on graph neural networks.
Jan-Nico Zäch, Dengxin Dai, Alexander Liniger, Martin Danelljan, Luc Van Gool
Cite arXiv
Learnable Online Graph Representations for 3D Multi-Object Tracking
Prototypical Cross-Attention Networks for Multiple Object Tracking and Segmentation
NeurIPS 2021 Spotlight
Efficient cross-attention for video instance segmentation.
Lei Ke, Xia Li, Martin Danelljan, Yu-Wing Tai, Chi-Keung Tang
Cite Video arXiv
Prototypical Cross-Attention Networks for Multiple Object Tracking and Segmentation
Warp Consistency for Unsupervised Learning of Dense Correspondences
ICCV 2021 Oral
Unsupervised method for learning dense correspondences and optical flow on real image pairs.
Prune Truong, Martin Danelljan, Luc Van Gool
Cite Code Project arXiv
Warp Consistency for Unsupervised Learning of Dense Correspondences
Deep Reparametrization of Multi-Frame Super-Resolution and Denoising
ICCV 2021 Oral
Deep optimization-based formulation for multi-frame super-resolution and denoising.
Goutam Bhat, Martin Danelljan, Luc Van Gool, Radu Timofte
Cite arXiv
Deep Reparametrization of Multi-Frame Super-Resolution and Denoising
Generating Masks from Boxes by Mining Spatio-Temporal Consistencies in Videos
ICCV 2021
An optimization-based architecture for converting video bounding box annotations to segmentation masks.
Bin Zhao, Goutam Bhat, Martin Danelljan, Luc Van Gool, Radu Timofte
Cite Code arXiv
Generating Masks from Boxes by Mining Spatio-Temporal Consistencies in Videos
Learning Target Candidate Association to Keep Track of What Not to Track
ICCV 2021
Associating the target and distractor objects for robust visual tracking.
Christoph Mayer, Martin Danelljan, Danda Pani Paudel, Luc Van Gool
Cite Code arXiv
Learning Target Candidate Association to Keep Track of What Not to Track
Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling
ICCV 2021
A unified hierarchical normalizing flow architecture for super-resolution and image rescaling.
Jingyun Liang, Andreas Lugmayr, Kai Zhang, Martin Danelljan, Luc Van Gool, Radu Timofte
Cite arXiv
Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling
Scaling Semantic Segmentation Beyond 1K Classes on a Single GPU
ICCV 2021
Reducing memory complexity for training semantic segmentation models with large number of classes.
Shipra Jain, Danda Pani Paudel, Martin Danelljan, Luc Van Gool
Cite Code arXiv
Scaling Semantic Segmentation Beyond 1K Classes on a Single GPU
Local Memory Attention for Fast Video Semantic Segmentation
IROS 2021
A local memory cross-attention module for fast video semantic segmentation.
Matthieu Paul, Martin Danelljan, Luc Van Gool, Radu Timofte
Cite Code arXiv
Local Memory Attention for Fast Video Semantic Segmentation
Learning Accurate Dense Correspondences and When to Trust Them
CVPR 2021 Oral
A method that gives you accurate dense optical flow and correspondences with robust uncertainty.
Prune Truong, Martin Danelljan, Luc Van Gool, Radu Timofte
Cite Code Project Video arXiv
Learning Accurate Dense Correspondences and When to Trust Them
DeFlow: Learning Complex Image Degradations from Unpaired Data with Conditional Flows
CVPR 2021 Oral
A novel unpaired learning formulation for conditional normalizing flows with applications to learning image degradations.
Valentin Wolf, Andreas Lugmayr, Martin Danelljan, Luc Van Gool, Radu Timofte
Cite Code Video arXiv
DeFlow: Learning Complex Image Degradations from Unpaired Data with Conditional Flows
Deep Burst Super-Resolution
CVPR 2021
An attention based architecture and real-world dataset for burst super-resolution.
Goutam Bhat, Martin Danelljan, Luc Van Gool, Radu Timofte
Cite Video arXiv
Deep Burst Super-Resolution
The Heterogeneity Hypothesis: Finding Layer-Wise Dissimilated Network Architecture
CVPR 2021
We tackle the problem of convolutional neural network design by adjusting the channel configurations of predefined networks.
Yawei Li, Wen Li, Martin Danelljan, Kai Zhang, Shuhang Gu, Luc Van Gool, Radu Timofte
Cite arXiv
The Heterogeneity Hypothesis: Finding Layer-Wise Dissimilated Network Architecture
Few-Shot Classification By Few-Iteration Meta-Learning
ICRA 2021
An optimization-based meta-learning approach for few-shot classification.
Ardhendu Tripathi, Martin Danelljan, Luc Van Gool, Radu Timofte
Cite Code Video arXiv
Few-Shot Classification By Few-Iteration Meta-Learning
DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation
NeurIPS 2020
Dataset and method for generating vector graphics.
Alexandre Carlier, Martin Danelljan, Alexandre Alahi, Radu Timofte
Cite Code Project Video arXiv
DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation
GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network
NeurIPS 2020
A fully differentiable dense matching module for your correspondence or optical flow network.
Prune Truong, Martin Danelljan, Luc Van Gool, Radu Timofte
Cite Code Project Video arXiv
GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network
How to Train Your Energy-Based Model for Regression
BMVC 2020
Investigating how to train a deep energy-based model for accurate regression.
Fredrik Gustafsson, Martin Danelljan, Radu Timofte, Thomas Schön
Cite Code Video arXiv Code Tracking Slides
How to Train Your Energy-Based Model for Regression
Learning What to Learn for Video Object Segmentation
ECCV 2020 Oral
An optimization-based few-shot learner for VOS.
Goutam Bhat, Felix Järemo Lawin, Martin Danelljan, Andreas Robinson, Michael Felsberg, Luc Van Gool, Radu Timofte
Cite Code arXiv Video (10 min) Video (2 min)
Learning What to Learn for Video Object Segmentation
SRFlow: Learning the Super-Resolution Space with Normalizing Flow
ECCV 2020 Spotlight
Normalizing flow based super-resolution method capable of learning the conditional distribution of the output given the low-resolution input.
Andreas Lugmayr, Martin Danelljan, Luc Van Gool, Radu Timofte
Cite Code arXiv Blog Post
SRFlow: Learning the Super-Resolution Space with Normalizing Flow
Energy-Based Models for Deep Probabilistic Regression
ECCV 2020
A general method for accurate regression by learning the conditional target probability distribution as a deep energy-based model.
Fredrik Gustafsson, Martin Danelljan, Goutam Bhat, Thomas Schön
Cite Code arXiv Video (1 min) Video (10 min) Code Tracking Slides
Energy-Based Models for Deep Probabilistic Regression
Video Object Segmentation with Episodic Graph Memory Networks
ECCV 2020 Spotlight
A graph-based memory module for Video Object Segmentation.
Xinkai Lu, Wenguan Wang, Martin Danelljan, Tianfei Zhou, Jianbing Shen, Luc Van Gool
Cite Code arXiv
Video Object Segmentation with Episodic Graph Memory Networks
Know Your Surroundings: Exploiting Scene Information for Object Tracking
ECCV 2020
A tracking architecture that exploits the knowledge about the presence of other objects in the surrounding scene to prevent failure.
Goutam Bhat, Martin Danelljan, Luc Van Gool, Radu Timofte
Cite Code arXiv Video (10 min) Video (1 min)
Know Your Surroundings: Exploiting Scene Information for Object Tracking
GLU-Net: Global-Local Universal Network for Dense Flow and Correspondences
CVPR 2020 Oral
A unified network architecture for dense correspondences applicable to geometric matching, optical flow and semantic matching.
Prune Truong, Martin Danelljan, Radu Timofte
Cite Code Project Video DOI arXiv
GLU-Net: Global-Local Universal Network for Dense Flow and Correspondences
Learning Fast and Robust Target Models for Video Object Segmentation
CVPR 2020 Oral
A light-weight optimization-based target model for fast VOS.
Andreas Robinson, Felix Järemo Lawin, Martin Danelljan, Fahad Shahbaz Khan, Michael Felsberg
Cite Code Video DOI arXiv
Learning Fast and Robust Target Models for Video Object Segmentation
Probabilistic Regression for Visual Tracking
CVPR 2020
Proposes a general formulation for probabilistic regression, which is then applied to visual tracking.
Martin Danelljan, Luc Van Gool, Radu Timofte
Cite Code Video DOI arXiv
Probabilistic Regression for Visual Tracking
Learning Human-Object Interaction Detection using Interaction Points
CVPR 2020
A fully-convolutional approach that directly detects the interactions between human-object pairs.
Tiancai Wang, Tong Yang, Martin Danelljan, Fahad Shahbaz Khan, Xiangyu Zhang, Jian Sun
Cite Code Video DOI arXiv
Learning Human-Object Interaction Detection using Interaction Points
Learning Discriminative Model Prediction for Tracking
ICCV 2019 Oral
An end-to-end tracking architecture, capable of fully exploiting both target and background appearance information for target model prediction.
Goutam Bhat, Martin Danelljan, Luc Van Gool, Radu Timofte
Cite Code Video DOI arXiv
Learning Discriminative Model Prediction for Tracking
Learning the Model Update for Siamese Trackers
ICCV 2019
Replacing the handcrafted update function in Siamese trackers with a learnable update mechanism.
Lichao Zhang, Abel Gonzalez-Garcia, Joost van de Weijer, Martin Danelljan, Fahad Shahbaz Khan
Cite Code DOI arXiv
Learning the Model Update for Siamese Trackers
Tracking the Known and the Unknown by Leveraging Semantic Information
BMVC 2019
“e propose a tracking framework that can exploit semantic information, without sacrificing the generic nature of the tracker.
Ardhendu Tripathi, Martin Danelljan, Luc Van Gool, Radu Timofte
PDF Cite
Tracking the Known and the Unknown by Leveraging Semantic Information
ATOM: Accurate Tracking by Overlap Maximization
CVPR 2019 Oral
Performing accurate bounding box estimation for generic visual tracking.
Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, Michael Felsberg
Cite Code Video DOI arXiv
ATOM: Accurate Tracking by Overlap Maximization
A Generative Appearance Model for End-to-end Video Object Segmentation
CVPR 2019 Oral
A generative appearance module for end-to-end VOS.
Joakim Johnander, Martin Danelljan, Emil Brissman, Fahad Shahbaz Khan, Michael Felsberg
Cite Code Video DOI arXiv
A Generative Appearance Model for End-to-end Video Object Segmentation
Unveiling the Power of Deep Tracking
ECCV 2019
How to better utilize deep features for correlation-based tracking.
Goutam Bhat, Joakim Johnander, Martin Danelljan, Fahad Shahbaz Khan, Michael Felsberg
Cite Code DOI arXiv Code (Unofficial Python)
Unveiling the Power of Deep Tracking
Density Adaptive Point Set Registration
CVPR 2018 Oral
Revisiting the foundations of probabilistic point cloud registration in order to tackle the key issue of sampling density variations.
Felix Järemo Lawin, Martin Danelljan, Fahad Shahbaz Khan, Per-Erik Forssén, Michael Felsberg
Cite Code Video DOI arXiv
Density Adaptive Point Set Registration
ECO: Efficient Convolution Operators for Tracking
CVPR 2017
Tackling the key causes behind the problems of computational complexity and over-fitting in correlation trackers.
Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, Michael Felsberg
Cite Code Poster DOI arXiv Code (Unofficial Python)
ECO: Efficient Convolution Operators for Tracking
Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
ECCV 2016 Oral
A theoretical framework for discriminatively learning a convolution operator in the continuous spatial domain.
Martin Danelljan, Andreas Robinson, Fahad Shahbaz Khan, Michael Felsberg
Cite Code Poster Slides Video DOI arXiv Supplementary Code (Unofficial Python)
Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
A Probabilistic Framework for Color-Based Point Set Registration
CVPR 2016
A probabilistic point set registration framework that exploits available color information associated with the points.
Martin Danelljan, Giulia Meneghetti, Fahad Shahbaz Khan, Michael Felsberg
PDF Cite DOI Supplementary Code (unofficial)
A Probabilistic Framework for Color-Based Point Set Registration
Adaptive Decontamination of the Training Set: A Unified Formulation for Discriminative Visual Tracking
CVPR 2016
A unified formulation for alleviating the problem of corrupted training samples in tracking-by-detection methods.
Martin Danelljan, Gustav Häger, Fahad Shahbaz Khan, Michael Felsberg
Cite Code DOI arXiv Supplementary Raw Results
Adaptive Decontamination of the Training Set: A Unified Formulation for Discriminative Visual Tracking
Learning Spatially Regularized Correlation Filters for Visual Tracking
ICCV 2015
Mitigating the unwanted boundary effects, which limits the performance of correlation based trackers.
Martin Danelljan, Gustav Häger, Fahad Shahbaz Khan, Michael Felsberg
Cite Code Project Poster DOI arXiv
Learning Spatially Regularized Correlation Filters for Visual Tracking
Discriminative Scale Space Tracking
TPAMI & BMVC 2014
Accurate and fast scale estimation for visual tracking.
Martin Danelljan, Gustav Häger, Fahad Shahbaz Khan, Michael Felsberg
Cite Code Project DOI arXiv (journal) PDF (conf.)
Discriminative Scale Space Tracking
Adaptive Color Attributes for Real-Time Visual Tracking
CVPR 2014 Oral
How to incorporate color information into visual tracking.
Martin Danelljan, Fahad Shahbaz Khan, Michael Felsberg, Joost van de Weijer
PDF Cite Code Project Poster DOI Supplementary
Adaptive Color Attributes for Real-Time Visual Tracking

Co-supervised PhD Students

Avatar

Siyuan Li

PhD Student, ETH Zurich

Avatar

Prune Truong

PhD Student, ETH Zurich

Avatar

Mohamad Shahbazi

PhD Student, ETH Zurich

Graduated PhD Students

Avatar

Lei Ke

PhD Student, ETH Zurich & HKUST

Avatar

Christoph Mayer

PhD Student, ETH Zurich

Avatar

Joakim Johnander

PhD Student, Linköping Univ.

Avatar

Matthieu Paul

PhD Student, ETH Zurich

Avatar

Ardhendu Tripathi

PhD Student, ETH Zurich

Avatar

Andreas Lugmayr

PhD Student, ETH Zurich

Avatar

Jan-Nico Zäch

PhD Student, ETH Zurich

Avatar

Goutam Bhat

PhD Student, ETH Zurich

Avatar

Evangelos Ntavelis

PhD Student, ETH Zurich

Avatar

Fredrik Gustafsson

PhD Student, Uppsala Univ.

Contact