Jan Hendrik Metzen's website

Senior AI Researcher at IPAI Aleph Alpha Research.

prof_pic.png

I am Senior AI Researcher at IPAI Aleph Alpha Research. I was Senior Expert at Bosch Center for Artificial Intelligence (BCAI) until 08/2024.

My primary research focuses on making AI (specifically computer-vision based perception) robust, reliable, and safe. For this, we levarage strong generative models for finding systematic errors of image classifiers on rare subgroups and systematic errors of object detectors. We also identified vulnerabilities of Transformer-based neural network against adversarial patch/token attacks. To counteract such vulnerabilities, we developed architectures that are certifiably robust against patch attacks for image classifiers as well as for semantic segmentation. Furthermore, we proposed methods for adversarially training neural networks to become robust against universal perturbations and universal adversarial patches. In addition, we provide methods for test-time adaptation of neural networks to improve robustness to domain shifts and study the role of shape-biased representations on robustness to common image corruptions.

A different strand of my research is automating machine learning (AutoML), specifically Neural Architecture Search. The latter research field is motivated by the vast design space of neural networks and the diversity of inference hardware. Manually tailoring a neural architecture for every type of hardware is cumbersome and not scalable - hardware-aware neural architecture search can vastly improve design efficiency and thus reduce cost of AI development. See our survey on Neural Architecture Search and a more recent survey on Neural Architecture Search for Dense Prediction Tasks in Computer Vision. Recently, we developed AutoCLIP, a method auto-tuning zero-shot classifiers for vision-language models that improves zero-shot performance across a broad range of domains.

I also love contributing to machine learning libraries, both open source and proprietary. I am a core contributor of scikit-learn, where I contributed tools for probability calibration of classifiers and for kernel ridge regression. Moreover, I have written a complete redesign of the Gaussian process module for scikit-learn. At BCAI, I am/was involved as core developer for frameworks for deep learning training pipelines, neural architecture search, and robustness evaluation.

I am a member of ELLIS and regularly review for scientific conferences and journals such as ICLR, ICML, NeurIPS, and TMLR. I was senior area chair of the AutoML 2022 conference and co-organizer of the workshops NAS@ICLR 2020 and NAS@ICLR 2021. I have been recognized by ICLR as Highlighted Reviewer in 2022 and Outstanding Reviewer in 2021.

See also my curriculum vitae for more information. Feel free to contact me via janmetzen@mailbox.org.

news

Aug 12, 2024 I have built SVGStud.io, an AI-based tool for searching and generating Scalable Vector Graphics (SVG) files. SVGStud.io offers the following core functionalities:
  • Semantic SVG Search Find SVG files that match a search term or a sample image as closely as possible, from a library of more than 10,000 SVGs.
  • AI-based SVG Generator Generate novel SVGs based on textual descriptions and (optionally) example images.
  • SVG Gallery Explore a gallery of all SVGs in our library. Perfect for serendipity!
  • SVG Bundles Browse a large variety of free pre-generated SVG bundles.
All SVGs in SVGStud.io are licensed under CC-BY-SA 4.0 license and can be downloaded at any time.

latest posts

selected publications

2024

  1. AutoCLIP: Auto-tuning Zero-Shot Classifiers for Vision-Language Models
    Jan Hendrik Metzen, Piyapat Saranrittichai, and Chaithanya Kumar Mummadi
    Transactions on Machine Learning Research, 2024
  2. Label-free Neural Semantic Image Synthesis
    Jiayi Wang, Kevin Alexander Laube, Yumeng Li, Jan Hendrik Metzen, Shin-I Cheng, Julio Borges, and Anna Khoreva
    In European Conference on Computer Vision (ECCV), 2024

2023

  1. Identification of Systematic Errors of Image Classifiers on Rare Subgroups
    Jan Hendrik Metzen, Robin Hutmacher, N Grace Hua, Valentyn Boreiko, and Dan Zhang
    In International Conference on Computer Vision, 2023
  2. Identifying Systematic Errors in Object Detectors with the SCROD Pipeline
    Valentyn Boreiko, Matthias Hein, and Jan Hendrik Metzen
    In BRAVO Workshop, ICCV, 2023

2022

  1. Give Me Your Attention: Dot-Product Attention Considered Harmful for Adversarial Patch Robustness
    Giulio Lovisotto, Nicole Finnie, Mauricio Munoz, Chaithanya Kumar Mummadi, and Jan Hendrik Metzen
    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022