publications | Jan Hendrik Metzen's website

2024

AutoCLIP: Auto-tuning Zero-Shot Classifiers for Vision-Language Models

Jan Hendrik Metzen, Piyapat Saranrittichai, and Chaithanya Kumar Mummadi

Transactions on Machine Learning Research, 2024

Abs arXiv PDF

Classifiers built upon vision-language models such as CLIP have shown remarkable zero-shot performance across a broad range of image classification tasks. Prior work has studied different ways of automatically creating descriptor sets for every class based on prompt templates, ranging from manually engineered templates over templates obtained from a large language model to templates built from random words and characters. Up until now, deriving zero-shot classifiers from the respective encoded class descriptors has remained nearly unchanged, i.e., classify to the class that maximizes cosine similarity between its averaged encoded class descriptors and the image encoding. However, weighing all class descriptors equally can be suboptimal when certain descriptors match visual clues on a given image better than others. In this work, we propose AutoCLIP, a method for auto-tuning zero-shot classifiers. AutoCLIP tunes per-image weights to each prompt template at inference time, based on statistics of class descriptor-image similarities. AutoCLIP is fully unsupervised, has very low computational overhead, and can be easily implemented in few lines of code. We show that AutoCLIP outperforms baselines across a broad range of vision-language models, datasets, and prompt templates consistently and by up to 3 percent point accuracy.
SciOL and MuLMS-Img: Introducing a Large-Scale Multimodal Scientific Dataset and Models for Image-Text Tasks in the Scientific Domain

Tim Tarsi, Heike Adel, Jan Hendrik Metzen, Dan Zhang, Matteo Finco, and Annemarie Friedrich

In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024

Abs PDF

In scientific publications, a substantial part of the information is expressed via figures containing images and diagrams. Hence, the retrieval of relevant figures given a natural language query is an important real-world task. However, due to the lack of training and evaluation data, most existing approaches are either limited to one modality or focus on non-scientific domains, making their application to scientific publications challenging. In this paper, we address this gap by introducing two novel datasets: (1) SciOL, the largest openly-licensed pre-training corpus for multimodal models in the scientific domain, covering multiple sciences including materials science, physics, and computer science, and (2) MuLMS-Img, a high-quality dataset in the materials science domain, manually annotated for various image-text tasks. Our experiments show that pre-training large-scale vision-language models on SciOL increases performance considerably across a broad variety of image-text tasks including figure type classification, optical character recognition, captioning, and figure retrieval. Using MuLMS-Img, we show that integrating text-based features extracted via a fine-tuned model for a specific domain can boost cross-modal scientific figure retrieval performance by up to 50%.
Identification of Fine-grained Systematic Errors via Controlled Scene Generation

Valentyn Boreiko, Matthias Hein, and Jan Hendrik Metzen

In , 2024

Abs arXiv PDF

Many safety-critical applications, especially in autonomous driving, require reliable object detectors. They can be very effectively assisted by a method to search for and identify potential failures and systematic errors before these detectors are deployed. Systematic errors are characterized by combinations of attributes such as object location, scale, orientation, and color, as well as the composition of their respective backgrounds. To identify them, one must rely on something other than real images from a test set because they do not account for very rare but possible combinations of attributes. To overcome this limitation, we propose a pipeline for generating realistic synthetic scenes with fine-grained control, allowing the creation of complex scenes with multiple objects. Our approach, BEV2EGO, allows for a realistic generation of the complete scene with road-contingent control that maps 2D bird’s-eye view (BEV) scene configurations to a first-person view (EGO). In addition, we propose a benchmark for controlled scene generation to select the most appropriate generative outpainting model for BEV2EGO. We further use it to perform a systematic analysis of multiple state-of-the-art object detection models and discover differences between them.
Zero-Shot Distillation for Image Encoders: How to Make Effective Use of Synthetic Data

Niclas Popp, Jan Hendrik Metzen, and Matthias Hein

In , 2024

arXiv PDF
Retinex-Diffusion: On Controlling Illumination Conditions in Diffusion Models via Retinex Theory

Xiaoyan Xing, Vincent Tao Hu, Jan Hendrik Metzen, Konrad Groh, Sezer Karaoglu, and Theo Gevers

In , 2024

arXiv PDF
Label-free Neural Semantic Image Synthesis

Jiayi Wang, Kevin Alexander Laube, Yumeng Li, Jan Hendrik Metzen, Shin-I Cheng, Julio Borges, and Anna Khoreva

In European Conference on Computer Vision (ECCV), 2024

Abs arXiv PDF

Recent work has shown great progress in integrating spatial conditioning to control large, pre-trained text-to-image diffusion models. Despite these advances, existing methods describe the spatial image content using hand-crafted conditioning inputs, which are either semantically ambiguous (e.g., edges) or require expensive manual annotations (e.g., semantic segmentation). To address these limitations, we propose a new label-free way of conditioning diffusion models to enable fine-grained spatial control. We introduce the concept of \emphneural semantic image synthesis, which uses neural layouts extracted from pre-trained foundation models as conditioning. Neural layouts are advantageous as they provide rich descriptions of the desired image, containing both semantics and detailed geometry of the scene. We experimentally show that images synthesized via neural semantic image synthesis achieve similar or superior pixel-level alignment of semantic classes compared to those created using expensive semantic label maps. At the same time, they capture better semantics, instance separation, and object orientation than other label-free conditioning options, such as edges or depth. Moreover, we show that images generated by neural layout conditioning can effectively augment real data for training various perception tasks.

2023

Certified Defences Against Adversarial Patch Attacks on Semantic Segmentation

Maksym Yatsura, Kaspar Sakmann, N Grace Hua, Matthias Hein, and Jan Hendrik Metzen

In International Conference on Learning Representations, 2023

Abs arXiv PDF

Adversarial patch attacks are an emerging security threat for real world deep learning applications. We present Demasked Smoothing, the first approach (up to our knowledge) to certify the robustness of semantic segmentation models against this threat model. Previous work on certifiably defending against patch attacks has mostly focused on image classification task and often required changes in the model architecture and additional training which is undesirable and computationally expensive. In Demasked Smoothing, any segmentation model can be applied without particular training, fine-tuning, or restriction of the architecture. Using different masking strategies, Demasked Smoothing can be applied both for certified detection and certified recovery. In extensive experiments we show that Demasked Smoothing can on average certify 64% of the pixel predictions for a 1% patch in the detection task and 48% against a 0.5% patch for the recovery task on the ADE20K dataset.
Identification of Systematic Errors of Image Classifiers on Rare Subgroups

Jan Hendrik Metzen, Robin Hutmacher, N Grace Hua, Valentyn Boreiko, and Dan Zhang

In International Conference on Computer Vision, 2023

Abs arXiv PDF

Despite excellent average-case performance of many image classifiers, their performance can substantially deteriorate on semantically coherent subgroups of the data that were under-represented in the training data. These systematic errors can impact both fairness for demographic minority groups as well as robustness and safety under domain shift. A major challenge is to identify such subgroups with subpar performance when the subgroups are not annotated and their occurrence is very rare. We leverage recent advances in text-to-image models and search in the space of textual descriptions of subgroups ("prompts") for subgroups where the target model has low performance on the prompt-conditioned synthesized data. To tackle the exponentially growing number of subgroups, we employ combinatorial testing. We denote this procedure as PromptAttack as it can be interpreted as an adversarial attack in a prompt space. We study subgroup coverage and identifiability with PromptAttack in a controlled setting and find that it identifies systematic errors with high accuracy. Thereupon, we apply PromptAttack to ImageNet classifiers and identify novel systematic errors on rare subgroups.
Neural Architecture Search for Dense Prediction Tasks in Computer Vision

Rohit Mohan, Thomas Elsken, Arber Zela, Jan Hendrik Metzen, Benedikt Staffler, Thomas Brox, Abhinav Valada, and Frank Hutter

International Journal of Computer Vision, 2023

Abs arXiv

The success of deep learning in recent years has lead to a rising demand for neural network architecture engineering. As a consequence, neural architecture search (NAS), which aims at automatically designing neural network architectures in a data-driven manner rather than manually, has evolved as a popular field of research. With the advent of weight sharing strategies across architectures, NAS has become applicable to a much wider range of problems. In particular, there are now many publications for dense prediction tasks in computer vision that require pixel-level predictions, such as semantic segmentation or object detection. These tasks come with novel challenges, such as higher memory footprints due to high-resolution data, learning multi-scale representations, longer training times, and more complex and larger neural architectures. In this manuscript, we provide an overview of NAS for dense prediction tasks by elaborating on these novel challenges and surveying ways to address them to ease future research and application of existing methods to novel problems.
Identifying Systematic Errors in Object Detectors with the SCROD Pipeline

Valentyn Boreiko, Matthias Hein, and Jan Hendrik Metzen

In BRAVO Workshop, ICCV, 2023

Abs arXiv PDF

The identification and removal of systematic errors in object detectors can be a prerequisite for their deployment in safety-critical applications like automated driving and robotics. Such systematic errors can for instance occur under very specific object poses (location, scale, orientation), object colors/textures, and backgrounds. Real images alone are unlikely to cover all relevant combinations. We overcome this limitation by generating synthetic images with fine-granular control. While generating synthetic images with physical simulators and hand-designed 3D assets allows fine-grained control over generated images, this approach is resource-intensive and has limited scalability. In contrast, using generative models is more scalable but less reliable in terms of fine-grained control. In this paper, we propose a novel framework that combines the strengths of both approaches. Our meticulously designed pipeline along with custom models enables us to generate street scenes with fine-grained control in a fully automated and scalable manner. Moreover, our framework introduces an evaluation setting that can serve as a benchmark for similar pipelines. This evaluation setting will contribute to advancing the field and promoting standardized testing procedures.

2022

Give Me Your Attention: Dot-Product Attention Considered Harmful for Adversarial Patch Robustness

Giulio Lovisotto, Nicole Finnie, Mauricio Munoz, Chaithanya Kumar Mummadi, and Jan Hendrik Metzen

In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

Abs arXiv PDF

Neural architectures based on attention such as vision transformers are revolutionizing image recognition. Their main benefit is that attention allows reasoning about all parts of a scene jointly. In this paper, we show how the global reasoning of (scaled) dot-product attention can be the source of a major vulnerability when confronted with adversarial patch attacks. We provide a theoretical understanding of this vulnerability and relate it to an adversary’s ability to misdirect the attention of all queries to a single key token under the control of the adversarial patch. We propose novel adversarial objectives for crafting adversarial patches which target this vulnerability explicitly. We show the effectiveness of the proposed patch attacks on popular image classification (ViTs and DeiTs) and object detection models (DETR). We find that adversarial patches occupying 0.5% of the input can lead to robust accuracies as low as 0% for ViT on ImageNet, and reduce the mAP of DETR on MS COCO to less than 3%.
Denoised Smoothing with Sample Rejection for Robustifying Pretrained Classifiers

Fatemeh Sheikholeslami, Wan-Yi Lin, Jan Hendrik Metzen, Huan Zhang, and J Zico Kolter

In Workshop on Trustworthy and Socially Responsible Machine Learning, NeurIPS 2022, 2022

PDF

2021

Does enhanced shape bias improve neural network robustness to common corruptions?

Chaithanya Kumar Mummadi, Ranjitha Subramaniam, Robin Hutmacher, Julien Vitay, Volker Fischer, and Jan Hendrik Metzen

In International Conference on Learning Representations, 2021

arXiv PDF
Efficient Certified Defenses Against Patch Attacks on Image Classifiers

Jan Hendrik Metzen, and Maksym Yatsura

In International Conference on Learning Representations, 2021

arXiv PDF
Meta Adversarial Training against Universal Patches

Jan Hendrik Metzen, Nicole Finnie, and Robin Hutmacher

In ICML 2021 Workshop on Adversarial Machine Learning, 2021

arXiv PDF
Test-Time Adaptation to Distribution Shift by Confidence Maximization and Input Transformation

Chaithanya Kumar Mummadi, Robin Hutmacher, Kilian Rambach, Evgeny Levinkov, Thomas Brox, and Jan Hendrik Metzen

In arXiv preprint arXiv:2106.14999, 2021

arXiv PDF
Bag of Tricks for Neural Architecture Search

Thomas Elsken, Benedikt Staffler, Arber Zela, Jan Hendrik Metzen, and Frank Hutter

In CVPR21 NAS Workshop, 2021

arXiv PDF
Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks

Maksym Yatsura, Jan Metzen, and Matthias Hein

In Advances in Neural Information Processing Systems, 2021

arXiv PDF

2020

Meta-Learning of Neural Architectures for Few-Shot Learning

Thomas Elsken, Benedikt Staffler, Jan Hendrik Metzen, and Frank Hutter

In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020

arXiv PDF
Increasing the Robustness of Semantic Segmentation Models with Painting-by-Numbers

Christoph Kamann, Burkhard Güssefeld, Robin Hutmacher, Jan Hendrik Metzen, and Carsten Rother

In 16th European Conference on Computer Vision, 2020

arXiv PDF

2019

Defending against universal perturbations with shared adversarial training

Chaithanya Kumar Mummadi, Thomas Brox, and Jan Hendrik Metzen

In International Conference on Computer Vision, 2019

arXiv PDF
Efficient Multi-objective Neural Architecture Search via Lamarckian Evolution

Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter

In International Conference on Learning Representations, 2019

arXiv PDF
Neural Architecture Search: A Survey

Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter

Journal of Machine Learning Research, 2019

arXiv PDF

2018

The BesMan Learning Platform for Automated Robot Skill Learning

Lisa Gutzeit, Alexander Fabisch, Marc Otto, Jan Hendrik Metzen, Jonas Hansen, Frank Kirchner, and Elsa Andrea Kirchner

Frontiers in Robotics and AI, 2018
Scaling provable adversarial defenses

Eric Wong, Frank Schmidt, Jan Hendrik Metzen, and J Zico Kolter

In Advances in Neural Information Processing Systems, 2018

arXiv PDF

2017

Simple And Efficient Architecture Search for Convolutional Neural Networks

Thomas Elsken, Jan-Hendrik Metzen, and Frank Hutter

In NeurIPS Workshop on Meta-Learning, 2017

arXiv PDF
Universal Adversarial Perturbations Against Semantic Image Segmentation

Jan Hendrik Metzen, Mummadi Chaithanya Kumar, Thomas Brox, and Volker Fischer

In International Conference on Computer Vision, 2017

arXiv PDF
Adversarial Examples for Semantic Image Segmentation

Volker Fischer, Mummadi Chaithanya Kumar, Jan Hendrik Metzen, and Thomas Brox

In International Conference on Learning Representations (ICLR), Workshop paper, 2017

arXiv PDF
On Detecting Adversarial Perturbations

Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff

In International Conference on Learning Representations, 2017

arXiv PDF

2016

Minimum Regret Search for Single- and Multi-Task Optimization

Jan Hendrik Metzen

In nternational Conference on Machine Learning, 2016

Abs arXiv PDF

We propose minimum regret search (MRS), a novel acquisition function for Bayesian optimization. MRS bears similarities with information-theoretic approaches such as entropy search (ES). However, while ES aims in each query at maximizing the information gain with respect to the global maximum, MRS aims at minimizing the expected immediate regret of its ultimate recommendation for the optimum. While empirically ES and MRS perform similar in most of the cases, MRS produces fewer outliers with high regret than ES. We provide empirical results both for a synthetic single-task optimization problem as well as for a simulated multi-task robotic control problem.
Cognitive AutonomouS CAtheters operating in Dynamic Environments

Emmanuel Vander Poorten, and al.

Journal of Medical Robotics Research, 2016

2015

Active Contextual Entropy Search

Jan Hendrik Metzen

In Proceedings of NIPS Workshop on Bayesian Optimization, 2015
Surgical Robotics Beyond Enhanced Dexterity Instrumentation

Yohannes Kassahun, Bingbin Yu, Abraham Temesgen Tibebu, Danail Stoyanov, Stamatia Giannarou, Jan Hendrik Metzen, and Emmanuel Vander Poorten

International Journal of Computer Assisted Radiology and Surgery, 2015
Bayesian Optimization for Contextual Policy Search

Jan Hendrik Metzen, Alexander Fabisch, and Jonas Hansen

In Proceedings of the Second Machine Learning in Planning and Control of Robot Motion Workshop (MLPC-2015)., 2015

Abs

Contextual policy search allows adapting robotic movement primitives to different situations. For instance, a locomotion primitive might be adapted to different terrain inclinations or desired walking speeds. Such an adaptation is often achievable by modifying a relatively small number of hyperparameters; however, learning when performed on an actual robotic system is typically restricted to a relatively small number of trials. In black-box optimization, Bayesian optimization is a popular global search approach for addressing such problems with low-dimensional search space but expensive cost function. We present an extension of Bayesian optimization to contextual policy search. Preliminary results suggest that Bayesian optimization outperforms local search approaches on low-dimensional contextual policy search problems.
Intuitive Interaction with Robots – Technical Approaches and Challenges

Elsa Andrea Kirchner, Jose de Gea Fernandez, Peter Kampmann, Martin Schröer, Jan Hendrik Metzen, and Frank Kirchner

In Formal Modeling and Verification of Cyber-Physical Systems, 2015
Intuitive Interaction with Robots – Technical Approaches and Challenges

Elsa Andrea Kirchner, Jose de Gea Fernandez, Peter Kampmann, Martin Schröer, Jan Hendrik Metzen, and Frank Kirchner

In Formal Modeling and Verification of Cyber-Physical Systems, 2015
Accounting for Task-Difficulty in Active Multi-Task Robot Control Learning

Alexander Fabisch, Jan Hendrik Metzen, Mario Michael Krell, and Frank Kirchner

German Journal of Artificial Intelligence, May 2015

Abs

Contextual policy search is a reinforcement learning approach for multi-task learning in the context of robot control learning. It can be used to learn versatilely applicable skills that generalize over a range of tasks specified by a context vector. In this work, we combine contextual policy search with ideas from active learning for selecting the task in which the next trial will be performed. Moreover, we use active training set selection for reducing detrimental effects of exploration in the sampling policy. A core challenge in this approach is that the distribution of the obtained rewards may not be directly comparable between different tasks. We propose the novel approach PUBSVE for estimating a reward baseline and investigate empirically on benchmark problems and simulated robotic tasks to which extent this method can remedy the issue of non-comparable reward.
Concept of a Data Thread Based Parking Space Occupancy Prediction in a Berlin Pilot Region

Tim Köhler, Thomas Vögele, Mario Michael Krell, Jan Hendrik Metzen, and Frank Kirchner

In Papers from the 2015 AAAI Workshop. Workshop on AI for Transportation (WAIT-2015), January 25-26, Austin, USA, May 2015

Abs

In the presented research project, a software and Hardware infrastructure for parking space focussed inter-modal route planning in a public pilot region in Berlin is developed. One central topic is the development of a prediction system which gives an estimated occupancy for the parking spaces in the pilot region for a given date and time in the future. Occupancy data will be collected online by roadside parking sensors developed within the project. The occupancy prediction will be implemented using “Neural Gas” machine learning in combination with a proposed method which uses data threads to improve the prediction quality. In this paper, a short overview of the whole research Project is given. Furthermore, the concept of the software Framework and the learning methods are presented and first collected data is shown. The prediction method using data threads is explained in more detail.

2014

Active Contextual Policy Search

Alexander Fabisch, and Jan Hendrik Metzen

Journal of Machine Learning Research, May 2014

Abs

We consider the problem of learning skills that are versatilely applicable. One popular approach for learning such skills is contextual policy search in which the individual tasks are represented as context vectors. We are interested in settings in which the agent is able to actively select the tasks that it examines during the learning process. We argue that there is a better way than selecting each task equally often because some tasks might be easier to learn at the beginning and the knowledge that the agent can extract from these tasks can be transferred to similar but more dificult tasks. The methods that we propose for addressing the task-selection problem model the learning process as a nonstationary multi-armed bandit problem with custom intrinsic reward heuristics so that the estimated learning progress will be maximized. This approach does neither make any assumptions about the underlying contextual policy search algorithm nor about the policy representation. We present empirical results on an artificial benchmark problem and a ball throwing problem with a simulated Mitsubishi PA-10 robot arm.
Velocity-Based Multiple Change-point Inference for Unsupervised Segmentation of Human Movement Behavior

Lisa Senger, Martin Schröer, Jan Hendrik Metzen, and Elsa A. Kirchner

In Proceedings of the 22nd International Conference on Pattern Recognition, May 2014

Abs

In order to transfer complex human behavior to a robot, segmentation methods are needed which are able to detect central movement patterns that can be combined to generate a wide range of behaviors. We propose an algorithm that segments human movements into behavior building blocks in a fully automatic way, called velocity-based multiple change-point inference (vMCI). Based on characteristic bell-shaped velocity patterns that can be found in point-to-point arm movements, the algorithm infers segment borders using Bayesian inference. Different segment lengths and variations in the movement execution can be handled. Moreover, the number of segments the movement is composed of need not be known in advance. Several experiments are performed on synthetic and motion capturing data of human movements to compare vMCI with other techniques for unsupervised segmentation. The results show that vMCI is able to detect segment borders even in noisy data and in demonstrations with smooth transitions between segments.
Learning the Structure of Continuous Markov Decision Processes

Jan Hendrik Metzen

May 2014

Abs

There is growing interest in artificial, intelligent agents which can operate autonomously for an extended period of time in complex environments and fulfill a variety of different tasks. Such agents will face different problems during their lifetime which may not be foreseeable at the time of their deployment. Thus, the capacity for lifelong learning of new behaviors is an essential prerequisite for this kind of agents as it enables them to deal with unforeseen situations. However, learning every complex behavior anew from scratch would be cumbersome for the agent. It is more plausible to consider behavior to be modular and let the agent acquire a set of reusable building blocks for behavior, the so-called skills. These skills might, once acquired, facilitate fast learning and adaptation of behavior to new situations. This work focuses on computational approaches for skill acquisition, namely which kind of skills shall be acquired and how to acquire them. The former is commonly denoted as "skill discovery" and the latter as "skill learning". The main contribution of this thesis is a novel incremental skill acquisition approach which is suited for lifelong learning. In this approach, the agent learns incrementally a graph-based representation of a domain and exploits certain properties of this graph such as its bottlenecks for skill discovery. This thesis proposes a novel approach for learning a graph-based representation of continuous domains based on formalizing the problem as a probabilistic generative model. Furthermore, a new incremental agglomerative clustering approach for identifying bottlenecks of such graphs is presented. Thereupon, the thesis proposes a novel intrinsic motivation system which enables an agent to intelligently allocate time between skill discovery and skill learning in developmental settings, where the agent is not constrained by external tasks. The results of this thesis show that the resulting skill acquisition approach is suited for continuous domains and can deal with domain stochasticity and different explorative behavior of the agent. The acquired skills are reusable and versatile and can be used in multi-task and lifelong learning settings in high-dimensional problems.
Towards Learning of Generic Skills for Robotic Manipulation

Jan Hendrik Metzen, Alexander Fabisch, Lisa Senger, Jose Gea Fernandez, and Elsa Andrea Kirchner

German Journal of Artificial Intelligence, May 2014

the paper is available at \urlhttp://link.springer.com/article/10.1007%2Fs13218-013-0280-1

Abs

Learning versatile, reusable skills is one of the key prerequisites for autonomous robots. Imitation and reinforcement learning are among the most prominent approaches for learning basic robotic skills. However, the learned skills are often very specific and cannot be reused in different but related tasks. In the project BesMan, we develop hierarchical and transfer learning methods which allow a robot to learn a repertoire of versatile skills that can be reused in different situations. The development of new methods is closely integrated with the analysis of complex human behavior.

2013

pySPACE - A Signal Processing and Classification Environment in Python

Mario M. Krell, Sirko Straube, Anett Seeland, Hendrik Wöhrle, Johannes Teiwes, Jan H. Metzen, Elsa A. Kirchner, and Frank Kirchner

Frontiers in Neuroinformatics, Dec 2013

Abs

In neuroscience large amounts of data are recorded to provide insights into cerebral information processing and function. The successful extraction of the relevant signals becomes more and more challenging due to increasing complexities in acquisition techniques and questions addressed. Here, automated signal processing and machine learning tools can help to process the data, e.g., to separate signal and noise. With the presented software pySPACE (http://pyspace.github.io/pyspace), signal processing algorithms can be compared and applied automatically on time series data, either with the aim of finding a suitable preprocessing, or of training supervised algorithms to classify the data. pySPACE originally has been built to process multi-sensor windowed time series data, like event-related potentials from the electroencephalogram (EEG). The software provides automated data handling, distributed processing, modular build-up of signal processing chains and tools for visualization and performance evaluation. Included in the software are various algorithms like temporal and spatial filters, feature generation and selection, classification algorithms and evaluation schemes. Further, interfaces to other signal processing tools are provided and, since pySPACE is a modular framework, it can be extended with new algorithms according to individual needs. In the presented work, the structural hierarchies are described. It is illustrated how users and developers can interface the software and execute offline and online modes. Configuration of pySPACE is realized with the YAML format, so that programming skills are not mandatory for usage. The concept of pySPACE is to have one comprehensive tool that can be used to perform complete signal processing and classification tasks. It further allows to define own algorithms, or to integrate and use already existing libraries.
Incremental Learning of Skill Collections based on Intrinsic Motivation

Jan Hendrik Metzen, and Frank Kirchner

Frontiers in Neurorobotics, Jul 2013

Abs

Life-long learning of reusable, versatile skills is a key prerequisite for embodied agents that act in a complex, dynamic environment and are faced with different tasks over their lifetime. We address the question of how an agent can learn useful skills efficiently during a developmental period, i.e., when no task is imposed on him and no external reward signal is provided. Learning of skills in a developmental period needs to be incremental and self-motivated. We propose a new incremental, task-independent skill discovery approach that is suited for continuous domains. Furthermore, the agent learns specific skills based on intrinsic motivation mechanisms that determine on which skills learning is focused at a given point in time. We evaluate the approach in a reinforcement learning setup in two continuous domains with complex dynamics. We show that an intrinsically motivated, skill learning agent outperforms an agent which learns task solutions from scratch. Furthermore, we compare different intrinsic motivation mechanisms and how efficiently they make use of the agent’s developmental period.
Learning Skill Templates for Parameterized Tasks

Jan Hendrik Metzen, and Alexander Fabisch

In 11th European Workshop on Reinforcement Learning, (EWRL 2013), Aug 2013

accepted

Abs
Learning Graph-based Representations for Continuous Reinforcement Learning Domains

Jan Hendrik Metzen

In Proceedings of the European Conference on Machine Learning, (ECML 2013), Sep 2013

the paper is available at \urlhttp://link.springer.com/chapter/10.1007%2F978-3-642-40988-2_6#

Abs

Graph-based domain representations have been used in discrete reinforcement learning domains as basis for, e.g., autonomous skill discovery and representation learning. These abilities are also highly relevant for learning in domains which have structured, continuous state spaces as they allow to decompose complex problems into simpler ones and reduce the burden of hand-engineering features. However, since graphs are inherently discrete structures, the extension of these approaches to continuous domains is not straight-forward. We argue that graphs should be seen as discrete, generative models of continuous domains. Based on this intuition, we define the likelihood of a graph for a given set of observed state transitions and derive a heuristic method entitled FIGE that allows to learn graph-based representations of continuous domains with large likelihood. Based on FIGE, we present a new skill discovery approach for continuous domains. Furthermore, we show that the learning of representations can be considerably improved by using FIGE.
Comparison of Sensor Selection Mechanisms for an ERP-Based Brain-Computer Interface

David Feess, Mario Michael Krell, and Jan Hendrik Metzen

PLoS ONE, Jul 2013

Abs

A major barrier for a broad applicability of brain-computer interfaces (BCIs) based on electroencephalography (EEG) is the large number of EEG sensor electrodes typically used. The necessity for this results from the fact that the relevant information for the BCI is often spread over the scalp in complex patterns that differ depending on subjects and application scenarios. Recently, a number of methods have been proposed to determine an individual optimal sensor selection. These methods have, however, rarely been compared against each other or against any type of baseline. In this paper, we review several selection approaches and propose one additional selection criterion based on the evaluation of the performance of a BCI system using a reduced set of sensors. We evaluate the methods in the context of a passive BCI system that is designed to detect a P300 event-related potential and compare the performance of the methods against randomly generated sensor constellations. For a realistic estimation of the reduced system’s performance we transfer sensor constellations found on one experimental session to a different session for evaluation. We identified notable (and unanticipated) differences among the methods and could demonstrate that the best method in our setup is able to reduce the required number of sensors considerably. Though our application focuses on EEG data, all presented algorithms and evaluation schemes can be transferred to any binary classification task on sensor arrays.

2012

Online Skill Discovery using Graph-based Clustering

Jan Hendrik Metzen

Journal of Machine Learning Research, Jul 2012

Abs

We introduce a new online skill discovery method for reinforcement learning in discrete domains. The method is based on the bottleneck principle and identifies skills using a bottom-up hierarchical clustering of the estimated transition graph. In contrast to prior clustering approaches, it can be used incrementally and thus several times during the learning process. Our empirical evaluation shows that “assuming high connectivity in the face of uncertainty” can prevent premature identification of skills. Furthermore, we show that the choice of the linkage criterion is crucial for dealing with non-random sampling policies and stochastic environments.
Online Skill Discovery using Graph-based Clustering

Jan Hendrik Metzen

In 10th European Workshop on Reinforcement Learning, (EWRL 2012), Jun 2012

Abs

We introduce a new online skill discovery method for reinforcement learning in discrete domains. The method is based on the bottleneck principle and identifies skills using a bottom-up hierarchical clustering of the estimated transition graph. In contrast to prior clustering approaches, it can be used incrementally and thus several times during the learning process. Our empirical evaluation shows that “assuming high connectivity in the face of uncertainty” can prevent premature identification of skills. Furthermore, we show that the choice of the linkage criterion is crucial for dealing with non-random sampling policies and stochastic environments.
Model-based Evolutionary Policy Search for Skill Learning in Continuous Domains

Jan Hendrik Metzen

In 10th European Workshop on Reinforcement Learning, (EWRL 2012), Jun 2012

2011

Choosing an Appropriate Performance Measure: Classification of EEG-Data with Varying Class Distribution

Sirko Straube, Jan Hendrik Metzen, Anett Seeland, Mario Krell, and Elsa Andrea Kirchner

In Proceedings of the 41st Meeting of the Society for Neuroscience 2011, Nov 2011
Rapid Adaptation of Brain Reading Interfaces based on Threshold Adjustment

Jan Hendrik Metzen, and Elsa Andrea Kirchner

In Proceedings of the 2011 Conference of the German Classification Society, (GfKl-2011), Aug 2011
Minimizing Calibration Time for Brain Reading

Jan Hendrik Metzen, Su-Kyoung Kim, and Elsa Andrea Kirchner

In Pattern Recognition, Aug 2011

The original publication is available under \urlhttp://link.springer.com/chapter/10.1007%2F978-3-642-23123-0_37

Abs

Machine learning is increasingly used to autonomously adapt brain-machine interfaces to user-specific brain patterns. In order to minimize the preparation time of the system, it is highly desirable to reduce the length of the calibration procedure, during which training data is acquired from the user, to a minimum. One recently proposed approach is to reuse models that have been trained in historic usage sessions of the same or other users by utilizing an ensemble-based approach. In this work, we propose two extensions of this approach which are based on the idea to combine predictions made by the historic ensemble with session-specific predictions that become available once a small amount of training data has been collected. These extensions are particularly useful for Brain Reading Interfaces (BRIs), a specific kind of brain-machine interfaces. BRIs do not require that user feedback is given and thus, additional training data may be acquired concurrently to the usage session. Accordingly, BRIs should initially perform well when only a small amount of training data acquired in a short calibration procedure is available and allow an increased performance when more training data becomes available during the usage session. An empirical offline-study in a testbed for the use of BRIs to support robotic telemanipulation shows that the proposed extensions allow to achieve this kind of behavior.
On Transferring Spatial Filters in a Brain Reading Scenario

Jan Hendrik Metzen, Su Kyoung Kim, Timo Duchrow, Elsa Andrea Kirchner, and Frank Kirchner

In Statistical Signal Processing Workshop (SSP), 2011 IEEE, Jun 2011

Abs

Machine learning approaches are increasingly used in brain-machine-interfaces to allow automatic adaptation to user-specific brain patterns. One of the most crucial factors for the practical success of these systems is that this adaptation can be achieved with a minimum amount of training data since training data needs to be recorded during a calibration procedure prior to the actual usage session. To this end, one promising approach is to reuse models based on data recorded in preceding sessions of the same or of other users. In this paper, we investigate under which conditions it is favorable to reuse models (more specifically spatial filters) trained on data from historic sessions compared to learning new spatial filters on the current session’s calibration data. We present an empirical study in a scenario in which Brain Reading, a particular kind of brain-machine-interface, is used to support robotic telemanipulation.

2010

Towards Operator Monitoring via Brain Reading - An EEG-based Approach for Space Applications

Elsa Andrea Kirchner, Hendrik Wöhrle, Constantin Bergatt, Su-Kyoung Kim, Jan Hendrik Metzen, David Feess, and Frank Kirchner

In Proceedings of the 10th International Symposium on Artificial Intelligence, Robotics and Automation in Space (iSAIRAS-10), Sep 2010
Model-based direct policy search

Jan Hendrik Metzen, and Frank Kirchner

In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems, Sep 2010

2009

Learning to Play the BRIO Labyrinth Game

Jan Hendrik Metzen, Elsa Andrea Kirchner, Larbi Abdenebaoui, and Frank Kirchner

Zeitschrift für Künstliche Intelligenz, Sep 2009
Matching of anatomical tree structures for registration of medical images

Jan Hendrik Metzen, Tim Kröger, Andrea Schenk, Stephan Zidowitz, Heinz-Otto Peitgen, and Xiaoyi Jiang

Image and Vision Computing, Jun 2009

Abs

Many medical applications require a registration of different images of the same organ. In many cases, such a registration is accomplished by manual placement of landmarks in the images. In this paper we propose a method which is able to find reasonable landmarks automatically. To achieve this, bifurcations of the vessel systems, which have been extracted from the images by a segmentation algorithm, are assigned by the so-called association graph method and the coordinates of these matched bifurcations can be used as landmarks for a non-rigid registration algorithm. Several constraints to be used in combination with the association graph method are proposed and evaluated on a ground truth consisting of anatomical trees from liver and lung. Furthermore, a method for preprocessing (tree pruning) as well as for postprocessing (clique augmentation) are proposed and evaluated on this ground truth. The proposed method achieves promising results for anatomical trees of liver and lung and for medical images obtained with different modalities and at different points in time.
Quantification and Minimization of the Simulation-Reality-Gap on a BRIO Labyrinth Game

Constantin Bergatt, Jan Hendrik Metzen, Elsa Andrea Kirchner, and Frank Kirchner

In Proceedings of the first International Workshop on Learning and Data Mining for Robotics (LEMIR-09), Jun 2009
Assisting Telemanipulation Operators via Real-Time Brain Reading

Elsa Andrea Kirchner, Jan Hendrik Metzen, Timo Duchrow, Su Kyong Kim, and Frank Kirchner

In Lemgoer Schriftenreihe zur industriellen Informationstechnik, Sep 2009
Incremental Acquisition of Neural Structures through Evolution

Yohannes Kassahun, Jan Hendrik Metzen, Mark Edgington, and Frank Kirchner

In Design and Control of Intelligent Robotic Systems, Sep 2009

2008

Evolving Neural Networks for Online Reinforcement Learning

Jan Hendrik Metzen, Mark Edgington, Yohannes Kassahun, and Frank Kirchner

In Parallel Problem Solving from Nature – PPSN X, Sep 2008

Abs

For many complex Reinforcement Learning problems with large and continuous state spaces, neuroevolution (the evolution of artificial neural networks) has achieved promising results. This is especially true when there is noise in sensor and/or actuator signals. These results have mainly been obtained in offline learning settings, where the training and evaluation phase of the system are separated. In contrast, in online Reinforcement Learning tasks where the actual performance of the systems during its learning phase matters, the results of neuroevolution are significantly impaired by its purely exploratory nature, meaning that it does not use (i.e. exploit) its knowledge of the performance of single individuals in order to improve its performance during learning. In this paper we describe modifications which significantly improve the online performance of the neuroevolutionary method Evolutionary Acquisition of Neural Topologies (EANT) and discuss the results obtained on two benchmark problems.
Learning Walking Patterns for Kinematically Complex Robots Using Evolution Strategies

Malte Römmermann, Mark Edgington, Jan Hendrik Metzen, Jose Gea, Yohannes Kassahun, and Frank Kirchner

In Parallel Problem Solving from Nature – PPSN X, Sep 2008
Accelerating neuroevolutionary methods using a Kalman filter

Yohannes Kassahun, Jose Gea, Mark Edgington, Jan Hendrik Metzen, and Frank Kirchner

In GECCO ’08: Proceedings of the 10th annual conference on Genetic and evolutionary computation, Sep 2008
Towards efficient online reinforcement learning using neuroevolution

Jan Hendrik Metzen, Frank Kirchner, Mark Edgington, and Yohannes Kassahun

In Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation, Sep 2008
Analysis of an evolutionary reinforcement learning method in a multiagent domain

Jan Hendrik Metzen, Mark Edgington, Yohannes Kassahun, and Frank Kirchner

In Proceedings of the 7th International Conference on Autonomous Agents and Multiagent Systems, May 2008

Abs

Many multiagent problems comprise subtasks which can be considered as reinforcement learning (RL) problems. In addition to classical temporal difference methods, evolutionary algorithms are among the most promising approaches for such RL problems. The relative performance of these approaches in certain subdomains (e. g. multiagent learning) of the general RL problem remains an open question at this time. In addition to theoretical analysis, benchmarks are one of the most important tools for comparing different RL methods in certain problem domains. A recently proposed multiagent RL benchmark problem is the RoboCup Keepaway benchmark. This benchmark is one of the most challenging multiagent learning problems because its state-space is continuous and high dimensional, and both the sensors and the actuators are noisy. In this paper we analyze the performance of the neuroevolutionary approach called Evolutionary Acquisition of Neural Topologies (EANT) in the Keepaway benchmark, and compare the results obtained using EANT with the results of other algorithms tested on the same benchmark.

2007

Performance Evaluation of EANT in the RoboCup Keepaway Benchmark

Jan Hendrik Metzen, Mark Edgington, Yohannes Kassahun, and Frank Kirchner

In ICMLA ’07: Proceedings of the Sixth International Conference on Machine Learning and Applications, May 2007
A common genetic encoding for both direct and indirect encodings of networks

Yohannes Kassahun, Mark Edgington, Jan H Metzen, Gerald Sommer, and Frank Kirchner

In GECCO ’07: Proceedings of the 9th annual conference on Genetic and evolutionary computation, Jul 2007
A General Framework for Encoding and Evolving Neural Networks

Yohannes Kassahun, Jan Metzen, Jose Gea, Mark Edgington, and Frank Kirchner

In KI 2007: Advances in Artificial Intelligence, Jul 2007
Matching of Tree Structures for Registration of Medical Images

Jan Metzen, Tim Kröger, Andrea Schenk, Stephan Zidowitz, Heinz-Otto Peitgen, and Xiaoyi Jiang

In Graph-Based Representations in Pattern Recognition, Jul 2007

Abs

Many medical applications require a registration of different images of the same organ. In many cases, such a registration is accomplished by manually placing landmarks in the images. In this paper we propose a method which is able to find reasonable landmarks automatically. To achieve this, nodes of the vessel systems, which have been extracted from the images by a segmentation algorithm, will be assigned by the so-called association graph method and the coordinates of these matched nodes can be used as landmarks for a non-rigid registration algorithm.
Matching von Baumstrukturen - Zuordnung von Gefäßsystemen aus Leber und Lunge

Jan Hendrik Metzen, Tim Kröger, Andrea Schenk, Stephan Zidowitz, Heinz-Otto Peitgen, and Xiaoyi Jiang

In Bildverarbeitung für die Medizin 2007, Mar 2007

2006

Matching von Baumstrukturen in der medizinischen Bildverarbeitung

Jan Hendrik Metzen

Westfälische Wilhelms-Universität Münster, Jul 2006

2005

Rokkatan: Scaling an RTS Game Design to the Massively Multiplayer Realm

Jens Müller, Jan Hendrik Metzen, Alexander Ploss, Maraike Schellmann, and Sergei Gorlatch

In ACM SIGHCHI International Conference on Advances in Computer Entertainment Technology (ACE 05), Jun 2005