Conférence
Notice
Langue :
Anglais
Crédits
CNRS - Centre National de la Recherche Scientifique (Publication), INRIA (Institut national de recherche en informatique et automatique) (Production), INRIA (Institut national de recherche en informatique et automatique) (Publication), UNS (Publication), Vittorio Ferrari (Intervention)
Conditions d'utilisation
Droit commun de la propriété intellectuelle
DOI : 10.60527/k3pq-a055
Citer cette ressource :
Vittorio Ferrari. Inria. (2018, 28 juin). Knowledge transfer and human-machine collaboration for training object class detector. [Vidéo]. Canal-U. https://doi.org/10.60527/k3pq-a055. (Consultée le 19 mars 2024)

Knowledge transfer and human-machine collaboration for training object class detector

Réalisation : 28 juin 2018 - Mise en ligne : 26 juillet 2018
  • document 1 document 2 document 3
  • niveau 1 niveau 2 niveau 3
Descriptif

Object class detection is a central area of computer vision. It requires recognizing and localizing all objects of predefined set of classes in an image. Detectors are usually trained under full supervision, which requires manually drawing object bounding-boxes in a large number of training images. This is tedious and very time consuming.  In this talk I will present two recent techniques for reducing this effort.

In the first part I will explore a knowledge transfer scenario: training object detectors for target classes with only image-level labels, helped by a set of source classes with bounding-box annotations. I will present a unified knowledge transfer framework based on training a single neural network multi-class object detector over all source classes, organized in a semantic hierarchy. This generates proposals with scores at multiple levels in the hierarchy, which we use to explore knowledge transfer over a broad range of generality, ranging from class-specific (bicycle to motorbike) to class-generic (objectness to any class).  Experiments on 200 object classes from the ILSVRC 2013 dataset demonstrate large improvements over weakly supervised baselines. Moreover, we also carry out several across-dataset knowledge transfer experiments, which establish the general applicability of our method

In the second part I will consider a human-machine collaboration scenario, where a human interacts with a computer model to carry the bounding-box annotation process together. I will introduce Intelligent Annotation Dialogs: we train an agent to automatically choose a sequence of actions for a human annotator to produce a bounding box in a minimal amount of time. We consider two actions: box verification, where the annotator verifies a box generated by an object detector, and manual box drawing. We explore two kinds of agents, one based on predicting the probability that a box will be positively verified, and the other based on reinforcement learning. We experimentally demonstrate that our agents are able to learn efficient annotation strategies in several scenarios, automatically adapting to the image difficulty, the desired quality of the boxes, and the detector strength.

Intervention

Sur le même thème