Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Continual learning using hash-routed convolutional neural networks

Abstract : Continual learning could shift the machine learning paradigm from data centric to model centric. A continual learning model needs to scale efficiently to handle semantically different datasets, while avoiding unnecessary growth. We introduce hash-routed convolutional neural networks: a group of convolutional units where data flows dynamically. Feature maps are compared using feature hashing and similar data is routed to the same units. A hash-routed network provides excellent plasticity thanks to its routed nature, while generating stable features through the use of orthogonal feature hashing. Each unit evolves separately and new units can be added (to be used only when necessary). Hash-routed networks achieve excellent performance across a variety of typical continual learning benchmarks without storing raw data and train using only gradient descent. Besides providing a continual learning framework for supervised tasks with encouraging results, our model can be used for unsupervised or reinforcement learning.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

Cited literature [29 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02961848
Contributor : Ahmad Berjaoui <>
Submitted on : Thursday, October 8, 2020 - 5:33:28 PM
Last modification on : Wednesday, November 18, 2020 - 3:18:02 PM

Files

arxiv.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02961848, version 1
  • ARXIV : 2010.05880

Collections

Citation

Ahmad Berjaoui. Continual learning using hash-routed convolutional neural networks. 2020. ⟨hal-02961848⟩

Share

Metrics

Record views

20

Files downloads

56