Titouan Parcollet

Funded Projects

SpeechBrain: simplify the access to speech technologies (2021 - 2022)
Principal Investigator

The Speechbrain project (see bellow for mor info) has been granted 400K hours (i.e. 234 000€) of GPU time in the French datacenter Jean Zay. This project is a joint collaboration between the Laboratoire Informatique d'Avignon (LIA) , Le Laboratoire de Traitement et Communication de l'Information (LTCI), the Laboratoire des Sciences du Numérique de Nantes (LS2N) and the Laboratoire Interdisciplinaire des Sciences du Numérique (LISN) that aims to support the effort devoted to SpeechBrain towards a democratization of the research and developpement of speech technologies. This project will gather top-tier researchers to build and release state-of-the-art and ground-breaking systems for speech translation, self-supervised learning of speech representations, speaker verification and identification, voice privacy, spoken language understanding, speech synthesis, speech for e-health and speech enhancement.

The consortium is composed with: Assoc. Prof. Titouan PARCOLLET (PI, LIA), Prof. Yannick ESTÈVE (LIA), Prof. Corinne FREDOUILLE (LIA), Prof. Jean-François BONASTRE (LIA), Prof. Richard DUFOUR (LS2N), Prof. Slim ESSID (LTCI), Assoc. Prof. Sahar GANNAY (LISN).

LeBenchmark (2020 - 2021)
Co-Principal Investigator for SSL models

LeBenchmark project has been granted 200K hours (i.e. 117 000€) of GPU time in the French datacenter Jean Zay. This project is a joint collaboration between the Laboratoire d'Informatique de Grenoble (LIG) and the Laboratoire Informatique d'Avignon (LIA) that aims to collect large quantities of raw speech in French (i.e. several thousand hours) with different styles (read speech, prepared speech, spontaneous speech), from various speakers and use them to learn self-supervised models to be shared with the research community. Furthermore, we also wish to establish a new benchmark data set for several speech processing tasks. I am in charge of finding the appropriate self-supervised methods (e.g wav2vec, PASE, MockingJay) and to deploy it on our new dataset. We wish to release various pre-trained models with a training set acounting for up to 10,000 hours of speech in French. Everything is nicely integrated to SpeechBrain for peoples interested in re-using our models for downstream tasks!

The consortium is composed with: Prof. Laurent BESACIER (PI, LIG), Prof. François PORTER (LIG), Assoc. Prof. Solange ROSSATO (LIG), Assoc. Prof. Benjamin LECOUTEUX (LIG), Assoc. Prof. Didier SCHWAB (LIG), Assoc. Prof. Fabien RINGEVAL, Dr., CR, Marco DINARELLI (CNRS, LIG), Prof. Alexandre ALLAUZEN (LAMSADE), Assoc. Prof. Titouan PARCOLLET (LIA), Prof. Yannick ESTÈVE (LIA).

Models are available on HuggingFace .
Publications:

• Task Agnostic and Task Specific Self-Supervised Learning from Speech with LeBenchmark
• Solène Evain · Ha Nguyen · Hang Le · Marcely Zanon Boito · Salima Mdhaffar · Salima Mdhaffar · Sina Alisamir · Ziyi Tong · Natalia Tomashenko · Marco Dinarelli · Titouan Parcollet · Alexandre Allauzen · Yannick Estève · Benjamin Lecouteux · François Portet · Solange Rossato · Fabien Ringeval · Didier Schwab · Laurent Besacier
• NeurIPS 2021.

• LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speech
• Solène Evain · Ha Nguyen · Hang Le · Marcely Zanon Boito · Sina Alisamir · Ziyi Tong · salima mdhaffar · Natalia Tomashenko · Marco Dinarelli · Titouan parcollet · Alexandre Allauzen · Yannick Estève · Benjamin Lecouteux · François Portet · Solange Rossato · Fabien Ringeval · Didier Schwab · Laurent Besacier
• Interspeech 2021.

SpeechBrain (2019 - )
Creator & Co-Principal Investigator

The SpeechBrain project aims to develop an open-source and all-in-one toolkit based on PyTorch. The goal is to develop a single, flexible, and user-friendly toolkit that can be used to easily develop state-of-the-art speech systems for speech recognition (both end-to-end and HMM-DNN), speaker recognition, speech separation, multi-microphone signal processing (e.g, beamforming), self-supervised learning, and many others. The project is funded thanks to generous donations from an ever growing number of sponsors: the Laboratoire Informatique d'Avignon (LIA), the Montréal Institute for Learning Algorithms (MILA), Samsung, HuggingFace, NVIDIA, Dolby, ViaDialog, OVH and NAVER Labs. SpeechBrain also benefits from the collaboration and expertise of around 20 different partner institutions (both academics and industrials) ranging from the University of Cambridge to the PyTorch Team. SpeechBrain already beats all the other toolkits on the considered datasets and with a much easier interface to play with. SpeechBrain reached 3.7K stars on GitHub in less than 7 months demonstrating a clear interest from the community for our toolkit. I am the co-creator and co-leader of SpeechBrain with Dr. Ravanelli Mirco, currently a post-doctoral researcher at MILA.

Have a look at: SpeechBrain! Or to our open access paper describing the toolkit!

Advised & Mentored Ph.D. Students and Interns

Jarod DURET — Ph.D. Student — Started in October 2021 and is co-advised with Prof. Yannick ESTÈVE ESSID from LIA
Title: Expressive Speech to Speech Translation
Ideas: Current speech to speech translation systems rely on cascade systems that combine both a speech to text and a text to speech block. In practice, existing end-to-end speech to speech translation systems do not work well and often completely bypass the concept of expressivity. With this thesis, we aim at inventing a simple end-to-end model enabling an expressivity transfer alongside with the translation. This appears as being particularly challenging as the expressivity may be defined differently from one language to an other. Multiple directions will be investigated, starting with the design of better speech representations capturing the expressivity to condition the generation of the speech signal.

Salah ZAIEM — Ph.D. Student — Started in October 2020 and is co-advised with Prof. Slim ESSID from Telecom Paris Sud.
Title: Informed Self-Supervised Speech Representations Learning.
Ideas: Self-supervised learning methods for speech are mostly empirically driven. In particular, there exist very few theoretical evidences on why a method performs better than an other one. With this thesis, we aim at providing theoreticaly grounded tools to design SSL models in an informed manner. For instance, we developed a solution to design a PASE-like architecture without the need for pre-text task search with empirical validation, potentially saving weeks of training (and compute / carbon emissions).

Publications:

• Pretext Tasks Selection for Multitask Self-Supervised Audio Representation Learning
• Salah Zaiem · Titouan parcollet · Slim Essid
• Submitted to the IEEE Journal of Selected Topics in Signal Processing.

• Pretext Tasks Selection for Multitask Self-Supervised Speech Representation Learning
• Salah Zaiem · Titouan parcollet · Slim Essid
• Submitted to NeurIPS 2022.

• End-to-End Speech Recognition from Federated Acoustic Models
• Yan Gao · Titouan parcollet · Salah Zaiem · Javier Fernandez-Marques · Pedro P. B. de Gusmao · Daniel J. Beutel · Nicholas Lane
• ICASSP 2022.

• Conditional Independence for Pretext Task Selection in Self-Supervised Speech Representation Learning
• Salah Zaiem · Titouan parcollet · Slim Essid
• Interspeech 2021. (ISCA Student Travel Grant)

Xinchi QIU — Mentored Ph.D. Student — Started in Sept. 2019, advised by Prof. Nicholas Lane from the University of Cambridge.
Title: Efficient Federated Learning
Ideas: With Xinchi, we wondered if the concept of federated learning that will soon cover a large part of the deep learning use cases could be more efficient than centralised training. With that in mind, we started by investigating its energy footprint before jumping into practical ways of increasing its efficiency including high-dimensional neural networks, parameters sharing or sparsity.

Publications:

• A First Look Into the Carbon Footprint of Federated Learning
• Xinchi Qiu · Titouan Parcollet · Javier Fernandez-Marques · Pedro Porto Buarque de Gusmao · Daniel J. Beutel · Taner Topal · Akhil Mathur · Nicholas D. Lane
• Submitted to Journal of Machine Learning Research (JMLR) IF: 4.091

• ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsity
• Xinchi Qiu · Javier Fernandez-Marques · Pedro P. B. de Gusmao · Yan Gao · Titouan Parcollet · Nicholas Lane
• ICLR 2022.

• Can Federated Learning Save The Planet ?
• Xinchi Qiu · Titouan Parcollet · Nicholas Lane
• NeurIPS 2020 : Tackling Climate Change with Machine Learning Workshop

• FusionRNN: Shared Neural Parameters for Multi-Channel Distant Speech Recognition
• Titouan Parcollet · Xinchi Qiu · Nicholas Lane
• INTERSPEECH 2020
• October 25-29, Virtual Conference (COVID)

• Quaternion Neural Networks for Multi-channel Distant Speech Recognition
• Xinchi Qiu · Titouan Parcollet · Mirco Ravanelli · Nicholas Lane · Mohamed Morchid
• INTERSPEECH 2020
• October 25-29, Virtual Conference (COVID)

Yan Gao — Mentored Ph.D. Student — Started in Sept. 2019, advised by Prof. Nicholas Lane from the University of Cambridge.
Title: Federated Speech Recognition
Ideas: Yan explores different ways of enabling on-device speech recognition training with edge data. In practice, state-of-the-art speech recognizers are highly resource intensive and we build the foundations necessary to properly assess the difficulty arising when such models are deployed on device. For instance, Yan investigates different solutions to aggregate numerous acoustic models coming from large pools of devices under a federated learning setup. His latest work tries to highlight the unstainability of current large scale self-supervised speech models under constrained resources.

Publications:

• End-to-End Speech Recognition from Federated Acoustic Models
• Yan Gao · Titouan parcollet · Salah Zaiem · Javier Fernandez-Marques · Pedro P. B. de Gusmao · Daniel J. Beutel · Nicholas Lane
• ICASSP 2022.

• ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsity
• Xinchi Qiu · Javier Fernandez-Marques · Pedro P. B. de Gusmao · Yan Gao · Titouan Parcollet · Nicholas Lane
• ICLR 2022.

• Distilling Knowledge from Ensembles of Acoustic Models for Joint CTC-Attention End-to-End Speech Recognition
• Yan Gao · Titouan Parcollet · Nicholas Lane
• ASRU 2021.
• 13-17 December, Cartagena (Columbia).

Adel Moumen — Undergrad. Intern — June 2021 to August 2021 and June 2022 to August 2022.
Title: On the limitations of LiGRU networks.
Ideas: Adel investigates different ways of making LiGRU a mandatory alternative to LSTM/GRU for speech processing. There are theoretical and empirical evidences that LiGRU simply are better than LSTM and GRU. Unfortunately, they suffer from a poor GPU implementation and an instability in the recurrent connection. These problems will be tackled during the internship.

SpeechBrain Full-Time Research Engineers — Started on January 2022.
Format: this is a list of all the researchers working full-time on SpeechBrain that I recruited and supervised.
Topics: all the topics and concepts developed within SpeechBrain.

Dr. Andreas Nautsch

SpeechBrain Interns — Started on January 2020.
Format: this is a list of all the researchers that I mentored during their internship to work on SpeechBrain. Internships took place either in Avignon (LIA) or Montréal (Mila) and were co-advised with Dr. Ravanelli
Topics: all the topics and concepts developed within SpeechBrain.

Mohamed Anwar, Naver LABS Europe (FR).
Aku Rouhe, Aalto University (FI).
Peter Plantinga, now at JP Morgan.
Loren Lugosch, Mila (CA).
Nauman Dawalatabad, now at MIT (USA).
Ju-Chieh Chou, National Taiwan University (TW).
Sung-Lin Yeh, now at University of Edinburg (UK).
Hwidong Na, Samsung SAIL (CA).
Abdel Heba, Linaroga / University of Toulouse (FR).
Samuele Cornell, now at Amazon.
Jianyuan Zhong, University of Rochester (USA).
Cem Subakan, University of Montréal (CA).
Szu-Wei Fu, Academia Sinica (TW).

Community Contributions

General Co-Chair

RECITAL (TALN session), June 28th (2022), Avignon (France).

Workshop & Session Co-Organizer

INTERSPEECH Green Speech Recognition Challenge , September 18th (2022), Incheon (Souh Korea).
ICML self-Supervision in Audio and Speech (held virtually), July 17th (2020), Vienna (Austria).

Tutorials

Interspeech 2022: "State of the PyTorch Ecosystem for Speech Technologies", August 2022.
IEEE ASRU 2021: "SpeechBrain: Unifying Speech Technologies and Deep Learning With an Open Source Toolkit", December 2021.
Interspeech 2021: "SpeechBrain: Unifying Speech Technologies and Deep Learning With an Open Source Toolkit", August 2021.
University of Sheffield: "SpeechBrain: Unifying Speech Technologies and Deep Learning With an Open Source Toolkit", June 2021.

Invited Talks

ADASP Group, Télécom Paris: "SpeechBrain: A General-Purpose Speech Toolkit", March 6th (2022).
Idiap Research Institute: "SpeechBrain: A General-Purpose Speech Toolkit", April 27th (2022).
Machine Intelligence Laboratory, University of Cambridge: "SpeechBrain: A General-Purpose Speech Toolkit", March 14th (2022).
Naver Labs Europe: "SpeechBrain: A General-Purpose Speech Toolkit", November 30th (2021).
FestivalIA Avignon: "SpeechBrain : un outil polyvalent pour le traitement automatique de la parole", November 17th (2021).
Microsoft Research Summit Workshop on Federated Learning and Confidential Computing: "Federated Speech Technologies", October 21th (2021).
Machine Learning Summer Schools — Taipei: "Task Agnostic and Task Specific Self-Supervised Learning from Speech with LeBenchmark", August 20th (2021).
The Machine Learning / Data Science Meetup of Rome: "SpeechBrain: A General-Purpose Speech Toolkit", July 7th (2021).
The 2nd Annual Federated & Distributed / Decentralized Machine Learning Conference (remote): "Can Federated Learning Save the Planet", June 16th (2021).
Flower Summit 2021 (remote): "Federated speech technologies made easy: Flower and SpeechBrain", March 11th (2021).
Centre de Recherche en Automatique de Nancy (FR): "Should we use quaternion neural networks? Recent advances and limitations.", March 29th (2021).
Samsung AI Cambridge (UK): "SpeechBrain", February 27th (2021).
Samsung AI Cambridge (UK): "Quaternion neural networks", July 10th (2019).
University of Oxford (UK): "Quaternion neural networks", July 9th (2019), Computer Science Department.

Program Committee Member

• IEEE Journal of Selected Topics in Signal Processing. 2022
• IEEE Signal Processing Letters. 2021
• IEEE Transactions on Neural Networks and Learning Systems. 2019, 2020
• IEEE International Journal of Wavelets, Multiresolution and Information Processing. 2020
• ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2020
• Elsevier, Computers & Graphics, 2021
• ACM Multimedia. 2021
• Springer, Neural Processing Letters. 2020
• IEEE Transactions on Image Processing. 2020
• NeurIPS. 20(18-21), top 10% best reviewer 2020.
• ICLR. 20(19-20-21), top 10% best reviewer 2020.
• INTERSPEECH. 20(19-20-21-22)
• ICASSP. 2020

Expert for Research Projects Evaluation

• Agence Nationale de la Recherche. 2021
• Idex Lyon St-Étienne. 2021

Press Coverage

Le Devoir (Quebec News) — "Google, dis-moi si tu comprends mon accent québécois", April 10th (2021).