Titouan Parcollet

Funded Projects

SpeechBrain: simplify the access to speech technologies (2021 - 2022)
Principal Investigator

The Speechbrain project (see bellow for mor info) has been granted 400K hours (i.e. 234 000€) of GPU time in the French datacenter Jean Zay. This project is a joint collaboration between the Laboratoire Informatique d'Avignon (LIA) , Le Laboratoire de Traitement et Communication de l'Information (LTCI), the Laboratoire des Sciences du Numérique de Nantes (LS2N) and the Laboratoire Interdisciplinaire des Sciences du Numérique (LISN) that aims to support the effort devoted to SpeechBrain towards a democratization of the research and developpement of speech technologies. This project will gather top-tier researchers to build and release state-of-the-art and ground-breaking systems for speech translation, self-supervised learning of speech representations, speaker verification and identification, voice privacy, spoken language understanding, speech synthesis, speech for e-health and speech enhancement.

The consortium is composed with: Assoc. Prof. Titouan PARCOLLET (PI, LIA), Prof. Yannick ESTÈVE (LIA), Prof. Corinne FREDOUILLE (LIA), Prof. Jean-François BONASTRE (LIA), Prof. Richard DUFOUR (LS2N), Prof. Slim ESSID (LTCI), Assoc. Prof. Sahar GANNAY (LISN).

LeBenchmark (2020 - 2022)
Co-Principal Investigator for SSL models

LeBenchmark project has been granted 200K hours (i.e. 117 000€) of GPU time in the French datacenter Jean Zay. This project is a joint collaboration between the Laboratoire d'Informatique de Grenoble (LIG) and the Laboratoire Informatique d'Avignon (LIA) that aimed to collect large quantities of raw speech in French (i.e. several thousand hours) with different styles (read speech, prepared speech, spontaneous speech), from various speakers and use them to learn self-supervised models to be shared with the research community. Furthermore, we also established a new benchmark data set for several speech processing tasks. I am in charge of finding the appropriate self-supervised methods (e.g wav2vec) and to deploy it on our new dataset. We released various [pre-trained models](https://huggingface.co/LeBenchmark) with various training sets acounting for up to 14,000 hours of speech in French. Everything is nicely integrated to SpeechBrain for peoples interested in re-using our models for downstream tasks!

The consortium is composed with: Prof. Laurent BESACIER (PI, LIG), Prof. François PORTET (LIG), Assoc. Prof. Solange ROSSATO (LIG), Assoc. Prof. Benjamin LECOUTEUX (LIG), Assoc. Prof. Didier SCHWAB (LIG), Assoc. Prof. Fabien RINGEVAL, Dr., CR, Marco DINARELLI (CNRS, LIG), Prof. Alexandre ALLAUZEN (LAMSADE), Assoc. Prof. Titouan PARCOLLET (LIA), Prof. Yannick ESTÈVE (LIA).

Models are available on HuggingFace .
Publications:

• Task Agnostic and Task Specific Self-Supervised Learning from Speech with LeBenchmark
• Solène Evain · Ha Nguyen · Hang Le · Marcely Zanon Boito · Salima Mdhaffar · Salima Mdhaffar · Sina Alisamir · Ziyi Tong · Natalia Tomashenko · Marco Dinarelli · Titouan Parcollet · Alexandre Allauzen · Yannick Estève · Benjamin Lecouteux · François Portet · Solange Rossato · Fabien Ringeval · Didier Schwab · Laurent Besacier
• NeurIPS 2021.

• LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speech
• Solène Evain · Ha Nguyen · Hang Le · Marcely Zanon Boito · Sina Alisamir · Ziyi Tong · salima mdhaffar · Natalia Tomashenko · Marco Dinarelli · Titouan parcollet · Alexandre Allauzen · Yannick Estève · Benjamin Lecouteux · François Portet · Solange Rossato · Fabien Ringeval · Didier Schwab · Laurent Besacier
• Interspeech 2021.

SpeechBrain (2019 - )
Creator & Co-Principal Investigator

The SpeechBrain project aims to develop an open-source and all-in-one toolkit based on PyTorch. The goal is to develop a single, flexible, and user-friendly toolkit that can be used to easily develop state-of-the-art speech systems for speech recognition (both end-to-end and HMM-DNN), speaker recognition, speech separation, multi-microphone signal processing (e.g, beamforming), self-supervised learning, and many others. The project is funded thanks to generous donations from an ever growing number of sponsors: the Laboratoire Informatique d'Avignon (LIA), the Montréal Institute for Learning Algorithms (MILA), Samsung, HuggingFace, NVIDIA, Dolby, ViaDialog, OVH and NAVER Labs. SpeechBrain also benefits from the collaboration and expertise of around 20 different partner institutions (both academics and industrials) ranging from the University of Cambridge to the PyTorch Team. SpeechBrain already beats all the other toolkits on the considered datasets and with a much easier interface to play with. SpeechBrain reached 4.6K stars on GitHub in a year demonstrating a clear interest from the community for our toolkit. I am the co-creator and co-leader of SpeechBrain with Dr. Ravanelli Mirco, currently a post-doctoral researcher at MILA.

Have a look at: SpeechBrain! Or to our open access paper describing the toolkit!

Advised & Mentored Ph.D. Students and Interns

Jarod DURET — Ph.D. Student — Started in October 2021 and is co-advised with Prof. Yannick ESTÈVE from LIA
Title: Expressive Speech to Speech Translation
Ideas: Current speech to speech translation systems rely on cascade systems that combine both a speech to text and a text to speech block. In practice, existing end-to-end speech to speech translation systems do not work well and often completely bypass the concept of expressivity. With this thesis, we aim at inventing a simple end-to-end model enabling an expressivity transfer alongside with the translation. This appears as being particularly challenging as the expressivity may be defined differently from one language to an other. Multiple directions will be investigated, starting with the design of better speech representations capturing the expressivity to condition the generation of the speech signal.

Salah ZAIEM — Ph.D. Student — Started in October 2020 and is co-advised with Prof. Slim ESSID from Telecom Paris Sud.
Title: Informed Self-Supervised Speech Representations Learning.
Ideas: Self-supervised learning methods for speech are mostly empirically driven. In particular, there exist very few theoretical evidences on why a method performs better than an other one. With this thesis, we aim at providing theoreticaly grounded tools to design SSL models in an informed manner. For instance, we developed a solution to design a PASE-like architecture without the need for pre-text task search with empirical validation, potentially saving weeks of training (and compute / carbon emissions).

Xinchi QIU — Mentored Ph.D. Student — Started in Sept. 2019, advised by Prof. Nicholas Lane from the University of Cambridge.
Title: Efficient Federated Learning
Ideas: With Xinchi, we wondered if the concept of federated learning that will soon cover a large part of the deep learning use cases could be more efficient than centralised training. With that in mind, we started by investigating its energy footprint before jumping into practical ways of increasing its efficiency including high-dimensional neural networks, parameters sharing or sparsity.

Yan Gao — Mentored Ph.D. Student — Started in Sept. 2019, advised by Prof. Nicholas Lane from the University of Cambridge.
Title: Federated Speech Recognition
Ideas: Yan explores different ways of enabling on-device speech recognition training with edge data. In practice, state-of-the-art speech recognizers are highly resource intensive and we build the foundations necessary to properly assess the difficulty arising when such models are deployed on device. For instance, Yan investigates different solutions to aggregate numerous acoustic models coming from large pools of devices under a federated learning setup. His latest work tries to highlight the unstainability of current large scale self-supervised speech models under constrained resources.

Adel Moumen — Undergrad. Intern — June 2021 to August 2021 and June 2022 to August 2022.
Title: On the limitations of LiGRU networks.
Ideas: Adel investigates different ways of making LiGRU a mandatory alternative to LSTM/GRU for speech processing. There are theoretical and empirical evidences that LiGRU simply are better than LSTM and GRU. Unfortunately, they suffer from a poor GPU implementation and an instability in the recurrent connection. These problems will be tackled during the internship.

SpeechBrain Full-Time Research Engineers — Started on January 2022.
Format: this is a list of all the researchers working full-time or part-time on SpeechBrain that I recruited and supervised.
Topics: all the topics and concepts developed within SpeechBrain.

Dr. Andreas Nautsch
Adel Moumen

SpeechBrain Interns — Started on January 2020.
Format: this is a list of all the researchers that I mentored during their internship to work on SpeechBrain. Internships took place either in Avignon (LIA) or Montréal (Mila) and were co-advised with Dr. Ravanelli
Topics: all the topics and concepts developed within SpeechBrain.

Mohamed Anwar, Naver LABS Europe (FR).
Aku Rouhe, Aalto University (FI).
Peter Plantinga, now at JP Morgan.
Loren Lugosch, Mila (CA).
Nauman Dawalatabad, now at MIT (USA).
Ju-Chieh Chou, National Taiwan University (TW).
Sung-Lin Yeh, now at University of Edinburg (UK).
Hwidong Na, Samsung SAIL (CA).
Abdel Heba, Linaroga / University of Toulouse (FR).
Samuele Cornell, now at Amazon.
Jianyuan Zhong, University of Rochester (USA).
Cem Subakan, University of Montréal (CA).
Szu-Wei Fu, Academia Sinica (TW).

Community Contributions

General Co-Chair and Area Chair

Area Chair, NeurIPS, 2022.
Chair, RECITAL (TALN session), June 28th (2022), Avignon (France).

Workshop & Session Co-Organizer

ICML self-Supervision in Audio and Speech (held virtually), July 17th (2020), Vienna (Austria).

Tutorials

Interspeech 2022: "State of the PyTorch Ecosystem for Speech Technologies", August 2022.
IEEE ASRU 2021: "SpeechBrain: Unifying Speech Technologies and Deep Learning With an Open Source Toolkit", December 2021.
Interspeech 2021: "SpeechBrain: Unifying Speech Technologies and Deep Learning With an Open Source Toolkit", August 2021.
University of Sheffield: "SpeechBrain: Unifying Speech Technologies and Deep Learning With an Open Source Toolkit", June 2021.

Invited Talks

Samsung AI Cambridge (UK): "Federated Speech Technologies", July (2022).
ADASP Group, Télécom Paris: "SpeechBrain: A General-Purpose Speech Toolkit", March 6th (2022).
Idiap Research Institute: "SpeechBrain: A General-Purpose Speech Toolkit", April 27th (2022).
Machine Intelligence Laboratory, University of Cambridge: "SpeechBrain: A General-Purpose Speech Toolkit", March 14th (2022).
Naver Labs Europe: "SpeechBrain: A General-Purpose Speech Toolkit", November 30th (2021).
FestivalIA Avignon: "SpeechBrain : un outil polyvalent pour le traitement automatique de la parole", November 17th (2021).
Microsoft Research Summit Workshop on Federated Learning and Confidential Computing: "Federated Speech Technologies", October 21th (2021).
Machine Learning Summer Schools — Taipei: "Task Agnostic and Task Specific Self-Supervised Learning from Speech with LeBenchmark", August 20th (2021).
The Machine Learning / Data Science Meetup of Rome: "SpeechBrain: A General-Purpose Speech Toolkit", July 7th (2021).
The 2nd Annual Federated & Distributed / Decentralized Machine Learning Conference (remote): "Can Federated Learning Save the Planet", June 16th (2021).
Flower Summit 2021 (remote): "Federated speech technologies made easy: Flower and SpeechBrain", March 11th (2021).
Centre de Recherche en Automatique de Nancy (FR): "Should we use quaternion neural networks? Recent advances and limitations.", March 29th (2021).
Samsung AI Cambridge (UK): "SpeechBrain", February 27th (2021).
Samsung AI Cambridge (UK): "Quaternion neural networks", July 10th (2019).
University of Oxford (UK): "Quaternion neural networks", July 9th (2019), Computer Science Department.

Program Committee Member

• IEEE Journal of Selected Topics in Signal Processing. 2022
• IEEE Signal Processing Letters. 2021
• IEEE Transactions on Neural Networks and Learning Systems. 2019, 2020
• IEEE International Journal of Wavelets, Multiresolution and Information Processing. 2020
• ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2020
• Elsevier, Computers & Graphics, 2021
• ACM Multimedia. 2021
• Springer, Neural Processing Letters. 2020
• IEEE Transactions on Image Processing. 2020
• NeurIPS. 20(18-21), top 10% best reviewer 2020.
• ICLR. 20(19-20-21), top 10% best reviewer 2020.
• INTERSPEECH. 20(19-20-21-22)
• ICASSP. 2020

Expert for Research Projects Evaluation

• Agence Nationale de la Recherche. 2021
• Idex Lyon St-Étienne. 2021

Press Coverage

Le Devoir (Quebec News) — "Google, dis-moi si tu comprends mon accent québécois", April 10th (2021).