NLP Breakfast 12: An Improved and Affordable feature-based RNN model for Language Modeling

Over the last years, language modeling pre-trained methods have yielded effective improvement in Natural Language Processing tasks.

Most of the improvements have come with ELMo followed by, large research on the Transformer approaches like BERT or XLNET despite the fact that the transformers are expensive to train. Unfortunately, we haven’t seen much improvement in the affordable ELMo while limitations in this architecture have been addressed in many papers.

Gregory Senay, CSO at xBrain, will present a new architecture improving ELMo and reaching higher performances in GLUE tasks for a feature-based model, and this, while keeping a limited computation cost.

This new architecture uses some improvements made with Transformers but also some tricks for speed-up and improving the quality of the training.

Whether you want to hear more about this new architecture or just want to participate in a nice discussion, feel free to join us!

When?

Thursday, November 14th, 2019 at 9:30am PDT

Where?