Google Makes NLP Model ALBERT Open Source

  • ALBERT has been released as an open source implementation on top of TensorFlow
  • It reduces model sizes in two ways- by sharing parameters across the hidden layers of the network and by factorising the embedding layer

According to a report by i-programmer, Google has made ALBERT (A Lite BERT) available in an open source version. It is a deep-learning natural language processing (NLP) model. the report said that developers claimed that ALBERT used fewer parameters than BERT without compromising accuracy.

Bidirectional Encoder Representations from Transformers (BERT) is a self-supervised method that was released by Google in 2018. The report said that the method delivered results on a range of NLP tasks by depending on un-annotated text drawn from the web.

Improved performance on 12 NLP tasks

According to the report, ALBERT is an upgrade to BERT. It offers improved performance on 12 NLP tasks which includes Stanford Question Answering Dataset (SQuAD v2.0) and the SAT-style reading comprehension RACE benchmark. ALBERT has been released as an open source implementation on top of TensorFlow. It will include various ready-to-use pre-trained language representation models.

The report said that, according to a paper given by its developers to the International Conference on Learning Representations, ALBERT reduces model sizes in two ways. These are by sharing parameters across the hidden layers of the network and by factorising the embedding layer.

Avatar

Previous articleProton Technologies Makes ProtonVPN Open Source
Next articleSecuring Data in IoT Constrained Node Networks
Avatar

source : http://opensourceforu.com

Write a Comment

view all comments