Thanks to theidioms.com

Google Brain Trains An AI Model Almost 6x Larger Than OpenAI’s GPT-3 Model

Switch Transformers: Google Brain trains an AI model almost 6x larger than OpenAI's GPT-3 model
Blog

Google Brain Trains An AI Model Almost 6x Larger Than OpenAI’s GPT-3 Model

Google Brain Trains An AI Model Almost 6x Larger Than OpenAI's GPT-3 ModelGoogle Brain Trains An AI Model Almost 6x Larger Than OpenAI's GPT-3 Model

This week, Google Brain researchers published a paper called ‘Switch Transformers: Scaling To Trillion Parameter Models With Simple And Efficient Sparsity’, which claims that they have trained a language model containing more than a trillion parameters. OpenAI’s GPT-3 only has about 175 billion parameters in comparison.

Google Brain trains an AI model almost 6x larger than OpenAI's GPT-3 model

The model uses a sparsely activated technique called Switch Transformers using 32 TPU cores. The model was trained using the Colossal Clean Crawled Corpus which is an 800+ GB dataset of text scraped from Wikipedia, Reddit, and other sources. Also, this model was able to achieve a 4x-speedup over the T5-XXL model (previously largest model by Google).

You can read the paper in detail from here: Read paper.


Are you interested in working with data? Get a free trial month of LinkedIn Learning and learn Data Science and Machine Learning courses using Python, R, and SQL from top instructors.

Enroll in any one of the courses from the following fields today on LinkedIn Learning and start your free month of learning:

Disclaimer: When you subscribe to LinkedIn Learning, we may earn a small commission as an affiliate. We love the platform so much so that we have been using it ourselves at The Click Reader.

Google Brain Trains An AI Model Almost 6x Larger Than OpenAI's GPT-3 ModelGoogle Brain Trains An AI Model Almost 6x Larger Than OpenAI's GPT-3 Model

Leave your thought here

Your email address will not be published. Required fields are marked *