Google DeepMind Reveals New Information on Efforts to Outdo ChatGPT

On Monday, Google DeepMind revealed new information on its work to develop a chatbot that outperforms ChatGPT, the current state-of-the-art open-source language model released by Microsoft in 2019. The ChatGPT system, which “generates plausible responses in conversational settings,” has been widely used for natural language processing (NLP) tasks ranging from question-answering to dialogue generation.

Google DeepMind’s team admitted that, to expand the current state-of-the-art, they have adjusted the architectural design of the system in order to improve its performance. According to the research paper, they have incorporated an encoder-decoder system as well as additional layers of self-attention which “transform a given utterance into a higherdimensional representation that can better capture hierarchical dependencies of the underlying sentences.” In addition, they have incorporated a “shallow transformer” which they believe can improve both the token representation and the token coherence.

The research team has also shared some interesting experimental findings. For example, they have found that adding additional self-attention layers, as well as the shallow transformer, has led to improvements of up to 8.4% in terms of perplexity. They also report that the incorporation of their shallow transformer has – in comparison with the ChatGPT model – improved automated evaluation metrics for engaging conversations across different domains.

Google DeepMind’s research team recognized that their work does not yet necessarily constitute a “fully functional dialogue system” and there is still much more work to be done before their chatbot is ready for general use. The new information they’ve revealed, however, provides a strong basis on which to build the future of NLP algorithms. As Andrew Gardner from the Google Research team stated in the paper: “We firmly believe that dialogue systems should constantly be striving to reach human parity.”