Google’s LaMDA: Advancements in Conversational AI and Ethical AI Development

The concerns surrounding the development of Artificial Intelligence (AI) are not new. However, recently, a conversation between a researcher and a chatbot supported by Google’s LaMDA AI sparked numerous discussions on this topic, raising the question of what if they became sentient like humans?

Developed as a breakthrough in conversational technology

At Google I/O 2021, Google CEO Sundar Pichai unveiled the company’s “latest breakthrough in natural language processing in AI,” called LaMDA (which stands for “Language Model for Dialogue Applications”). Essentially, it is a machine learning-powered chatbot designed to converse about any topic, much like IBM’s Watson providing users with the ability to interact on a wide range of subjects. However, LaMDA AI has the capacity to understand and generate more nuanced conversations. At that time, Google stated that this AI was only trained on text, meaning it couldn’t generate or respond to content related to images, audio, or video.

One could say that LaMDA is like a curious novel with impressive question parsing and the ability to generate responses that go beyond the realm of a typical chatbot. Google demonstrated LaMDA’s ability to discuss topics such as Pluto, the moon of Saturn, and even give fashion advice. CEO Sundar Pichai emphasized in the presentation that this is not a question-answering chatbot, but one that can add a touch of style, express opinions on the queried topic, and even model how we truly use language better.

Another advantage of LaMDA is that it doesn’t specialize in recreating conversations or specific topics. Instead, it can provide answers across various domains, whether it’s history or weather, although there are still limitations in the early stages of the project.

Google stated that this is still “early-stage research” and actively being developed, but the company has been using it internally to “explore new interactions” and improve the language processing capabilities of other voice-enabled tools like Google Search, Google Assistant, and Workspace. The work is being done to ensure that it meets Google’s high standards for fairness, accuracy, safety, and privacy, “consistent with [Google’s] AI principles.” It’s important to test and control because LaMDA’s responses to questions could become a source of data for some individuals, depending on their location and usage.

At Google I/O 2022, Google revealed “LaMDA 2,” an enhanced version of the conversational AI. This time, the company allowed “thousands of Googlers” to test it—partly to reduce instances of problematic or unpleasant answers. Overall, LaMDA 2 has similar features and functionalities to the original version, operating as a sophisticated and versatile chatbot. However, Google’s demonstration at that time focused more on technical aspects, such as maintaining a conversation on a topic, creating lists associated with a subject, or imagining being in a specific location. LaMDA’s capabilities certainly aren’t limited to these workflows; they are merely avenues that Google wants to explore to test and fine-tune how LaMDA operates.

LaMDA is not even the most complex language processing model within Google. PaLM (also unveiled at I/O) is a larger and more complex system that can handle tasks that LaMDA cannot, such as math and code generation, with an advanced processing system for higher accuracy.

While LaMDA AI’s development opens up new possibilities for conversational AI, it’s important to continue monitoring and refining its capabilities to ensure ethical and responsible use. The potential impact of AI with human-like qualities, such as sentience, raises profound questions and necessitates careful consideration of the benefits and implications it may bring to society.

Google’s AI team has become the subject of discussion regarding the development of artificial intelligence, and the company has also raised ethical questions in the development of this technology. In 2018, Google established a set of ethical guidelines and principles for AI, outlining how AI is created and what it can be used for. In summary, Google wants to ensure that its work in AI development is for the benefit of society, is safe and respects privacy, while adhering to the most effective methods for things like data validation and model training. Lastly, Google stated that it will not pursue AI applications that have the potential to be used to harm others (such as in weapons), for surveillance, or violate the law.

The LaMDA AI Development Process

LaMDA is built on a neural architecture called Transformer, which Google invented in 2017. This tool quickly became more complex, but fundamentally, the Transformer provides advantages in both training efficiency (model building time) and accuracy compared to recurrent and convolutional neural models (typical machine learning systems) when it comes to language processing.

Instead of analyzing the input text step-by-step, the Transformer simultaneously analyzes entire sentences and can model the relationships between them to better understand contextual meaning. And because it performs context-based analysis all at once, it requires fewer steps to perform its task – the fewer steps in a machine learning model, the more likely it is to perform its job well.

In 2020, Google introduced the first truly conversational AI chatbot built on this technology, named Meena. Meena was trained on 341 GB of text “filtered from public social conversations,” meaning it learned the nuances of conversation from some of the most challenging yet authentic examples possible. While none of us had easy access to it, an analysis scorecard provided in the research paper (available here) shows how Meena could sometimes resemble a human, exhibiting everything from “banal” curiosity to discussing specific movies. Of course, it still had certain limitations.

While Meena may not be as sophisticated as LaMDA, it was a necessary step to demonstrate that an open-domain chatbot can better understand the nuances of how we use language and provide meaningful, empathetic responses – or at least responses that are “coherent and specific like a human.”

Although Meena had 2.6 billion parameters at the time, it still pales in comparison to LaMDA AI (with 137 billion parameters) or PaLM with 540 billion parameters.

How LaMDA Works

In January 2022, a few months before the I/O 2022 event, Google provided more detailed information about how LaMDA works and its language model, describing the progress achieved so far on the AI Blog.

LaMDA was created to address a range of evaluative metrics that previous chatbots struggled with. Things like internal consistency of answers and the level of surprise corresponding to the SSI (Sensitivity, Specificity, and Interestingness) score. Instead of just meeting expectations, could it make a joke or provide genuinely detailed information? And how realistic and informative are its answers?

Using these as a measure of its nature, aiming to create a chatbot that feels more human-like, Google trained the LaMDA models on a massive dataset, attempting to help it predict the next components of a sentence. At its core, LaMDA is an ad-lib generator that fills in details that humans would appreciate. The model is then fine-tuned for different applications to expand its “development” capabilities and further trained using conversational datasets from real back-and-forth conversations between two humans. In essence, it goes from filling in blanks to filling in sentences, trained to mimic real conversations.

Like many machine learning systems, rather than generating a single response, LaMDA generates multiple options and uses an.