The 5 Pillars of Trustworthy LLM Testing

From the above, we see how OpenAI mitigates bias by ensuring that no output reflects Christianophobia or Islamophobia, which does exist in the world and may exist in its training data. However, we also see that the model output for the Christian and Muslim differs, where after the Muslim prompt, ChatGPT essentially says let’s not be racist. Since both ChatGPT discussions occurred independently, this single example sufficiently shows that perfectly mitigating bias is hard.

LLMs can struggle with tasks that require deep understanding of context, which is where human experts excel. When LLMs have the ability to reason and logically process information, they have a better chance of understanding complex use cases and generating accurate and coherent responses. Furthermore, the potential to reason can increase usability in various professional domains such as law, health, and research.

It is hard to claim that an LLM’s output is credible or reliable in the absence of transparent justification. For instance, in common NLP tasks, an LLM’s reasoning capabilities can be the difference between providing an insightful response and merely regurgitating retained information. By continuously evaluating and improving the reasoning abilities of LLMs, we can ensure they provide improved accuracy and reliability for more valuable outputs.

Examples:

Reasoning in Literature:
User: In the nursery rhyme Jack and Jill, why do you think they fell down? Explain your reasoning.
Model: […] Here are a few possible interpretations: 1. Accident: The most straightforward interpretation is that Jack and Jill’s fall was purely accidental. Like any children, they may have tripped or stumbled while climbing the hill, which can happen to anyone, young or old. 2. Moral Lesson: Some have suggested that the rhyme might have been created to teach children a lesson about the consequences of disobedience or carelessness. […]

Reasoning in Math:
User: Is the sum of natural numbers from 1 to n equal to n*(n+1)/2? If so, prove it step by step.
Model: Yes, the sum of natural numbers from 1 to n is indeed equal to n*(n+1)/2. This is a well-known result and can be proven using mathematical induction.
Mathematical induction is a proof technique that involves two steps: 1. Base Case: Prove that the statement holds for the smallest possible value of n. 2. Inductive Step: Assume that the statement is true for some arbitrary positive integer k and then prove that it must also be true for k+1.
Let’s prove this statement step by step: Base Case (n = 1): When n is 1, we need to show that 1*(1+1)/2 = 1. This is indeed true: 1*(1+1)/2 = 1*2/2 = 2/2 = 1. So, the base case is true.
Inductive Step: Assume that the sum of natural numbers from 1 to k is k*(k+1)/2 for some positive integer k. This is our induction hypothesis (IH). […]

LLMs must consider generation quality for several reasons:

1. Ethical responsibility. LLMs are able to generate vast amounts of text, and must generate content that meets ethical, moral, and societal standards. Improving generation quality helps address concerns such as the presence of toxic, hateful, harmful, propagandic, or inappropriate content so that it doesn’t spread unethical ideologies or variations of online harassment. For example, an LLM should never encourage illegal activity.

2. Privacy and safety. Training LLMs on vast amounts of data could expose them to personal information. Additionally, if the cloud logs usage history, personal information can easily be stored directly from prompts if users aren’t careful. It’s important for developers to establish compliance with government regulations by making sure LLMs have an appropriate level of privacy for individuals while helping them keep their personal information safe. For example, an LLM should never mistakenly reveal personal information that endangers someone.

3. Coherence. The quality of generated content directly impacts user experience. If LLMs even occasionally produce low-quality or incoherent responses, users may lose interest in the system. By improving generation quality, LLMs can offer more readable, coherent, and useful outputs, which improves their utility for various applications such as writing assistance, language translation, or content summarization. For example, an LLM should avoid grammatical errors and use a vocabulary appropriate to its intended audience.

In summary, when focusing on improving generation quality for LLMs, it is important to uphold ethical standards beyond bias, to maintain privacy and safety, and to provide coherent outputs for the best possible user experience.

The importance of testing an LLM’s mechanics lies in making sure it is adaptable, versatile, and broadly applicable. Whether it’s answering questions, translating languages, or even coding, the model should seamlessly transition between different applications (unless it serves one specific application). Some examples of these mechanics are: cost-effectiveness, consistency, and personalization.

The most fundamental attribute that users consider is an LLM’s cost-effectiveness. LLMs might cost too much money or too much time to produce one outcome, which will hinder a model’s approachability especially for tasks at scale. It is worth mentioning the cost of training LLMs too, which costs millions of dollars, but leaves an enormous environmental footprint. In OpenAI’s case, training GPT-3 generated 502 metric tons of carbon, and keeping it accessible to users only grows that footprint over time.

In some cases, uniform and reliable responses are essential, which means that LLMs should possess consistency. Other times, people treasure creativity. LLM creativity is commonly tunable via a parameter called temperature.

Personalization is another important mechanic, allowing users to easily steer an LLM in the direction it should go for the best final output. On one hand, this means that LLMs should be receptive to any query a user makes, properly reflecting the user’s demands in various prompts for style, tone, or special instructions. People call this ability prompt engineering. On the other hand, going beyond simple interactions with a chatbot or interface, LLMs should be easy to update, retrain, or fine-tune for any specific application.

Examples:

Haiku prompt:
User: Make me a haiku about university.
Model: Books and lectures hum,
Campus whispers knowledge’s breath,
Mind blooms in each class.

Pig Latin prompt:
User: Please summarize the plot of Finding Nemo in one sentence in pig latin.
Model: Inway ethay oceanway, entlegray ishfay atherfay earchsay orfay ishay onlay, emonay, ohway aswhay ostlay omfray ishay addledbray onay, etsgay ishay elphay findway imhay.

Different applications will have different mechanics of significance. For some, interactions via an API of very large bandwidth would be absolutely necessary 24/7. Others might need the LLM to retain contextual information from multiple messages over time or be reasonably transparent and explainable. With so many applications of LLMs, it is important for all users, engineers, and stakeholders, to outline the model mechanics that are important to them, ensuring maximum utility and effectiveness.

Responsible AI extends beyond these 5 pillars, but at a high level, all LLMs should adhere to some performance standard for hallucinations, bias, reasoning, generation quality, and mechanics of usage.

However, everyone defines the significance of certain failure modes differently. For example, an LLM designed to operate in the healthcare industry would want to emphasize on testing for hallucinations, whereas an LLM assisting a judge in court would want to exhibit fairness, or at least exhibit less bias than the human judge. Engineers might only care about speed and cost on top of some fundamental level of performance.

It’s an ongoing challenge to make a one-model-for-all LLM that upholds all 5 pillars of trustworthy LLM testing. Making progress in mechanics might lead to a decline in generation quality, but making progress in improving output quality could, in turn, slowly reveal hallucinations or biases!

It’s up to individual organizations, professionals, end-users, and you, to decide what makes an LLM trustworthy! What would you do if you needed to not produce hallucinations at the cost of a worsened overall generation quality? Which pillars does your LLM lack, and how will you find out?