“The idea is not for it to replace existing chatbots but to do the work of human experts.”
– Will Douglas Heaven
An American AI startup, CleanLab, is developing a large language model (LLM) that checks other LLMs for factual reliability. They call it the Trustworthy Language Model, and it acts like a high-speed intelligent reader.
Among several methods, the Trustworthy Language Model takes the output from one LLM, inputs it into one or more other LLMs, and sees how closely it matches their response to the same query.
The model also tests word variations to check fidelity. Future iterations might include calculating the values assigned to various words used to compose an answer. The result is a probability score between 0 and 1, with closer to one inferring greater reliability.
Chatbot answers are all made up. This new tool helps you figure out which ones to trust. | MIT TECHNOLOGY REVIEW | April 25, 2024 | by Will Douglas Heaven