LATAM

INDUSTRY VIEW

RODRIGO PEREIRA , CEO OF A3DATA

Rodrigo Pereira , CEO of A3Data , a consulting firm specializing in data and AI , questions whether LLMs fuel a vicious circle of misinformation .

With quick responses , in natural language , and seemingly reliable , these models are becoming the first choice for many ordinary citizens . But are we aware of the risks embedded in this new resource ?

According to a recent article written by researchers from Stanford University , University of Southern California , Carnegie Mellon University , and the Allen Institute for AI , LLMs , such as GPT and LLaMA-2 , are often reluctant to express uncertainties , even when their responses are incorrect : about 47 % of the answers provided with high confidence by the models were wrong .

Moreover , the research addresses the issue of biases in both the models and human annotation . During the process of Reinforcement Learning with Human

About 47 % of the answers provided with high confidence by the models were wrong .

Feedback ( RLHF ), the language models are trained to optimise their responses based on human feedback . However , this process can amplify certain biases present in the training data or in the feedback itself .

Among the biases that should be considered are gender and race biases . If feedback is provided with these stereotypes or uncertainties are avoided in contexts involving minorities , the models end up perpetuating and amplifying these human perspectives .

Another concerning bias is the annotators ’ preference for responses that sound more assertive , even when there are uncertainties about the information . This leads the models to avoid expressing doubt to the user , creating the false illusion of solid knowledge , when in fact they may be wrong . For example , a categorical statement about a country ’ s capital might be preferred by annotators , even if the model was uncertain , resulting in a potentially incorrect response but presented with confidence .

These biases are troubling because they shape how the responses are generated and perceived by users . When combined with the excessive trust users tend to place in the answers of LLMs , these biases can lead to the spread of distorted information and the reinforcement of social prejudices .

We are , therefore , facing a possible vicious cycle . As more people turn to LLMs to search for information , the overconfidence in these models can amplify the spread of misinformation .

In this sense , the alignment process of models with human feedback ( RLHF ) may be exacerbating this issue , reinforcing assertive responses and underestimating the importance of expressing uncertainties . This not only perpetuates incorrect information but can also reinforce social prejudices and biases , creating a self-reinforcing cycle that intensifies over time .

To prevent this vicious cycle from taking hold , it is important to take action on several fronts , such as transparency and clarification in the tools . LLMs should be designed to express uncertainties clearly and contextually , allowing users to better understand the reliability of the information provided . Additionally , including a more diverse range of feedback during model training can help mitigate the biases introduced by a limited subset of users or annotators . •

INTELLIGENT TECH CHANNELS LATAM

INTELLIGENT TECH CHANNELS

17