ChatGPT across all languages!
How does it benchmark in different languages?
Paper: Phoenix: Democratizing ChatGPT across Languages
Github: LLM Zoo: democratizing ChatGPT
Researchers at The Chinese University of Hong Kong have attempted to democratize ChatGPT across all languages.
Mainly, They have developed new language model which benchmarks better across all latin and non Cyrillic languages. They refer to these are Pheonix and Chimera since these are Eastern and Western legendary creatures.
Here are some important part of their evaluation:
The main philosophy is to combine instruction data and conversation data to focus models to follow instructions in a chat
They collect 100 questions spanning ten categories and feed them to each model.
For automatic evaluation, they use GPT-4 as a reviewer to rate each answer in helpfulness, relevance, accuracy, and level of detail.
For human evaluation, they ask the evaluation participants to judge the overall performance of the generation results of each model
As per results, here are some limitations of the current models:
Lack of common sense, emotional understanding and has biases
Limited domain knowledge domain unless trained
Promote contextual misunderstanding and fake information without verifiability
So Essentially,
We have language models in Latin and non Latin language with ChatGPT but they also have the limitations of other language models.