Japanese-Chinese language Translation with GenAI: What Works and What Doesn’t

Alex (Qian) Wan: Alex (Qian) is a designer specializing in AI for B2B merchandise. She is at the moment working at Microsoft, specializing in machine studying and Copilot for knowledge evaluation. Beforehand, she was the Gen AI design lead at VMware.
Eli Ruoyong Hong : Eli is a design lead at Robert Bosch specializing in AI and immersive expertise, growing methods that bridge technical innovation with human social dynamics to create extra culturally conscious and socially responsive applied sciences.

Think about you’re scrolling by social media and are available throughout a put up a couple of home makeover written in one other language. Right here’s a direct, word-for-word translation:

Lastly, cleaned up this home fully and adjusted the design plan. Subsequent, simply ready for the development group to come back in. Wanting ahead to the ultimate outcome! Hope the whole lot goes easily!

For those who had been the English translator, how would you translate this? Gen AI responded with:

I lastly completed cleansing up this home and have adjusted the design plan. Now, I’m simply ready for the development group to come back in. I’m actually trying ahead to the ultimate outcome and hope the whole lot goes easily!

The interpretation appears to be clear and grammarly excellent. Nonetheless, what if I advised you this can be a social put up from an individual who’s notoriously recognized for exaggerating their wealth? They don’t personal the home—they simply unnoticed the topic to make it appear to be they do. Gen AI added “I” mistakenly with out admitting the vagueness. A greater translation can be:

The home has lastly been cleaned up, and the design plan has been adjusted. Now, simply ready for the development group to come back in. Wanting ahead to seeing the ultimate outcome—hope the whole lot goes easily!

The languages the place the “unspoken” context performs an necessary function in literature and each day life are known as “high-context language“. 

Translating high-context languages akin to Chinese language and Japanese is uniquely difficult for a lot of causes. As an illustration, by omitting pronouns, and utilizing metaphors which can be extremely related to historical past or tradition, translators are extra depending on context and are anticipated to have a deep information of tradition, historical past, and even variations amongst areas to make sure accuracy in translation.

This has been a long-time subject in conventional translation instruments akin to Google Translate and DeepL, however fortuitously, we’re within the period of Gen AI, the interpretation has considerably improved due to context-aware capacity, and Gen AI is ready to generate far more human-like content material. Motivated by technological development, we determined to develop a Gen-AI powered translation browser extension for each day studying function.

Our extension makes use of Gen AI API. One of many challenges we encountered was selecting the AI mannequin. Given the varied choices in the marketplace, this has been a multi-month battle. We realized that there is perhaps many individuals like us – not techy, with a decrease finances, however curious about utilizing Gen AI to bridge the language hole, so we examined 10 fashions with the hope of bringing insights to the viewers.

This text paperwork our journey of testing completely different fashions for Chinese language Japanese translation, evaluating the outcomes primarily based on particular standards, and offering sensible ideas and methods to resolve points to extend translation high quality.

Anybody who’s working or curious about utilizing multi-language generative AI for subjects like us: perhaps you’re a group member working for an AI-model tech firm and on the lookout for potential enhancements. This text will assist you perceive the important thing elements that uniquely and considerably impression the accuracy of Chinese language and Japanese translations.

It could additionally encourage you in the event you’re growing a Gen Ai Agent devoted to language translation. For those who occur to be somebody who’s on the lookout for a high-quality Gen AI mannequin in your each day studying translation, this text will information you to pick AI fashions primarily based in your wants. You’ll additionally discover ideas and methods to jot down higher prompts that may considerably enhance translation output high quality.

This text is based on our personal expertise. We centered on sure Gen AI as of Feb 2, 2025 (when Gemini 2.0 and DeepSeek had been launched), so that you may discover a few of our observations are completely different from present efficiency as AI fashions maintain evolving. 

We’re non-experts, and we tried our greatest to point out correct data primarily based on analysis and actual testing. The work we did is solely for enjoyable, self-learning and sharing, however we’re hoping to carry discussions to Gen AI’s cultural views. 

Many examples on this article are generated with the assistance of Gen AI to keep away from copyright considerations.

Our preliminary consideration was simple. Since our translation wants are associated to Chinese language, Japanese and English, the interpretation of the three languages was the precedence. Nonetheless, there have been only a few corporations that detailed this capacity particularly on their doc. The one factor we discovered is Gemini which specifies the efficiency of Multilingual.

Functionality	Multilingual
Benchmark	World MMLU (Lite)
Description	MMLU translated by human translators into 15 languages. The lite model consists of 200 Culturally Delicate and 200 Culturally Agnostic samples per language.
Gemini 1.5 Flash	73.7%
Gemini 1.5 Professional	80.8%

Kavukcuoglu, Koray. 2025. “Gemini Mannequin Updates.” Google DeepMind Weblog, February. https://blog.google/technology/google-deepmind/gemini-model-updates-february-2025/.

Second, however equally necessary, is the worth. We had been cautious concerning the finances and tried to not go bankrupt due to the usage-based pricing mannequin. So Gemini 1.5 Flash grew to become our major selection at the moment. Different causes we determined to proceed with this mannequin are that it’s essentially the most beginner-friendly choice due to the well-documented directions and it has a user-friendly testing atmosphere–Gemini AI studio, which causes even much less friction when deploying and scaling our undertaking.

Now Gemini 1.5 Flash has set a robust basis, throughout our first dry run, we discovered it has some limitations. To make sure a easy translation and studying expertise, now we have evaluated a number of different fashions as backups:

• Grok-beta (xAI): In late 2024, Grok didn’t have as a lot fame as OpenA’s fashions, however what attracted us was zero content material filters (This is among the points we noticed from AI fashions throughout translation, which will probably be mentioned later). Grok supplied $20 free credit per 30 days earlier than 2025, which makes it a horny, budget-friendly choice for frugal customers like us.
• Deepseek-V3: We built-in Deeseek proper after its stride into market as a result of it has richer Chinese language coaching knowledge than different options (They collaborated with employees from Peking College for knowledge labeling). Another excuse is its jaw-dropping low worth: With the low cost, it was almost 1/100 of Grok-beta. Nonetheless, the excessive response time was a giant subject.
• OpenAI GPT-4o: It has good documentation and powerful efficiency, however we didn’t actually think about this as an choice as a result of there isn’t a free tier for low-budget constraints. We used it as a reference however didn’t actively use it. We’ll combine it later only for testing functions.

We additionally explored a hybrid resolution –  suppliers that provide a number of fashions:

• Groq w/ Deepseek: it’s first an built-in mannequin platform to deploy Deepseek. This model is distilled from Meta’s LLM, though it’s 72B makes it much less highly effective however with acceptable latency. They supplied a free tier however with noticeable TPM constraints
• Siliconflow:  A platform with many Chinese language mannequin decisions, they usually supplied free credit.

When utilizing these fashions for each day translation (principally between languages Simplified Chinese language, Japanese, and English). We discovered that there are a lot of noticeable points.

1. Inconsistent translation of correct nouns/terminology

When a phrase or phrase has no official translation (or has completely different official translations), AI fashions like to supply inconsistent replies in the identical doc.

For instance, the Japanese title “Asuka” has a number of potential translations in Chinese language. Human translators often select one primarily based on character setting (in some circumstances, there’s a Japanese kanji reference for it, and the translator might merely use the Chinese language model). For instance, a feminine character could possibly be translated into “明日香”, and a male character is perhaps translated as “飞鸟” (extra meaning-based) or “阿斯卡” (extra phonetical-based). Nonetheless, AI output typically switches between completely different variations of the identical textual content.

There are additionally many alternative official translations for a similar noun within the Chinese language-speaking areas. One instance is the spell “Expecto Patronum” in Harry Potter. This has two accepted translations: 

---

Though I specify prompts to the AI ​​to translate to simplified Chinese language, it typically goes backwards and forwards between simplified and the normal Chinese language model.

2. Overuse of pronouns

One factor that Gen AI typically struggles with when translating from decrease context language to increased context language is including extra pronouns.

In Chinese language or Japanese literature, there are a number of methods when referring to an individual. Like many different languages,  third-person pronouns like She/Her are generally used. To keep away from ambiguity or repetition, the two approaches under are additionally quite common:

• Use character names.
• Descriptive phrases (“the woman”, “the trainer”).

This writing choice is the explanation that the pronoun use is far much less frequent in Japanese and Chinese language. In Chinese language literature. The pronoun throughout translation to Chinese language is simply about 20-30%, and in Japanese, this quantity might go decrease.

What I additionally need to emphasize is that this: There may be nothing proper or flawed with how ceaselessly, when, and the place so as to add the extra pronoun (In actual fact, it’s a standard follow for translators), nevertheless it has dangers as a result of it may well make the translated sentence unnatural and never align with reader’s studying behavior, or worse, misread the meant which means and trigger mistranslation.

Beneath is a Japanese-to-English translation:

Unique Japanese sentence (pronoun omitted)

Jack sees the CEO getting into the constructing. With confidence, pleasure, and powerful hope in coronary heart, go to convention room.

AI-generated translation (w/ incorrect pronoun)

Jack sees the CEO getting into the constructing. With confidence, pleasure, and powerful hope in his coronary heart, he goes to the convention room.

On this case, the writer deliberately avoids mentioning the pronoun, leaving room for interpretation. Nonetheless, as a result of the AI is making an attempt to comply with the grammar guidelines, it conflicts with the writer’s design.

Higher translation that preserves the unique intent

Jack sees the CEO getting into the constructing. With confidence, pleasure, and powerful hope in coronary heart, heads to the convention room.

3. Incorrect pronoun utilization in AI translation

The extra pronoun would doubtlessly result in a better fee of incorrect pronouns brought on by biased knowledge; typically, it’s gender-based errors. Within the instance above, the CEO is definitely a lady, so this translation is wrong. AI typically defaults to male pronouns except explicitly prompted

Jack sees the CEO getting into the constructing. With confidence, pleasure, and powerful hope in his coronary heart, he she goes to the convention room.

One other widespread subject is AI overuses “I” in translations. For some cause, this subject persists throughout virtually all fashions like GPT-4o, Gemini 1.5, Gemini 2.0, and Grok. GenAI fashions default to first-person pronouns when the topic is unclear. 

4. Combine Kanji, Simplified Chinese language, Conventional Chinese language

One other subject we encountered was AI fashions mixing Simplified Chinese language, Conventional Chinese language, and Kanji within the output. Due to historic and linguistic causes, many trendy Kanji characters are visually just like Chinese language however have regional or semantic variations.

Whereas some mix-use is wrong however is perhaps acceptable, for instance:

These three characters additionally look visually comparable, they usually share sure meanings, so it could possibly be acceptable in some informal eventualities, however not for formal or skilled communication.

Nonetheless, different circumstances can result in critical translation points. Beneath is an instance:

If AI immediately makes use of this phrase when changing Japanese to Chinese language (in a contemporary situation), the sentence “Jane acquired a letter from her distant household” might find yourself with “Jane acquired a bathroom paper from her distant household,” which is each incorrect and unintentionally humorous.

Please word that the browser-rendered textual content also can have points due to the shortage of characters within the system font library.

5. Punctuation

Gen AI typically doesn’t do a fantastic job of distinguishing punctuation variations between Chinese language, Kanji and English. Beneath is among the examples to point out how completely different languages use distinct methods to jot down dialog (in trendy widespread writing model):

This may appear minor however might impression professionalism.

6. False content material filtering triggers

We additionally discovered that Gen AI content material filter is perhaps extra delicate to Japanese and Chinese language (This occurred when utilizing Gemini 1.5 Flash). Even when the content material was fully innocent. For instance:

人並みにはできますよ！

I can do it at a mean degree!

Roughly talking, there have been about 2 out of 26 samples that triggered false content material filters. This subject confirmed up randomly.

Fully out of curiosity and to raised perceive the Chinese language/Japanese translation capacity of various Gen AI fashions, we carried out structured testing on 10 fashions from 7 suppliers.

Testing setup

Activity: Every AI mannequin was used to translate an article written in Japanese into simplified Chinese language by our translation extension. The Gen AI fashions had been linked by API.

Pattern: We chosen a 30-paragraph third-person article. Every paragraph is a pattern of which the character varies from 4 to 120.

Processed outcome: every mannequin was examined thrice, and we used the median outcome for evaluation.

Analysis metrics

We totally respect that the standard of translation is subjective, so we picked three metrics which can be quantifiable and signify the challenges of high-context language translation.

Pronoun error fee

This metric represents the frequency of faulty pronouns that appeared within the translated pattern, which incorporates the next circumstances:

• Gender pronoun incorrectness  (e.g., utilizing “he” as an alternative of “she”).
• Mistakenly swap from third-person pronoun to a different perspective

A paragraph was marked as affected (+1) if any incorrect pronoun was detected.

Non-Chinese language return fee

Some fashions randomly output Kanji, Hiragana, or Katakana of their responses. We had been to rely the samples that contained any of these, however each paragraph contained no less than one non-Chinese language character, so we adjusted our analysis to make it extra significant:

• If the returned translation comprises Hiragana, Katakana, or Kanji that have an effect on readability, will probably be counted as a translation error. For instance: If the AI output 対 as an alternative of 对, it received’t be flagged, since each are visually comparable and don’t have an effect on which means.
• Our translation extension has a built-in non-Chinese language characters operate. If detected, the system retranslates the textual content as much as thrice. If the non-Chinese language stays, it would show an error message.

Pronoun Addition Fee

If the translated pattern comprises any pronoun that doesn’t exist within the authentic paragraph, will probably be flagged.

Scoring method

All three metrics had been calculated utilizing the next method. 𝑁 represents the variety of affected paragraphs (samples). Please word, if a paragraph (pattern) comprises a number of same-type errors, will probably be counted 1 time.

Fee=N/30*100%

High quality rating: to have a greater sense of general high quality. We additionally calculated the standard rating by weighting the three metrics primarily based on their impression on translation: Pronoun Error Fee > Non-CN Return Fee > Pronoun Addition Fee.

Within the first run, we solely supplied a foundational immediate by specifying persona and translation duties with out including any particular translation tips. The aim was to judge AI translation baseline efficiency.

Statement

Typically talking, the general translation high quality is just not enough sufficient to carry the viewers an “optimum studying expertise”. 

For error return fee, even the highest-rated mannequin, Claude 3.5 Sonnet, nonetheless received a 30% error fee. This implies apparent translation deficiencies could possibly be simply noticed roughly 1 in each 4 sentences. Apparently, we discovered that the incorrectly added pronouns had been at all times first-person “I”. It is perhaps as a result of the gap between the phrase “I” is nearer to the verb vectors than different pronouns in vector house. 

Pronoun Addition Charges exceeded 50% in most fashions. This frequency is far more aligned with English writing habits than with Chinese language (20–30%) or Japanese (even decrease). This may stem from the AI mannequin coaching knowledge.  In line with OpenAI’s dataset statistics, GPT-3’s coaching knowledge consists of 92.65% English, 0.11% Japanese, 0.1% Simplified Chinese language, and 0.02% Conventional Chinese language. The variations present coaching knowledge focuses on English and revealed the potential cause for translating struggles, together with the difficulty of blending simplified Chinese language and conventional Chinese language in output, which was additionally noticed in testing.

Language	Variety of phrases	% of complete phrases
English	181014683608	92.64708%
Japanese	217047918	0.11109%
Simplified Chinese language	193517396	0.09905%
Conventional Chinese language	38583893	0.01975%

(OpenAI, “Languages by Phrase Rely in GPT-3 Dataset,” final modified 2020, https://github.com/openai/gpt-3/blob/master/dataset_statistics/languages_by_word_count.csv).

We did a number of not-so-fancy options as a way to have a constant good translation. 

Re-translation with completely different fashions

If situations enable (finances and technical feasibility), you could possibly use the backup fashions to re-translate circumstances that the first mannequin can’t translate. This is applicable to untranslated Japanese textual content (non-Chinese language returns). We primarily used Grok-beta until mid-Jan 2025.

Translation steerage: pronoun 

To forestall the AI ​​from inserting topics unnecessarily, we particularly instruct AI to disregard grammar guidelines. Listed below are the hints we use:

**Pronoun Dealing with Necessities:** 

* **Pronoun Consistency** Comply with the unique textual content strictly.

* **Pronoun dealing with** Don’t add topics except explicitly talked about within the authentic textual content, even when it leads to grammatical errors.

Within the meantime, offering examples is fairly helpful for AI to grasp your necessities.

**Pronoun Dealing with**

* **Unique Japanese sentence (topic omitted): ジャックは最高経営責任者が建物に入るのを見た。自信と興奮、そして強い希望を胸に、会議室へ向かった

* **Incorrect AI-generated translation (pointless topic added): Jack sees the CEO getting into the constructing. With confidence, pleasure, and powerful hope in his coronary heart, he goes to the convention room

* **Good instance (grammatically appropriate with out pronoun): Jack sees the CEO getting into the constructing. With confidence, pleasure, and powerful hope in coronary heart, heads to the convention room.

* **Acceptable instance (omitted topic however grammatically incorrect): “Jack sees the CEO getting into the constructing. With confidence, pleasure, and powerful hope in coronary heart, go to convention room.”

Translation steerage: glossary

I additionally wrote a glossary record like under. This considerably reduces the looks of faulty pronouns and standardizes the terminology translation.

| Japanese | English | Chinese language | Notes | 

| シカゴ | Chicago | 芝加哥 | Official location title | 

| 俺 | I | 我 | First-person pronoun, casual, daring, and tough in tone, principally utilized by males | | アスカ | Asuka | 飞鸟 | A younger male character title  |
…

Adjusting Mannequin Parameters

Typically talking, decreasing the parameters helps keep away from randomness. As somebody who likes writing prompts, AI following the immediate extra strictly is far more of a precedence than being inventive in output. So, we lowered top-p, top-k and temperature. Deepseek AI formally recommends a temperature of 1.3 for translation, however for higher immediate adherence, we adjusted it to 1.0 or decrease. TopK was diminished by 20. This works fairly effectively. Gemini 1.5 flash was used to randomly output a full paragraph content material that didn’t exist within the authentic article. This subject by no means exhibits once more after adjusting the parameters.

This technique reduces variability however is just not scalable, as a result of every mannequin responds in another way relying on their dimension, development, and so forth. 

For the second spherical of the take a look at, we apply the interpretation steerage as a comparability.

Statement

After making use of translation steerage, the general translation high quality of all fashions improved considerably. Beneath is an in depth comparability of the efficiency of various AI fashions beneath these improved situations.

You possibly can simply inform that with translation steerage the interpretation high quality has been considerably improved. 

For the first metric Pronoun Error Fee: Claude-3.5 Sonnet, OpenAI GPT-4o, DeepSeek V3, because the entrance runner, confirmed robust accuracy. Gemini 2.0 Flash and Moonshot-V1 (Kimi) had minor points however had been enough for many non-professional Japanese-to-Chinese language translation wants.

Based mostly on the results of the Pronoun Addition Fee. Claude-3.5 Sonnet strictly adopted translation steerage and executed precisely with solely an 8% Pronoun Addition Fee. Gemini 2.0 Flash had a 20% pronoun addition fee. It’s a suitable outcome because it’s aligned with Chinese language writing habits.

The most effective mannequin choice will depend on private wants, contemplating elements akin to finances, request per minute (RPM) limits, and ecosystem compatibility. Selecting an AI mannequin for English-Chinese language-Japanese translation.

For these with out finances constraints, Claude-3.5 Sonnet and OpenAI GPT-4o are the strongest decisions due to their general robust efficiency.

For entry-level builders in North America, Gemini 2.0 Flash is a wonderful selection due to its reasonably priced worth, and good response time. Another excuse we selected it as the first supplier is as a result of Google’s cloud service ecosystem (OCR, cloud storage, and so forth.) makes it simpler to scale growth initiatives.

For Gen AI energy customers trying to stability worth and high quality, DeepSeek affords low costs, limitless RPMs, and open-source flexibility. It is a robust selection for cost-sensitive customers who don’t need to compromise translation high quality. Nonetheless, when utilizing the official API platform in North America, we skilled lengthy response time, which generally is a limitation in case you have a necessity for real-time or long-context translations. Thankfully, there are a lot of companies built-in DeepSeek on different servers (akin to Microsoft Azure, Groq, and Siliconflow, and even you could possibly deploy into your individual native servers), or utilizing it inside China can keep away from these points. Moreover, mannequin dimension can considerably have an effect on translation efficiency – in the event you might, use the full-power 671B model for greatest outcomes.

We perceive that these assessments should not excellent. Even when we tried to make sure a various and proper knowledge quantity, there may be a lot room for enchancment. For instance, our pattern dimension is just not massive sufficient for statistical significance. AI mannequin efficiency fluctuates at any second, points like terminology translation inconsistency weren’t captured however is perhaps necessary indicators for some audiences, and the interpretation high quality wasn’t capable of be mirrored quantitatively. We supplied the take a look at only for studying and hopefully, function reference factors for you.

We’re actually grateful for the advances in Generative Ai, which have helped bridge the hole of language and make information extra accessible for individuals talking completely different languages and from completely different cultures.

Nonetheless, we will nonetheless see many challenges stay to be overcome—particularly for non-English languages.

There may be an opinion that translation doesn’t want superior AI fashions, however“ok” is just not sufficient. I can see that this view is perhaps appropriate from a price perspective and is sensible from an English-centric perspective. Nonetheless, if the usual “good” is predicated on official efficiency stories from AI suppliers, it’d precisely mirror the efficiency of non-English translation. As you’ll be able to clearly see, high-context languages ​​akin to Japanese and Chinese language translation nonetheless wrestle with accuracy and fluency. There may be nonetheless a street forward to enhance AI translation high quality, higher contextual understanding and cultural consciousness are mandatory.

Value

Deepseek has introduced extra competitors to the AI ​​translation market. Pricing remains to be a key issue for individuals and typically has extra weight than efficiency.

In case you have mid to high-volume each day translation wants (educational studying, information, video caption, and so forth.), utilizing a premium mannequin can value anyplace from $20 to $80 per 30 days. For companies coping with localization and internationalization, these prices would enhance rapidly.

No method round it: prompting for higher translation

One other main problem is AI fashions nonetheless require customers to jot down lengthy, complicated prompts to attain primary readability. For instance, when translating skilled subjects in sure area of interest domains, I’ve no selection however to jot down prompts of over 5000 characters in English (virtually writing a complete doc) simply to information the AI ​​to a suitable high quality. To not point out the longer prompts = increased token utilization.

If AI is actually going to interrupt language obstacles, there may be nonetheless plenty of room for enchancment to make translation fashions extra correct, extra context-aware, and fewer depending on lengthy prompts. There’s nonetheless plenty of work to do to make AI translation simple, cost-effective, and really accessible to everybody, however AI has already achieved greater than anybody might have imagined, and I have a good time and am grateful for these technological developments.