Be taught a language by passively turning the pages of a textbook.
You actually make progress when the language comes again to you.
As you take a look at pictures, hear precise sentences, strive talking, and get suggestions, every part will ultimately begin to click on in your head.
Beforehand, such suggestions required the instructor to be current always.
Immediately, generative AI can act as an always-available AI language tutor in your cellphone or pc.
After I began studying Mandarin 10 years in the past, I noticed many foreigners who had poor pronunciation and struggled to be understood by locals in on a regular basis conversations.
I’m satisfied {that a} wealthy vocabulary is ineffective if the pronunciation is just not good.
I nonetheless keep in mind sitting in my house in Shanghai repeating the identical sentences again and again with out anybody correcting me.
Years later, once I found generative AI, I remembered the engineers in China who have been scuffling with grammar books and tone.
I wished to construct a instrument that might have been helpful earlier than.
As a startup founder, I haven’t got lots of free time, so I wanted a option to shortly construct and check new instruments.
That is why I turned to n8n to construct an assistant that makes practising Chinese language simpler.
This text reveals how you need to use n8n and multimodal AI to construct a language studying “studying companion” that:
- Appropriate pronunciation utilizing text-to-speech
- Create workouts to be taught vocabulary lists
- Generate pictures to clarify phrases and context for flashcard-style observe
They display how AI and low-code platforms like n8n can assist individuals studying advanced languages.
All this collectively prices lower than 1 euro per 30 days, even in the event you use it day by day.
AI for pronunciation and oral understanding
My title is Sameer. I am a provide chain skilled who struggled with Mandarin throughout my six years in China.
Let me introduce you to ying, an AI-powered language coach I developed final week.
This can be a net software designed to assist my Chinese language studying journey after not practising Chinese language for over 5 years.
It contains three options:
- pronunciation observe
- A number of Alternative Questions (MCQ)
- flash playing cards
Utilizing every function, we’ll present you find out how to use multimodal AI to enhance your Chinese language studying, listening, and pronunciation.
Why is Chinese language pronunciation so vital?
To emphasise the significance of utilizing the right tone in Mandarin, let me share a real story from China.
Sooner or later, I used to be invited to a job interview with China’s largest transportation firm, valued at billions of {dollars}.
All conversations have been in Chinese language.
I fastidiously ready my essay to focus on how I leveraged knowledge science to enhance warehouse operations.
Sooner or later, I wished to say this. “I take advantage of knowledge science to enhance choosing productiveness in my warehouse.”
The verb “choose” means to take away items from cabinets or racks in a warehouse.
In Chinese language my colleague used the next verbs Chess buy (jiƎn huò) Let me clarify this course of.
However as an alternative of claiming Jianfuhe stated. building fireplace.
This can be a utterly totally different phrase, and one you positively do not need to use in a job interview.
To be well mannered right here, as an instance: building fireplace is a impolite phrase.
The supervisor burst out laughing.
I did not perceive why till I later reported it to the headhunter and repeated these phrases to her.
In that second, I realized that Chinese language pronunciation does not simply sound pure.
You could know hundreds of phrases, but when your tone is unsuitable, individuals will not perceive you.
That is why the primary function of my app is AI pronunciation coach.
Observe utilizing speech-to-text recognition
Utilizing speech-to-text and inference, the app listens to what I am saying, compares it to the goal sentence, and offers suggestions on which tones and syllables have been off.
The main target right here is to enhance the pronunciation of logistics and provide chain phrases (my space of experience).
For every phrase:
- Simplified Chinese language phrases: joint
- Sentences I used to observe my pronunciation: This connection should be stopped earlier than publishing.
- English translation: This contract of carriage should be signed earlier than the products will be shipped.
For rookies, it’s also possible to add phonetic symbols (Mandarin Pinyin) utilizing a toggle.
How can I observe my pronunciation?
To document your personal writing, merely press the microphone button on the backside.
The recording is mechanically despatched to the backend for evaluation, the place my pronunciation is in comparison with the right pronunciation.
After a number of seconds, I acquired the suggestions.
The suggestions could be very detailed. Concentrate on the phrase you mispronounced.
It is like having a private instructor right your work in actual time, however this instructor by no means will get bored.
After all, that is no substitute for instructor for one-on-one classes, however it may be helpful for post-class observe.
After I began studying Chinese language, I’d spend evenings (after work) alone, repeating easy sentences and getting used to the nuances of tone.
I did not have a suggestions loop again then. This instrument would have been very useful.
How does it work?
GenAI text-to-speech and inference capabilities
The backend is an easy n8n workflow related to the frontend through a webhook.
The text-to-speech function is used to transform audio information despatched from the entrance finish into speech (Pinyin).
The output of this Gemini Audio Transcription node contains audio.
[
{
"content": {
"parts": [
{
"text": "zuò pǐn huò zǒnggòng fàng zài èrshí ge tuōpán shàng.n"
}
],
"function": "mannequin"
},
"finishReason": "STOP",
"avgLogprobs": -0.16858814502584524
}
]This pinyin is distributed to the AI node Pronounciation Evaluation Additionally embody the pronunciation of the goal.
On this instance, you mispronounced the penultimate phrase.
That is precisely what the agent stated in his suggestions.
It reveals how text-to-speech capabilities can be utilized along with generative AI mannequin inference to enhance pronunciation.
This may be tailored to any language.
What about picture era and speech-to-text conversion?
Generative AI for content material era
In the event you observe the appliance’s person interface, you’ll discover that every phrase has the next content material:
- Picture diagram
- Contextual sentences
- Audio transcription is feasible from the microphone icon
This content material is generated utilizing an AI mannequin and offers quite a lot of studying supplies for the second function: flashcards.
Textual content-to-speech resolution
An effective way to observe pronunciation is to pay attention and repeat.
So earlier than you document your sentences, you need to use this primary text-to-speech function to discover ways to pronounce phrases.
For this, we use Google’s Textual content-to-Speech API, which could be very handy and free.
from gtts import gTTS
def generate_speech(textual content: str, lang: str):
filename = f"{uuid4().hex}.mp3"
filepath = f"./knowledge/gtts/{filename}"
tts = gTTS(textual content=textual content, lang=lang)
tts.save(filepath)With a number of strains of code, you possibly can generate speech synthesis for any phrase utilizing the suitable language code.
That is precisely the identical instrument we used to generate flashcards that we launched three years in the past in In direction of Information Science.
The concept on the time was to enhance listening comprehension by including audio to flashcard solutions.
What about lengthy texts?
The issue with Google text-to-speech is that it sounds robotic.
Fortuitously, we’ve 11 laboratories.
The above workflow is related to the app through a webhook.
Eleven lab nodes that obtain the output of the AI agent Generate Instance Generate an audio model of a sentence.
Customers can hear sentences pronounced like a local speaker.
What’s left? Questions, illustrations, and so forth…
Creating educating supplies
As defined within the earlier part, the textual content can be generated utilizing AI.
The Gemini-powered AI agent node takes the phrases to be taught as enter and generates sentences utilizing the system prompts beneath.
You're a Chinese language language tutor for professionals.
Given a Chinese language phrase, you MUST return a JSON object with EXACTLY these keys:
- "sentence": a brief Chinese language sentence utilizing the phrase in a enterprise or
daily-life context
- "pinyin": the pinyin of the total sentence
- "english": the English translation of the sentence
Return ONLY legitimate JSON. No explanations, no backticks, no additional textual content.
Instance:
{
"sentence": "我去仓库检查货物。",
"pinyin": "Wǒ qù cāngkù jiǎnchá huòwù.",
"english": "I'm going to the warehouse to examine the products."
}This enables for an virtually infinite number of workouts.
And, most significantly, pictures generated with Gemini’s Nano Banana that assist join phrases to their context.
After studying hundreds of kanji, I noticed that pictures assist me keep in mind new phrases.
That is precisely what I take advantage of with the flashcard function.
The n8n backend offers the frontend with:
- Chinese language phrases to recollect with pinyin and English translations
- Instance sentences generated with GPT and their translations
- Instance picture generated by Gemini
The entrance finish then manages the cardboard reversal mechanism.
If you need to recreate this resolution on your wants, I’ve shared the next: Similar workflow on my GitHub.
Do you want a number of selection questions? Gen AI is right here to assist!
Generate workouts from vocabulary checklist
The ultimate function generates multiple-choice questions for studying the identical vocabulary checklist.
Ask Gemini to generate questions from a vocabulary checklist utilizing a number of selection choices with just one right reply.
[
{
"output": {
"question": "Which of the following is the correct Chinese translation for 'Variable Pricing'? Please answer with A, B, C, or D.",
"options": {
"A": "仓库",
"B": "可变定价",
"C": "卡车司机",
"D": "投标"
},
"correct": "B",
"right_feedback": "Great job! 可变定价 (kě biàn dìng jià) means Variable Pricing.",
"wrong_feedback": "Oops! The correct answer is B: 可变定价 (kě biàn dìng jià), which means Variable Pricing."
}
}
]The frontend makes use of this output to supply tailor-made suggestions to your questions.
The backend for this function is predicated on the n8n workflow, which I additionally shared on GitHub. AI-powered language teacher using GPT.
conclusion
We developed this app to experiment with how AI can improve studying capabilities.
After not talking Chinese language for almost 5 years, this multimodal AI assistant proved to be an important assist.
Your complete backend is constructed on n8n for speedy prototyping and seamless integration.
Not acquainted with n8n and need to be taught?
My YouTube channel has an entire tutorial for rookies that guides you from creating an occasion to organising your credentials.
After finishing this tutorial, it is possible for you to to make use of one of many workflows shared on my web site. repository.
I haven’t got the time to dedicate to face-to-face Chinese language lessons, so I can have an assistant who can accommodate my schedule.
Can we do higher?
The “roadmap” for this small venture contains:
- Provides advanced grammar workouts that may be performed orally (combining studying comprehension, grammar and pronunciation)
- Implement a writing module that makes use of picture processing to right calligraphy
We anticipate to ship by the primary quarter of 2026, topic to availability.
about me
Let’s join linkedin and Twitter;I am a provide chain engineer who makes use of knowledge analytics to enhance logistics operations and scale back prices.
In the event you want consulting or recommendation on analytics and sustainable provide chain transformation, please contact me within the following methods: Logigreen Consulting.
