KAI-CHOU YANG · 楊凱州
From data to capability.
I build data and training systems for language models and agents, from large-scale data curation to agent post-training.
I build the systems behind how language models learn and act.
I’m a machine learning engineer building data and training systems for language models and agents. At Taiwan AI Labs, I lead NLP work across data engineering, post-training, and agent systems, with a current focus on learning from synthetic environments, tool use, and verifiable interactions.
I came to the field through competitions and open source, and still think of the work as a craft—something to practise with care. Outside work, I make small things and publish what comes out: code, models, and datasets around conversational AI, language-model training, and evaluation.
I write, too. I’m interested in technology where it meets everyday life, and when I explain something difficult, I try to find the story around it and examples beyond the field. If something I’ve written is wrong or could be better, I’d be glad to hear about it.
Deep research · Personal project
From open questions to inspectable answers.
A deep-research system turns open-ended questions into source-grounded reports. Specialist agents gather evidence before writing, then verify claims and citations against that record. Each report exposes its source trail, supporting excerpts, and audit results, not just an answer.
Evidence-first research · Multi-agent orchestration
Language models · Open project
Better Traditional Chinese through better data.
Retrieval-augmented data synthesis and post-training for Traditional Chinese language models, with released training recipes, models, and instruction data.
2nd placeGoogle / Kaggle2025US$30,000 prize
Chinese writing · Open project
Writing as plan, draft and revision.
A Chinese writing model trained on its own process: planning, drafting, review and revision form a data-synthesis loop, with rejection sampling selecting the examples worth training on. Released as an open model collection alongside Zhiyin, a Chinese writing benchmark with its scoring code and run records.
Data synthesis · Model evaluation
Code, models and learning data, from early NLP tools to post-training and evaluation.
Code & implementations
Retrieval-based Chinese conversation, with a PTT-derived dialogue corpus.
Solution code from four competition entries in text matching and classification.
Hands-on Chinese word embeddings and sequence-to-sequence learning.
Models & training data
Learning data built around execution feedback and chosen–rejected pairs.
Chinese science QA and mathematical reasoning, with answer-level checks.
Multi-turn tool-use trajectories for training agents: ToolScale rollouts from two frontier models, and a larger set filtered against ground-truth tool calls.
Google · KaggleAdapting Gemma to a language or cultural context.
Create a Gemma 2 model variant for a specific language or cultural context, with a clear, reproducible account of the tuning approach.
Kyara focuses on Traditional Chinese: retrieval-grounded instruction synthesis, translation and rewriting, followed by supervised fine-tuning and preference optimization. The work includes released models, training recipes, and instruction and preference datasets.
Google AI · KaggleResolving ambiguous pronouns across genders.
Given a passage, an ambiguous pronoun and two candidate entities, predict A, B or neither. The challenge evaluates pronoun resolution across genders using multiclass log loss.
ByteDance · WSDM CupClassifying the relationship between two headlines.
Given a known fake-news headline A and a new headline B, classify their relation: agreed, disagreed or unrelated. The task compares headline pairs against a known claim, rather than judging an isolated article as true or false.
Alibaba · CIKMAdapting question-pair matching from English to Spanish.
Determine whether two questions have the same meaning in Spanish, using English as the source language for adaptation. The challenge focuses on short-text semantic matching under limited target-language supervision.
Journal of Medical Internet ResearchPaper
This retrospective study examines how Taiwan’s online news environment and public search interest were associated with COVID-19 vaccination uptake in 2021. News from 26 online outlets was analyzed with Islander to derive incitement and suspicion indicators, which were aggregated into a measure of suspicious-news prevalence. These signals were combined with Google Trends and official infection and vaccination data. Weekly statistical analyses examined vaccination patterns while accounting for vaccine availability and the interaction between news quality and public attention.
The analysis found that the combination of greater suspicious-news prevalence and stronger search interest was negatively associated with vaccinations in the following week. The study connects large-scale media analysis with public-health monitoring, rather than treating news volume alone as a measure of influence. Its findings concern population-level associations: automated news scores and search activity are proxies, not direct measurements of individual exposure or proof of a causal effect on vaccination decisions.
arXiv · PreprintPaper
Islander brings news collection, event organization, and quality analysis into a real-time monitoring system. It clusters a stream of articles from multiple outlets into evolving events, allowing readers to compare coverage across sources instead of navigating isolated stories. Representative titles and generated topic tags support browsing and retrieval. The tagging component uses a conditional language model trained on news fields, and the paper evaluates its retrieval-oriented tag generation against extractive baselines such as TextRank and TF-IDF.
The system also exposes headline incitement, title bias, and content subjectivity as signals for examining how an event is presented. Social-media discussions provide additional views of attention and sentiment, while user co-occurrence representations support the analysis of potentially coordinated activity. These components are integrated into an interface for searching events and following their development. The contribution is an end-to-end news-analysis workflow; its quality and coordination indicators support investigation rather than establish whether an article is true or an account is inauthentic.
AAAI 2020Paper
Different sentence encoders capture different aspects of language, making simple concatenation or averaging an unreliable way to combine them across tasks. The Self-Inference Neural Network (SINN) addresses this by learning how its own representations should interact. Recurrent and convolutional encoders provide complementary contextual features; an interaction layer and self-attention construct additional semantic evidence. A shared vector gate then selects and mixes these representations before pooling them into a fixed-size sentence embedding.
Experiments cover natural language inference, news classification, and sentiment analysis on MultiNLI, SciTail, AG News, and SST. The paper reports strong results relative to the sentence-encoding baselines evaluated, with particularly competitive inference performance. Fusion comparisons, ablations, and error analyses show how adaptive gating can retain useful information even when one component encoder is weaker. The work presents representation fusion as a learned inference problem, rather than a fixed combination rule, and examines how the preferred mixture changes with the task.
GeBNLP Workshop at ACL 2019Paper
Fill the GAP studies how pretrained BERT representations can resolve ambiguous pronouns when the answer is one of two candidate entities or neither. Instead of relying on the final transformer layer by default, the approach examines which layers contain the most useful reference information. Frozen contextual embeddings are processed through low-dimensional projections and interaction-based classifiers, with dropout used to reduce reliance on redundant features. Predictions from different BERT configurations are combined into an ensemble.
The system placed fourth in Kaggle’s Gendered Pronoun Resolution competition without data augmentation. Beyond the competition result, the paper analyzes layer selection, feature redundancy, and pronoun–candidate attention patterns to understand why the representation choices matter. The analysis shows that the final layer is not necessarily the most effective choice for this task, while error cases illustrate the continued difficulty of examples requiring broader contextual or world knowledge. The contribution combines a practical resolution system with an empirical study of BERT’s internal representations.
WSDM Cup 2019 · arXivPaper
This WSDM Cup system treats Chinese fake-news analysis as a relation-prediction problem. Given a headline already identified as false and a second headline, the task is to decide whether the second agrees, disagrees, or is unrelated, rather than to classify an isolated article as true or false. The solution combines natural-language-inference architectures with fine-tuned BERT models, using complementary representations and ensemble predictions to handle the headline pairs.
To address noisy supervision, the training procedure uses model-generated soft labels and staged retraining. A further inference step exploits transitive relations among overlapping headlines in the competition data, while ensemble predictions handle the remaining cases. The final system achieved 88.063% accuracy and third place. The report shows how task formulation, label refinement, and dataset-specific relational structure can complement pretrained language models. Its results apply to the competition’s pairwise classification setting; the transitivity step depends on that data structure and is not a general-purpose guarantee of factual correctness.
Data Science Collective · MediumArticle
Kyara approaches Traditional Chinese language-model adaptation through the design of training data. Its instruction-synthesis pipeline retrieves related evidence, checks whether that evidence is sufficient to answer a question, and reformulates the findings into grounded responses. Translation, rewriting, and persona-based augmentation broaden the instruction mixture used for supervised fine-tuning, connecting knowledge coverage with the variety of requests a conversational model must handle.
Preference optimization adds a second source of learning signal. The article describes reward-model scoring, LLM judges with response-order consistency checks, and answer-verified mathematics examples for constructing chosen–rejected pairs. The released resources include one million instruction examples and approximately 150,000 preference pairs. Evaluation combines tasks with reference answers and open-ended Chinese conversation assessed by LLM judges. Alongside Kyara’s second-place Gemma competition result, the article documents practical choices around data filtering and training objectives, while acknowledging that retrieval and generated supervision can still introduce errors.
TDS Archive · MediumArticle
Closer addresses cross-lingual short-text matching: using English resources to support semantic matching between Spanish questions. The solution combines word- and character-level inputs with engineered similarity features, then builds complementary models around decomposable attention, recurrent comparison, and densely augmented convolutions. Because matching should not depend on which question is presented first, the architecture explicitly symmetrizes pair interactions rather than relying on a single input order.
Training proceeds in two stages. Initial models produce blended predictions that become soft supervision for a second round of training, helping the system cope with noisy labels and share information across model families. A final DART-based stacking model combines their outputs. The second-place CIKM AnalytiCup solution is presented as a reproducible engineering account: the article explains feature construction, architectural choices, regularization, distillation, and ensembling, and discusses why label inconsistency matters for question matching. Together, these choices connect cross-lingual representation learning with a carefully structured prediction pipeline.
More implementation notes and derivations live at 雷德麥的藏書閣.
Taiwan AI Labs
Mar 2023 — Present
Lead the technical direction of language-model and agent development, connecting large-scale data engineering, post-training and agent systems.
Data curation, knowledge synthesis, supervised fine-tuning, rejection sampling, reinforcement learning, and stateful agent systems.
Sep 2019 — Mar 2023
Built retrieval, question-answering and language-understanding systems, with an emphasis on weak supervision, domain adaptation and reusable training pipelines.
Multilingual dense retrieval, passage reranking, RAG, and Kubeflow training pipelines.
National Cheng Kung University
Computer Science and Information Engineering
2017 — 2019
GPA 4.30 / 4.30 Ranked 1st
Honorary member of the Phi-Tau-Phi Scholastic Honor Society.
Teaching assistant for Introduction to Data Science, Data Mining, and Discrete Mathematics.
Speaker for machine-learning lectures at the Industrial Technology Research Institute and Delta Electronics.
2013 — 2017
GPA 3.94 / 4.30 Ranked 4th
Academic Excellence Awards, 2015 & 2016.
Honorable mention at the graduation exhibition.
Research assistant on a question-answering project funded by the Ministry of Science and Technology.
Qwen Dev Ambassador