Zheren Dong
Iām a research engineer on the AI Research Team at Augment Code, where I focus on post-training LLMs and data curation strategies that improve retrieval performance for coding agents in production. This involves building pipelines to process real user data, handle distribution shifts, and maintain training data quality. Outside of work, I pursue independent ML research; my recent work on spelling-aware embeddings shows how simple architectural changes can improve language modeling across benchmarks.
Previously, I worked at Applied Intuition and Rivos (now part of Meta).
News
| Jan 25, 2026 | New preprint on Spelling Bee Embeddings for Language Modeling is now on arXiv! š |
|---|
Selected Publications
Spelling Bee Embeddings for Language Modeling
arXiv preprint arXiv:2601.18030, Jan 2026
We introduce a simple modification to the embedding layer. The key change is to infuse token embeddings with information about their spelling. Models trained with these embeddings improve not only on spelling, but also across standard benchmarks. We conduct scaling studies for models with 40M to 800M parameters, which suggest that the improvements are equivalent to needing about 8% less compute and data to achieve the same test loss.
Experience
Jan 2025 ā Present
Palo Alto, CA
Palo Alto, CA
Member of Technical Staff, AI Research Team
Augment Code
- Retrieval Perforamce and Context Engineering
- Prev: Inline image support in Agent
Sep 2023 ā Jan 2025
Mountain View, CA
Mountain View, CA
Software Engineer, Vehicle Platform Team
Applied Intuition
- Next-gen Software Defined Vehicle (SDV) platform
- Data infrastructure for vehicle telemetry and fleet health monitoring
- On-board runtime environment and applications
Jun 2022 ā Aug 2023
Mountain View, CA
Mountain View, CA
Member of Technical Staff
Rivos Inc.
- Rust runtime support library for RISC-V system bootstrapping
- Rust-based DDR5 SPD decoder/encoder CLI tool per JEDEC standard (intern project in summer 2022)
May 2021 ā Sep 2021
Beijing, China
Beijing, China
Software Engineer Intern
Alibaba Group
- Redesigned TensorFlow-based user vector generation module in C++ for vector and tree-based deep match retrieval system
Education
2021 ā 2022
Irvine, CA
Irvine, CA
M.S. Computer Science
University of California, Irvine
2016 ā 2020
Santa Barbara, CA
Santa Barbara, CA
B.S. Computer Science (Honors)
University of California, Santa Barbara
Skills
Languages
Python C/C++ Rust Go Java TypeScript SQL
ML & Data
PyTorch TensorFlow Ray Spark CUDA
Infrastructure
Kubernetes Docker GCP AWS BigTable BigQuery Kafka Redis PostgreSQL
</div>