Yiming Peng /i' miŋ/
Senior Data Engineer · PhD in Reinforcement Learning
I design data systems, also fix the leaky "pipes" 😏
- → Currently at Wētā FX, building data infrastructure behind world-class visual effects
- → 10+ years building DataOps/MLOps platforms across New Zealand and global production environments
- → Contributor to Apache Airflow with several merged fixes and improvements
- → Volunteering in the tech community through meetups, talks, and experience-sharing events
Recent
Community & Events
IEEE Young Professionals panel on transitioning from study and research into industry careers.
Certifications
View all on Credly
Selected Projects
Eliminated recurring pipeline failures; production-critical at Wētā FX.
DataOps Platform @ Wētā FX
On-premises data platform powering BI and ML workflows. Built with Airflow, ClickHouse, and Ansible across a complex production environment.
Open source work focused on practical, production-informed improvements.
Open Source Contributions — Apache Airflow
Contributed around 4-5 pull requests and issue fixes to Apache Airflow, focused on reliability and developer experience improvements.
Dramatically reduced data incidents; stakeholders now trust the data.
Data Quality & Observability System
Platform-wide data quality framework using Great Expectations, with GitLab CI integration for automated validation on every pipeline run.
Made ML model deployment routine rather than heroic at Chorus NZ.
Kubernetes MLOps Platform
Kubernetes-based MLOps platform to close the gap between experimentation and production. First production ML infrastructure at the organisation.
Experience
Publications
12 peer-reviewed papers in reinforcement learning and evolutionary computation (2012-2024) - 165+ citations.