Yiming Peng /i' miŋ/
Senior Data Engineer · PhD in Reinforcement Learning
I design data systems, also fix the leaky "pipes" 😏
- → Currently at Wētā FX, building data infrastructure behind world-class visual effects
- → 10+ years building DataOps/MLOps platforms across New Zealand and global production environments
- → Contributor to Apache Airflow with several merged fixes and improvements
- → Volunteering in the tech community through meetups, talks, and experience-sharing events
Recent
Community & Events
IEEE Young Professionals panel on transitioning from study and research into industry careers.
Certifications
View all on Credly
Selected Projects
Eliminated recurring pipeline failures; stakeholder teams now rely on it as production-critical infrastructure.
DataOps Platform
Production on-premises data platform powering BI and ML workflows across Wētā FX. Built and maintains 25+ ETL pipelines ingesting from disparate sources — solving the persistent problem of unreliable, undocumented data hand-offs between departments.
Python · Airflow · ClickHouse · Ansible
deep dive comingOpen source work focused on practical, production-informed improvements.
Apache Airflow OSS Contributions
Contributor to Apache Airflow — the de facto standard for data pipeline orchestration used by thousands of organizations worldwide. Contributions focus on stability, usability, and operator improvements drawn from real production experience.
Python · Open Source · Apache Airflow
deep dive comingDramatically reduced data incidents; stakeholders gained confidence to act on data without manual verification.
Data Quality & Observability System
Designed and built a platform-wide data quality framework from the ground up using Great Expectations — moving the team from reactive fire-fighting to proactive anomaly detection. Integrated into the CI/CD pipeline so quality checks run automatically on every deploy.
Great Expectations · Python · GitLab CI
deep dive comingFirst production ML infrastructure at the organisation — made model deployment a routine operation rather than a heroic effort.
Kubernetes MLOps Platform
Co-designed and built a Kubernetes-based MLOps platform at Chorus NZ to close the gap between data science experimentation and production deployment. Enabled model training, versioning, and serving pipelines within a unified, reproducible infrastructure.
Kubernetes · Kubeflow · AWS EKS
deep dive comingExperience
Publications
12 peer-reviewed papers in reinforcement learning and evolutionary computation (2012-2024) - 165+ citations.