$ cat experience.txt

Projects:

• Developing and managing an on-premises DataOps platform that includes 25+ ETL pipelines, focusing on improving data ingestion/transformation and reporting in Tableau. Tech stacks: Python, SQL, Ansible, Gitlab Pipelines, MySQL, PostgresSQL, Clickhouse, Tableau.
• Led design and development of a data quality monitoring system using Great Expectations, improving observability across the entire data platform. Tech Stacks: Python, Great Expectations, SQL, Ansible, Gitlab Pipelines, PostgresSQL, Clickhouse.
• Building a resource planning tool for managing and scheduling crew workloads and capacity optimization. Tech Stacks: Python, ReactJS, SQL, Ansible, Gitlab Pipelines, PostgresSQL.
• Created real-time dashboards and analytics reports for Talent & Culture department to track Return to Office (RTO) policy implementation and organizational compliance. Tech Stacks: Python, SQL, PostgresSQL, Tableau.
• Contributing to productionizing machine learning models for predicting workflow schedules and budgets in production environments. Tech Stacks: Python, SQL, PostgresSQL.
• Collaborating with data practitioners to form enterprise BI data strategy for the organization.
• Developed and maintained multiple production-grade applications for internal stakeholders. Tech Stack: Python, SQL, ReactJS, PostgresSQL, MongoDB.

Data Analytical Projects:

• Successfully migrated, redesigned, and maintained 8 data ETL pipelines from a Docker-compose solution to OpenShift, utilizing various technologies such as Airflow, DBT (dbt-expectations, dbt-elementary), Dynatrace, OpenShift, and Azure Pipelines.
• Virtual Mobile Location Center (vMLC REST API) to PostgresSQL:
  • Base Station Almanacs
  • Location Detail Records
• GSMA Device Database (REST API) to PostgresSQL
• Genuine Call Records (SFTP) to PostgresSQL
• Emergency Call Records (Flat Files) to Postgres SQL:
  • Fire Emergency
  • St Johns
  • Wellington Ambulance
  • NZ Police
• Designed a modern DataOps platform, encompassing Data Observability, Data Catalog, Data Linage, Data Management, Data Governance and Data Strategy.

Data Analytical Projects:

• Designed, developed, and maintained an automated user provisioning process, from ServiceNow/Payglobal to Azure Active Directory. Techniques: Python (Airflow), Snowflake, REST API.
• Designed and implemented an AWS Elastic Container Services based data platform for data analytical tools including Retool, N8N, and Airflow. Techniques: AWS CloudFormation, Python, and Docker.
• Designed, developed, and maintained multiple ETL pipelines by using Airflow, DBT (dbt-expectations, dbt-elementary):
  • Kronos (MySQL) to Snowflake
  • MySupport (MySQL) to Snowflake
  • Payglobal (MSSQL) to Snowflake
  • OneChart (Flatfiles in SFTP) to Snowflake
  • GreenTree/RiskManager/Business Central/Azure AD (REST API) to Snowflake
• Built 33 Power BI reports/dashboards for different analytical purposes. Techniques: Power BI, Snowflake.

Data Analytical Projects:

• Designed, developed, and maintained multiple internal data ingestion pipelines using Python (Apache Airflow)/PySpark:
  • Oracle to Redshift/Snowflake
  • SAP Hana to Redshift/Snowflake
  • Flat Files to Redshift/Snowflake
  • JSON/XML to Redshift/Snowflake
• Designed, developed and maintained an internal financial data analytical system. Techniques: Python, SAP Hana, and Tableau.
• Designed and developed an internal data warehousing replication system from Oracle Data Warehouse to AWS Redshift using Python, Spark.
• Worked with other colleagues to design and build a Kubernetes based MLOps platform. Techniques: CloudFormation, AWS EKS, Kubeflow.

Data Analytical Projects:

• Co-developed/Maintained three data analytical projects for three different government agencies and four different companies, including suspicious transaction analysis, financial data modeling, financial transaction continuous monitoring, and geo-location analysis. Techniques: Python, R, SQL and NodeJS.
• Co-developed/Maintained two cloud-based data analytical platforms (AWS and Azure) for two different government agencies. Among them, the platform built on Azure is particularly high-profile national-wide project. Techniques: NodeJS and SQL.

Data Analytical/Machine Learning Projects:

• Implemented/Co-implemented a number of new Reinforcement Learning systems (more than 100,000 lines of code) using Python, Theano, TensorFlow, PyTorch, Numpy, Scipy, Scikit-Learn, Pandas, Java, DeepLearning4J, and MATLAB.
• Implemented a new deep learning based ensemble system for image classification (~3,000 lines of code) using Python, TensorFlow, Keras, Numpy, Pandas and Scikit-Learn.

Conference Technical Support:

• Collaborated with colleagues for organizing two international conferences, i.e., the 31st Australasian Joint Conference on Artificial Intelligence (AI 2018) and 2019 IEEE Congress on Evolutionary Computation (CEC 2019).
• Developed and maintaining the two conference websites (more than 20,000 lines of code) using Bootstrap 3 with HTML5, CSS3.0, and JavaScript.
• Managing 4 social media channels of CEC 2019 (i.e., Facebook, Twitter, Linkedin, and Google+) for publicity of the conference.

Teaching Technical Support:

• Designed/Developed a CNN based Image Classification framework as a course project template (~1,000 lines of code) using Python, Keras, Scrapy, and Scikit-Learn.
• Designed/Developed an Android mobile image retriever APP as a course project template (~1,500 lines of code).
• Designed/Developed a J2EE WebService application as a course project template (~1,500 lines of code) using J2EE, Tomcat 6.0.

Data Analytical/Machine Learning Projects:

• Led/Coordinated a technical team (2 Ph.D. students and 2 master students) to develop a decentralized online network traffic monitoring system with real-time visualization (more than 150,000 lines of code) using J2SE, Java Agent Development Framework (JADE), Unity3D and MySQL, which remains in use at Unitec till today.
• Implemented/Co-implemented the financial forecast systems for stock and crude oil market trend prediction (~1,000 lines of code) using MATLAB.

System Technical Support:

• Deployed three real-time network traffic monitoring systems "NICTER", "NIRVANA" and "NIRVANA Rev. 2", in collaboration with three colleagues from National Institute of Communication and Technology (NICT), Japan.
• Maintained/ensured the smooth operation of the above three systems from 2012 to 2015 in the Cyber Security Research Center at Unitec, which later in 2015 contributed as one of the core projects for a $10.6 million (excl. GST) cyber-security research project funded by Ministry of Business, Innovation, and Employment (MBIE) and led by the University of Waikato.

Teaching Technical Support:

• Developed 6 Android mobile APPs for an outreach teaching project.