Summary
Overview
Work History
Education
Skills
Timeline
Generic

Harry Lynch

Sydney,Australia

Summary

Senior Data Engineer with expertise in cloud platforms and scalable data solutions. Skilled in Python, Apache Spark, dbt, and Kubernetes. Proven ability to ensure data quality and governance while mentoring teams and driving analytics and machine learning initiatives. Experienced in aligning technical solutions with business objectives, enhancing data availability and accuracy.

Overview

5
5
years of professional experience

Work History

Senior Data Engineer

Gumtree
Sydney, NSW
10.2024 - Current
  • Engineered data ingestion from an API into a centralised warehouse for unified analytics.
  • Developed and maintained data pipelines in Python, ensuring efficient ETL processes.
  • Redesigned data models to maximise efficiency and significantly reduce costs.
  • Designed a comprehensive alerting framework to monitor data reliability and enhance incident response.
  • Mentored junior data engineers to promote high-quality code and adherence to best practices.
  • Maintained operational health of systems through proactive monitoring and debugging efforts.
  • Collaborated with cross-functional teams to gather requirements for seamless backend integration.
  • Partnered with SRE team to manage Kubernetes clusters, ensuring system reliability.

Software Engineer

Quantium
Eveleigh, Australia
09.2022 - 09.2024
  • Engineered complex data pipelines with dbt and PySpark on Google Dataproc.
  • Facilitated ingestion, transformation, and loading processes into BigQuery's data warehouse.
  • Oversaw Spark cluster pipeline management within a Kubernetes environment using ArgoCD.
  • Spearheaded Vertex AI integration to automate machine learning workflows, enhancing predictive analytics.
  • Orchestrated ETL pipelines utilizing GCP's BigQuery, Kubeflow, Cloud Run, Pub/Sub, and Confluent, achieving 23% cost reduction.
  • Implemented data testing and quality checks with dbt and Elementary for accuracy across CDM.
  • Collaborated with cross-functional teams to align data solutions with business needs and enhance performance.

Associate Software Engineer

Cognizant Servian
Sydney, Australia
10.2021 - 09.2022
  • Executed extensive data migrations on AWS for seamless integration with Snowflake.
  • Architected sophisticated integration tools to harmonise data from APIs, databases, and standalone files.
  • Designed and developed innovative data extraction processes using Talend and Python.
  • Maintained and troubleshot ETL jobs, ensuring efficient data transitions to Snowflake.
  • Enhanced auditing databases to broaden application support, including API functionalities.

Associate Data Collection Specialist

Appen
Chatswood, Australia
11.2020 - 10.2021
  • Developed metrics for data collection and quality assurance, enhancing efficiency tracking.
  • Extracted raw data from diverse sources, creating a performance dashboard for real-time monitoring.
  • Streamlined data refreshing process, saving 1.5 hours daily per team member.
  • Created Python script templates for rapid generation of internal HTML data collection scripts.
  • Increased script generation speed by 40% while ensuring accuracy and optimizing resources.
  • Identified and corrected latent data precision issues to meet high-velocity operational demands.
  • Collaborated with Engineering Team to enhance internal data collection and quality assurance systems.
  • Partnered with Engineering and Computer Vision Teams to establish workflows for PII identification and obfuscation.

Education

Master's degree - Business Analytics and Big Data

University of Sydney
Sydney, NSW
10-2019

Skills

  • Data extraction and transformation
  • Python programming
  • Data modeling techniques
  • SQL proficiency
  • CI/CD practices
  • Google Cloud Platform expertise
  • Problem-solving skills
  • DevOps methodologies

Timeline

Senior Data Engineer

Gumtree
10.2024 - Current

Software Engineer

Quantium
09.2022 - 09.2024

Associate Software Engineer

Cognizant Servian
10.2021 - 09.2022

Associate Data Collection Specialist

Appen
11.2020 - 10.2021

Master's degree - Business Analytics and Big Data

University of Sydney

Similar Profiles

CREATE PROFILE
Harry Lynch