NAN XIAO
Sydney,NSW
Summary
Experienced Data Engineer with a proven track record in designing scalable data platforms and workflow management systems using AWS technologies. Skilled in improving data processing and visualization, leading to significant operational efficiencies and decision-making enhancements. Accomplished in deploying complex data pipelines and advanced reporting dashboards, contributing to a notable increase in data accuracy and business insight. Committed to driving innovation and efficiency in data engineering projects.
Overview
3
3
years of professional experience
Work History
Data Engineer
Rich Data Cooperation
10.2021 - Current
- Data Platform Development: Designed and implemented a comprehensive data platform from scratch, enabling efficient data ingestion, processing, and serving across various services and models.
- Cloud Solutions and work-flow Management: Utilized AWS technologies (Aurora, SQS, EBS, MWAA, EC2, S3) to develop a scalable, configurable, and flexible work-flow management platform. Spearheaded the design and implementation of distributed clustered large data set processing solutions, catering specifically to projects in collaboration with large banks.
- Data Pipeline Engineering: Employed MWAA to architect multiple complex python Airflow data pipelines, addressing diverse processes such as data sensors, data connectors, customer review, data anonymisation, and model performance reporting calculations.
- Project Support and Product Development: Actively supported project initiatives in tandem with product release cycles, ensuring seamless integration and deployment, such as release packaging vulnerability scan.
- Data Visualization and Reporting: Innovated the data visualization platform, significantly boosting report generation efficiency and dashboard reliability. Created and implemented specialized dashboards for model performance reporting and merchant lending.
- Backend development: Developed Java Spring micro services, Kubernetes deployments and automated deployment pipelines.
Software Engineer
Monash Action IT Lab
03.2021 - 10.2021
- Worked with different NGOs and provided IT consultations and solutions.
- Communicated with clients to identify and discuss their requirements.
- Containerized Solferino Academy’s existing WordPress application.
- Planed, designed and implemented the CI/CD pipeline for automatic testing and continuous deployment, cloud server configuration.
- Documented key points of the whole project, including details of docker services architecture, CI/CD work flow, server configuration.
Education
Master of Science - Information Techonology
Monash University
Melbourne, VIC08.2021
Bachelor of Science - Clinical Pharmacy & Science in Economics
Jilin University
Jilin, University07.2019
Skills
- Python
- SQL Programming
- Java
- Airflow
- Spark Framework
- Docker & Kubernetes
- AWS
- Data Modeling
- Big Data Processing
- Data visualisation
Reference
Available upon request
Timeline
Data Engineer
Rich Data Cooperation
10.2021 - Current
Software Engineer
Monash Action IT Lab
03.2021 - 10.2021
Master of Science - Information Techonology
Monash University
Bachelor of Science - Clinical Pharmacy & Science in Economics
Jilin University
Similar Profiles
Gloria WeiGloria Wei
Site Reliability Engineer(System Administrator) at Melissa Data CooperationSite Reliability Engineer(System Administrator) at Melissa Data Cooperation
Claire KellyClaire Kelly
Law Clerk at Rich, Rich, Cooksey, and Chappell, P.C.Law Clerk at Rich, Rich, Cooksey, and Chappell, P.C.
Ron MausolfRon Mausolf
Packaging Operator at RICH PRODUCTS CORPORATIONPackaging Operator at RICH PRODUCTS CORPORATION
Parshant SharmaParshant Sharma
Customer Service Representative at Viva EnergyCustomer Service Representative at Viva Energy
Amy MalekAmy Malek
Volunteer - Art Assistant at Sydney FestivalVolunteer - Art Assistant at Sydney Festival