Summary
Overview
Work History
Education
Skills
Websites
Certification
Personal Information
Awardsrecognition
Timeline
Generic
Bala Aggala

Bala Aggala

Granville,Australia

Summary

Data Engineering Architect with 15+ years of experience in database and data warehouse design, development, and architecture, specializing in Delta Lake house implementations and AI-driven solutions. Proficient in designing scalable architectures for data engineering pipelines using Azure Data Factory, Synapse Analytics, and Databricks, with hands-on expertise in ETL workflows, data governance, and real-time analytics. Skilled in AI/ML model development, leveraging Databricks for predictive analytics and cross-sell/up-sell product recommendations. Certified Azure Data Engineer with a strong background in Microsoft BI (SSIS, SSRS, SSAS), Power BI, and Salesforce API integrations, delivering data-driven solutions that enhance decision-making and business growth. Knowledgeable Data Specialist with robust background in data architecture and pipeline development. Proven ability to streamline data processes and enhance data integrity through innovative solutions. Demonstrates advanced proficiency in SQL and Python, leveraging these skills to support cross-functional teams and drive data-driven decision-making.

Overview

14
14
years of professional experience
1
1
Certification

Work History

Data Architect

Freelancer
09.2024 - 12.2024

Bankers Crossell/Upsell Chatbot Project

  • Designed a robust data architecture to integrate and unpack diverse data sources, enabling seamless transformation and analytics.
  • Implemented a compound AI chatbot model for cross-sell and up-sell product recommendations, driving business growth through intelligent matching algorithms.
  • Extensively utilized Databricks for data transformation, AI/ML model training, and orchestration of personalized push notifications.
  • Used Teams webhook API for implementing Push notifications from databricks platform.
  • Created more than 20 databricks sql fucntions for the implementation of Compund AI model

Data Engineer

NSW Department of Education
07.2022 - Current
  • Designed and implemented Delta Lake architecture (bronze, silver, gold layers) on Azure Synapse
  • Migrated large datasets from SQL Server to Azure Data Lake in Parquet format
  • Automated data workflows with PySpark, Databricks, and Synapse Pipelines
  • Established a robust data validation framework ensuring governance compliance
  • Integrated Salesforce data into Azure Synapse using Bulk API and AWS Lambda
  • Designed cross-cloud solutions for AWS PostgreSQL to Azure Synapse migrations
  • Implemented Value added Keyvault mechanism which handled data platform from cyber seurity compliance rules.

Senior Data Engineer / BI Specialist

QBE Insurance
05.2022 - 07.2022
  • Developed ETL processes using SSIS, Azure Data Factory, and T-SQL
  • Migrated on-prem databases to Azure Managed SQL Instances
  • Implemented Azure Synapse Analytics for improved data governance
  • Established End to End Data quality Frame work using Business and Technical rules for Insurance policy domain.
  • Collaborated with cross-functional teams to define requirements and develop end-to-end solutions for complex data engineering projects.

Data Engineer / BI Specialist

NSW Health
07.2020 - 05.2022
  • Migrated legacy systems to Azure Synapse Analytics, improving performance and scalability
  • Built end-to-end ETL pipelines using Databricks and PySpark
  • Created dynamic Power BI dashboards integrating Azure Data Factory and Synapse
  • Designed a new data mart in Azure SQL datawarehouse for Mental health data platform.

BI Report Developer

Australian Institute of Company Directors
11.2018 - 06.2020
  • Led enterprise reporting projects using Power BI and Azure Data Factory
  • Migrated on-premise SQL Server and SSIS packages to Azure
  • Developed DAX measures and time-based functions for advanced reporting
  • Desgined and Delivered more than 50 Power BI Reports across Enterprise with different domains

Database Developer

Tata Consultancy Services
03.2011 - 12.2016

· Developed Cube for Operational Deposit decision reporting with time based Measures.

· Developed Complex Calculations in PLSQL like standard deviation using analytical functions

· Worked on creating tables, synonyms and views in the External Portal.

· Developed Procedures, views, packages and collections to process client data.

Education

Master of Computer Applications -

Andhra University
India
05-2010

Skills

  • SQL Server, Oracle, AWS PostgreSQL, Azure SQL DB
  • SQL, T-SQL, PL/SQL
  • DAX, MDX
  • SSIS, Azure Data Factory, Informatica
  • SSRS, Tableau, Power BI
  • Azure Synapse Analytics, Databricks
  • LLM Models, Azure Open AI
  • AWS Lambda
  • SAP BODS
  • Python
  • C
  • GCP
  • MongoDB, Azure Cosmos DB
  • Azure DevOps, Git, SVN

Certification

  • Hacker Rank : 5-Star SQL Badge
  • Microsoft Certified: Azure Data Engineer Associate
  • Databricks: Databricks Data Engineer Associate

Personal Information

  • Title: Senior Data Engineer | Data platform Specialist
  • Visa Status: Australia Citizen

Awardsrecognition

  • National Level 2nd Prize: Bank of America Tech Fest (White Paper on Real-Time Distributed Computation).
  • Technical Excellence Award: TATA Consultancy Services.

Timeline

Data Architect

Freelancer
09.2024 - 12.2024

Data Engineer

NSW Department of Education
07.2022 - Current

Senior Data Engineer / BI Specialist

QBE Insurance
05.2022 - 07.2022

Data Engineer / BI Specialist

NSW Health
07.2020 - 05.2022

BI Report Developer

Australian Institute of Company Directors
11.2018 - 06.2020

Database Developer

Tata Consultancy Services
03.2011 - 12.2016
  • Hacker Rank : 5-Star SQL Badge
  • Microsoft Certified: Azure Data Engineer Associate
  • Databricks: Databricks Data Engineer Associate

Master of Computer Applications -

Andhra University
Bala Aggala