Summary
Overview
Work History
Skills
Certification
Timeline
Generic

Bhavani

Mount Cottrell

Summary

Senior Azure Data Engineer with over 5+ years of proven experience in designing, developing, and deploying scalable data solutions that drive business impact across diverse cloud and hybrid infrastructures. Proficient in designing and building scalable, enterprise-grade data pipelines using Azure Data Factory (ADF), Azure Databricks (PySpark/Scala), and Azure Synapse Analytics. Adept at supporting batch and real-time data ingestion, transformation, and orchestration to deliver robust data solutions. Expert in advanced Power BI solutions, including composite models, incremental refresh, and deployment pipelines. Utilized Azure Data Lake Gen2, Dataverse, and Azure Analysis Services to ensure scalable and governed BI delivery. Designed and deployed high-performance analytics using Power BI Direct Lake, enabling large-scale data analysis with significantly reduced data duplication and improved efficiency. Hands-on experience with Azure Databricks, including Delta Live Tables for automated pipeline orchestration and MLflow for comprehensive experiment tracking in Python environments. Designed modern data lake house architectures leveraging Azure Data Lake Storage Gen2, Delta Lake, and Spark SQL. Proficient in managing diverse data formats including Parquet, Avro, JSON, and CSV for robust data solutions. Successfully migrated on-premises data solutions (SQL Server, Oracle) to Azure, delivering high-performance, secure, and cost-effective cloud-based systems. Proficient in designing dimensional data models with Star and Snowflake schemas, complemented by deep experience in the Microsoft SQL Server BI stack (SSIS, SSAS, SSRS) and strong T-SQL scripting capabilities. Proven expertise in Power BI, building interactive dashboards with advanced DAX and real-time capabilities. Seamlessly integrated with Azure SQL, Synapse, and Kafka streams to deliver valuable insights across finance, inventory, operations, and high-volume data. Experienced in handling high-volume, distributed data workloads using a robust Big Data stack, including Hadoop, Hive, HDFS, Sqoop, Spark, Kafka, HBase, Oozie, and Zookeeper. Implemented robust Data Quality Checks and Data Validation Frameworks using Python libraries like Great Expectations and Pandas Profiling. Integrated seamlessly with ADF and Databricks for early detection of data anomalies and enhanced data governance. Expert in Azure Data Factory V2 (ADF V2), specializing in developing flexible parameterized pipelines, configuring trigger-based executions, and orchestrating complex data workflows across heterogeneous systems. Applied DevOps practices by automating CI/CD pipelines using Azure DevOps and GitHub Actions for efficient building, testing, and deployment of data pipelines and Azure services. Expertise in utilizing Azure PaaS components (Logic Apps, Azure Functions, Event Hubs, Azure Monitor, Azure App Services, Azure Key Vault) to deliver scalable cloud-native data solutions. Developed Python scripts for data quality validations, profiling, and automation across both cloud and on-premise systems, significantly enhancing data reliability and reducing manual intervention. Experienced in Agile/Scrum methodologies, driving collaborative, sprint-based delivery with a commitment to documentation and iterative improvements throughout the SDLC. Implemented enterprise-grade metadata management and governance frameworks utilizing Azure Purview and Microsoft Fabric Data Governance. Enabled robust lineage tracking, data classification, access control, and compliance for comprehensive data oversight. Proficient in implementing Azure security measures such as RBAC, Managed Identities, and Private Endpoints. Secured data access and ensured compliance across Azure Data Factory, Databricks, and various storage layers. Proficient in building comprehensive observability solutions using Azure Monitor, Log Analytics, and custom metric alerts. Enabled efficient monitoring for pipeline health, SLA adherence, and data latency detection. Hands-on experience in automating infrastructure provisioning using Bicep templates and Azure DevOps, ensuring efficient, scalable, and version-controlled deployments for data engineering.

Overview

9
9
years of professional experience
1
1
Certification

Work History

Senior Azure Data Engineer

Wipro Technologies
05.2024 - Current
  • Led the design and development of an Azure-based data analytics platform at Deutsche Bank, delivering capabilities for regulatory reporting, liquidity risk management, and real-time financial insights. This robust solution encompassed integrated BI dashboards, governed data pipelines, and real-time data streams.
  • Built and deployed scalable ELT pipelines utilizing Azure Data Factory, Azure Databricks (PySpark/Scala), and Azure Synapse. Successfully integrated diverse structured and semi-structured financial data from various banking systems.
  • Developed modern Lakehouse architectures using Azure Data Lake Gen2 and Delta Lake, applying bronze-silver-gold data zones and schema evolution best practices for efficient and governed regulated data workflows.
  • Implemented Delta Live Tables (DLT) to facilitate real-time data ingestion and applied Change Data Capture (CDC) logic for efficient incremental loads from transaction systems.
  • Developed and deployed robust ADF V2 pipelines with a focus on flexibility, utilizing dynamic parameters, Lookup, For Each, Set Variable, until, and Execute Pipeline activities for effective and scalable data workflow orchestration.
  • Implemented real-time data streaming solutions using Azure Event Hubs and Azure Stream Analytics. Successfully processed high volumes of financial transactions for immediate insights into fraud and trade surveillance.
  • Developed interactive Power BI dashboards incorporating advanced DAX, custom measures, and bookmarks to effectively visualize critical KPIs, including liquidity ratios, exposure trends, and regulatory thresholds.
  • Implemented Power BI Direct Lake, incremental refresh, and composite models to enable high-performance analytics on large volumes of historical data.
  • Developed and deployed high-performance semantic models in Azure Analysis Services, facilitating deep analytical insights through drill-through and drill-down functionality across finance, risk, and operations data.
  • Integrated Power BI with multiple Azure data sources like Azure SQL, Synapse, Dataverse, and Data Lake Gen2, empowering users with self-service access to clean and prepared data.
  • Implemented robust Power BI deployment pipelines within the Power BI Service, enabling automated and validated transitions of reports across Dev, UAT, and Production stages.
  • Applied Row-Level Security (RLS) and Azure workspace permissions to ensure secure, role-based access to sensitive dashboards and datasets.
  • Implemented comprehensive data profiling and validation solutions leveraging Python, Pandas profiling, and Great Expectations. Integrated within ADF and Databricks to ensure early identification of data anomalies and maintain high data integrity.
  • Environment: ADF, Azure Data Lake Gen2, Azure Databricks (PySpark), Azure SQL, Synapse Analytics, Power BI, DAX, Python, Pandas, Azure Batch, ADF Mapping Data Flows, Git, Azure DevOps, SQL Server, Oracle, REST API, PCI DSS, SOX Compliance.

Azure Data Engineer

Persistent Systems
01.2022 - 05.2024
  • Built and deployed a scalable data analytics and reporting platform for Fiserv, streamlining financial transaction processing, customer behavior analytics, and regulatory compliance. The solution integrated multiple legacy data sources into a centralized Azure-based solution for real-time insights and BI reporting.
  • Designed and implemented end-to-end ELT pipelines leveraging Azure Data Factory (ADF) and Azure Data Lake Storage Gen2 to efficiently ingest, transform, and store diverse structured and semi-structured financial data.
  • Developed and optimized Delta Lake tables via Azure Databricks (PySpark), delivering ACID compliance and superior performance essential for critical downstream analytics.
  • Developed scalable ADF pipelines leveraging parameterization for efficient and dynamic data ingestion from a wide array of sources (SQL Server, Oracle, flat files, and REST APIs).
  • Designed and published interactive Power BI dashboards for comprehensive financial portfolio analysis, fraud detection metrics, customer transaction insights, and key operational KPIs.
  • Developed complex DAX measures and composite models, seamlessly integrating data from Azure SQL, Azure Databricks, and Azure Synapse Analytics to deliver comprehensive, multi-source insights.
  • Streamlined Power BI reporting with the implementation of incremental refresh, row-level security (RLS), and efficient deployment pipelines, ensuring scalable and secure data delivery.
  • Implemented comprehensive data quality frameworks using Python scripts, with Pandas Profiling and Great Expectations for automated validation and anomaly detection.
  • Implemented proactive data quality monitoring within ADF by integrating validation logic that triggers alerts for schema drift, null bursts, and SLA violations, improving data integrity and operational oversight.
  • Configured Azure Monitor and Log Analytics to provide comprehensive tracking of pipeline health, latency, and detailed failure diagnostics.
  • Implemented stringent security protocols such as RBAC, Private Endpoints, and Managed Identities across ADF, Databricks, and storage environments, successfully meeting SOX and PCI-DSS compliance standards.
  • Implemented Azure Purview for comprehensive metadata cataloging, data lineage tracking, and classification of sensitive financial data, enhancing data governance and compliance.
  • Automated infrastructure provisioning using Bicep templates and deployed ADF/Databricks components through Azure DevOps CI/CD pipelines, ensuring consistent and efficient environment setup.
  • Environment: ADF, Azure Data Lake Gen2, Azure Databricks (PySpark), Azure SQL, Synapse Analytics, Power BI, DAX, Python, Pandas, Great Expectations, Azure Monitor, Bicep, Azure DevOps, Oracle, SQL Server, REST APIs, SOX, PCI DSS, Azure Purview.

Azure Data Engineer

IBN Technologies
09.2016 - 07.2018
  • Built a centralized data foundation for retail lending analytics at HDFC Bank, integrating disparate data sources to enable robust loan disbursal tracking, repayment monitoring, and compliance reporting. This project also initiated the groundwork for their Azure cloud migration.
  • Built and managed data ingestion pipelines using Azure Data Factory (ADF) to efficiently extract customer, loan, and EMI data from SQL Server, Oracle, and flat file systems.
  • Assisted in data transformation processes, leveraging Mapping Data Flows and Lookup activities within ADF to cleanse, enrich, and validate raw datasets.
  • Managed data ingestion of structured and semi-structured datasets into Azure Blob Storage and Azure SQL Database, establishing them as crucial interim storage for analytics.
  • Developed and tested T-SQL scripts for comprehensive data profiling, anomaly detection, and business rule validation.
  • Collaborated with senior engineers on the configuration of datasets, linked services, and triggers in Azure Data Factory.
  • Contributed to developing Power BI reports for internal stakeholders, including loan performance dashboards and delinquency trends.
  • Enabled early exposure to Azure Data Lake Storage Gen2 (ADLS Gen2) during its pilot phase, with a specific focus on testing its hierarchical namespace and access control features.
  • Supported data transparency through the creation of data dictionaries, source-to-target mappings, and clear transformation logic notes.
  • Executed basic data quality checks through both manual inspection and scripted validations, ensuring data integrity before pushing to reporting layers.
  • Collaborated with QA teams to conduct unit testing of data pipelines and reconcile record-level mismatches, ensuring data accuracy and integrity.
  • Developed understanding and practical experience with Azure Key Vault for securing sensitive credentials in linked services.
  • Developed expertise in Git workflows to manage version control for ADF JSON definitions and SQL scripts, supporting robust development practices.
  • Contributed to pipeline stability by assisting in troubleshooting failed runs, performing error logging, and coordinating RCA efforts with senior engineers.
  • Environment: Azure Data Factory V2, Azure Blob Storage, Azure SQL Database, T-SQL, SQL Server, Oracle, Power BI (Basic), Excel, CSV, JSON, Git, Azure Key Vault (Intro), Agile/Scrum.

Skills

  • Cloud & Data: ADF, ADB, Azure Synapse, Data Lake Gen2, Event Hubs
  • Big Data: Spark, Hadoop, Hive, Kafka, HDFS, HBase, Oozie, Zookeeper
  • Programming: Python (Pandas, MLflow, Great Expectations), T-SQL
  • BI & Reporting: Power BI (DAX, Direct Lake, AAS, Dataverse), SSRS, SSAS
  • ETL / Data Pipelines: ADF V2, Delta Live Tables, SSIS, Python ETL Scripts
  • DevOps & CI/CD: Azure DevOps, GitHub Actions, Git, Bicep, ARM Templates
  • Monitoring: Azure Monitor, Log Analytics, Alerts & Metrics
  • Security: RBAC, Managed Identities, Private Endpoints, Azure Key Vault
  • Methodologies: Agile, Scrum, Documentation, Collaboration

Certification

DP 700

Timeline

Senior Azure Data Engineer

Wipro Technologies
05.2024 - Current

Azure Data Engineer

Persistent Systems
01.2022 - 05.2024

Azure Data Engineer

IBN Technologies
09.2016 - 07.2018

Similar Profiles

CREATE PROFILE
Bhavani