Summary
Overview
Work History
Education
Skills
Certification
Timeline
Generic

Deepali Lamkane

Kellyville,Australia

Summary

Experienced Data Engineer with 8 years of experience in data engineering, data warehousing, data modelling, building ETL, ELT pipelines, data migration, business intelligence, building reporting solutions, analysis, testing, and deployment in on prem and cloud platforms. Hands on experience working with Azure services like azure data factory, azure SQL DB, azure synapse analytics, azure functions, databricks, data lake, key vault, logic apps.

Overview

9
9
years of professional experience
1
1
Certification

Work History

SENIOR DATA ENGINEER

NBN
11.2021 - Current
  • Designed and delivered data warehouse modernisation project and migrated data, reports from legacy data warehouse onto Azure cloud and built ELT pipelines, dimensional models in azure data lake and Azure Synapse Analytics using Azure data factory, databricks and SQL, Python, PySpark
  • Worked on schema designing, built fact and dimensions in star and snowflake schema in gold layer, implemented SCD and built ETL, ELT process to load data from SQL server, Data Lake, Oracle into data lake using azure data factory, cleansed, and transformed data using databricks and generated aggregated data in gold layer and generated reporting tables and views in synapse analytics
  • Scheduled, orchestrated tasks, business processes, workflows using logic apps
  • Driving the development of data platform by building frameworks and developing pipelines on the ADF framework, worked on azure monitoring
  • Built azure devOps CI/CD pipelines for deploying adf, etl, azure synapse code changes and documentation
  • Created and deployed azure resources using Terraform and ARM template in Azure environment.

Data Engineer

IBM Envizi
02.2021 - 11.2021
  • Involved in data engineering project in sustainability domain where extracted data from SQL server and loaded it in lakehouse in azure data lake, build dimensional model using Pyspark, SparkSQL
  • Worked on loading json and csv files from Azure blob storage and into ADLS gen2
  • Orchestrated data acquisition, transformation, utilizing Azure Databricks, managing over 10 TB of data within Azure Data Factory, and implemented security measures to safeguard the confidentiality, integrity
  • Acted as SME in delta-live-tables (modern data ingestion platform for streaming and batch pipelines), CDC use cases, and Spark structured streaming
  • Used logic apps to send email notification when new reports are deployed
  • Worked on weather stations project where loaded weather data using API in Json format and then transform that data and created reporting table and reports
  • Generated several feeding reports for meters, locations, monthly, weekly data summary, cost rates report for clients using Power BI Optimized queries performance by applying partitioning, bucketing, and spark optimization strategies
  • Participated in agile Environment, daily stand-ups and sprint and scrum meetings.

Data Engineer

BizCover
07.2020 - 01.2021
  • Developed data processing solutions to transform and load data into data warehouses and data marts and created dimensional model for business processes like broker & customer policies, insurer payments in Azure SQL DB
  • Built adf pipelines, activities for data ingestion, data cleansing, generating invoice and receipt, processing payments and fund drawings and built reports in Power BI
  • Used Azure data factory to load data from transactional system into Azure SQL DB
  • Worked in schema design, data modelling and development of new and existing workflows
  • Built stored procs for data ingestion, data cleansing, generating invoice and receipts, processing payments and fund drawings and building reports in Azure SQL using Power BI
  • Built star and snowflake schema model for insurer, broker policies, fund drawing models
  • Used GitHub for change management and Jenkins for automation deployment.

Data Engineer

Tata Consultancy Services
05.2015 - 11.2019
  • Worked on data migration and report migration project where migrated SQL server database and reports onto new Azure SQL database using SQL framework and SSIS packages and created end to end ETL pipeline for incremental data load
  • Designed, developed, and supported new and existing ETL processes by employing industry standards and best practices to enhance data loading process using SQL framework
  • Created SSIS packages for data extraction from source files and did transformations like multicast, derived column, data conversion, conditional split, etc to validate and move data into Azure SQL DB
  • Experience in schema design, query tuning, writing stored proc to transform and load data and establish and enforcing security auditing mechanism
  • Setup monitoring and alerts for pipeline failure, automated pipelines, reports refresh using SQL agent jobs
  • Created paginated, parameterised, drill through reports in SSRS and Power BI.

Education

Bachelor of Computer Science -

University of Pune
India
01.2015

Skills

  • Azure data factory
  • Azure synapse analytics
  • Azure data lake
  • Databricks
  • SQL
  • Python Programming
  • Pyspark Programming
  • Apache spark
  • Power BI
  • Azure KeyVault
  • Erwin data modeler
  • SQL Server
  • Cosmos DB
  • Azure DevOps

Certification

  • Microsoft Certified Solutions Expert in Data Management and Analytics, MS0989715734
  • Oracle Autonomous Cloud Specialist, OC2001903

Timeline

SENIOR DATA ENGINEER

NBN
11.2021 - Current

Data Engineer

IBM Envizi
02.2021 - 11.2021

Data Engineer

BizCover
07.2020 - 01.2021

Data Engineer

Tata Consultancy Services
05.2015 - 11.2019

Bachelor of Computer Science -

University of Pune

Similar Profiles

CREATE PROFILE
Deepali Lamkane