Revati Deshmukh
Data Engineer
Melbourne,VIC
Summary
Senior Data Engineer leading Azure and Snowflake data platform work across 2–3 platforms, delivering reporting datasets for downstream users in ingestion, transformation, and reporting layers. Delivered CI/CD-enabled deployments monthly, incremental processing with dynamic tables, and SQL redesigns that improve responsiveness and reduce manual effort by ~60%. Partners with IT and business teams to translate regulator-facing reporting needs into dependable production pipelines.
Overview
8
8
years of professional experience
Work History
Data Engineer
Cleanaway
03.2023 - Current
- Developed and deployed consumption emissions fuel and carbon reporting solutions through Snowflake and Azure technologies, partnering with IT teams to create comprehensive reporting frameworks and ensure adherence to regulatory standards.
- Developed and managed scalable ETL/ELT data pipelines leveraging Azure Databricks, Azure Data Factory, and Snowflake to ingest, transform, and ready data from various enterprise sources for reporting and analytics.
- Implemented data mirroring to synchronize operational data from PostgreSQL with Snowflake and developed DBT transformations to produce curated datasets for effective operational reporting.
- Configured Snowflake dynamic tables to facilitate incremental data processing from Databricks and uphold up-to-date reporting-ready datasets.
- Refined snowflake reporting performance via optimization of data loading and transformation workflows, improving both query efficiency and report responsiveness.
- Constructed robust data validation framework with Python and SQL, decreasing data errors by 30% and improving accuracy of financial transactions.
- Engineered automated solutions for Coles ETL process and established CI/CD pipelines using Azure DevOps, reducing need for manual interaction and improving deployment streamlining across environments.
- Developed Power BI dashboards to monitor data quality issues and highlight data discrepancies, enhancing visibility into dataset accuracy and reliability.
Data Engineer
Synechron Technologies
09.2020 - 12.2022
- Transformed on-premises PL/SQL ETL processes to cloud infrastructure utilizing Databricks, Azure Data Factory, Azure Data Lake Storage, and Azure Synapse Analytics, enhancing scalability and performance.
- Transformed and optimized queries into Spark actions and transformations, decreasing runtime and lowering cloud expenses by leveraging RDDs and DataFrames.
- Strengthened system stability and performance by diagnosing and rectifying database issues.
- Configured queries into Spark actions and transformations, optimizing runtime and minimizing cloud expenditures.
- Secured data integrity and quality through comprehensive QA processes assessing data across source staging and destination layers post-job execution.
- Strengthened system stability and performance by diagnosing and addressing database issues through efficient troubleshooting.
- Engaged with business analysts to gather and articulate requirements, guaranteeing development adheres to business goals.
Full Stack Developer
Tata Consultancy Services
09.2018 - 09.2020
- Managed code maintenance and troubleshooting, resolving reported issues and responding to customer queries to ensure application functionality
- Automated reporting procedures to retrieve data from source tables, produce reports, and deliver through automated email, enhancing reporting efficiency
- Enhanced batch job procedures optimizing performance while reducing runtime for efficient operations
- Developed and maintained batch jobs according to client requirements, delivering consistent and accurate data processing
- Automated reporting processes to extract data from source tables, generate reports, and distribute via automated email, streamlining reporting workflow
Education
Bachelor of Electronics & Telecommunication -
Nagpur University
Nagpur, India05-2018
Skills
SQL
Snowflake
Azure Databricks
Azure DevOps
Microsoft Power BI
MS SQL Server
ETL pipeline development
Data validation frameworks
SQL query optimization
Pipeline troubleshooting
Batch job automation
Reporting solution development
Data reconciliation
Requirements documentation
Data transformation
Cross-functional collaboration
dbt
Data modeling
Data warehousing
Accomplishments
- Data Validation Improvement: Reduced data errors by 30% with an automated validation in Python and SQL workflows and issue resolution.
- ETL Performance Enhancement: Improved ETL runtime efficiency by 20% through optimized framework.
- Data Ingestion Innovation: Streamlined data processing by 25% with a new data SharePoint. increasing system performance by 40%.
- ETL Automation Success: Automated Coles ETL pipeline, reducing manual effort and ingestion pipeline from SharePoint.
Timeline
Data Engineer
Cleanaway
03.2023 - Current
Data Engineer
Synechron Technologies
09.2020 - 12.2022
Full Stack Developer
Tata Consultancy Services
09.2018 - 09.2020
Bachelor of Electronics & Telecommunication -
Nagpur University