Sumit Rauniyar
Melbourne,Victoria
Summary
Seasoned Data Engineer with proficiency in architecting and implementing end to end data solutions across wide spectrum of business domain with expertise revolving around cloud. Ready to bring 9+ years of progressive experience and take on a challenging new role with growth potential.
Overview
10
10
years of professional experience
15
15
Certifications
Work History
Manager Projects - Senior Data Engineer
Cognizant Servian
Australia
08.2023 - Current
- Working on Data Warehouse and Reporting Modernization by creating a deltalake with medallion architecture.
- Implementing data quality and data transformation measures based on Databricks best practices.
- Implementing Databricks workflows to drive metadata based jobs that use parameters to drive required behavior.
- Implementing Data Governance through Unity Catalog.
- Assisted solution providers with definition and implementation of technical and business strategies.
- Created conceptual, logical and physical data models for use in different business areas.
- Worked on implementing Snowflake data warehouse with medalion architecture.
Principal Data Engineer
Neudesic Technologies
India
08.2021 - 05.2023
- Designed an Enterprise Data Lake Platform for Advanced Analytics using Azure Synapse Analytics, Azure Data Factory, Azure Purview, Azure DevOps, Azure Databricks (Pyspark), Azure SQL Data Warehouse.
- Designed real time solution to stream OBD2 data from connected cars to load operational database and real time analytics using Kafka and Spark Structured Streaming.
- Designed Data Quality Framework for Data Pipelines.
- Designed Master Data Management using Profisee to manage golden records for the enterprise.
- Designed and Built Datalake/Deltalake with Medallion Architecture with principle of progressively increasing the Data Quality and optimization for better Query Performance
- Efficiently Handled PII Data for customers across the delta lake platform and Data Warehouse Layer
- Worked on NoSQL Database such as CosmosDB for querying Documents with low latency
- Experience in Designing and Building CICD Pipelines using Azure DevOps and Azure Repos.
- Designed Project level Git Branching Strategy and Promotion of Code to higher environments.
- Implemented Data Governance , Data Lineage and Data Security using Azure Purview, Azure AD and Azure Access Control.
- Architected Fault tolerant, Metadata based ELT Data Pipelines using Azure Data Factory.
- Designed Data Warehousing Strategy for Data Storage and DataWarehouse Modeling.
Technical Project Manager
Bridgei2i Analytics Solutions
India
04.2020 - 08.2021
- Worked on designing and implementing end-to-end Infra and Data solutions across cloud platforms like AWS and Azure Technologies including AKS, ADF, SQL Data Warehouse, Databricks, Glue, ECS (Fargate), Redshift, etc
- Built a layer for Data Quality, Data Governance, Data Lineage across Data Solutions
- Responsible for leading teams of various sizes depending on the engagements.
- Responsible for Budgeting and part of pre sales and RFP team.
- Responsible for IPSEC compliant Architecture Designing (OWASP, SANS25,HIPAA)
- Responsible for Handling Security, CICD, Networking for Client Projects.
- Deploying Kubernetes and VM Version of in-house web app product to the client environment using Helm Charts.
- Devised project strategies and scopes in accordance with established methodologies from conceptualizing ideas to implementing plans and optimizing operations.
- Managed submittals and processing of Requests for Information between general contractor, owner and owner's consultants.
- Worked with end users, technical professionals, administrators and testers to review requirements, conduct tests and implement solutions.
Senior Consultant II - Data Engineering
Neudesic Technologies
India
06.2019 - 04.2020
- Working on Hive to BigQuery Workload Migration
- Implementing a Server less Data Processing Pipeline
- Devops Management for applications.
- Participated in training and managing new hires to bring new team members up to speed.
- Mitigated project conflicts by proactively seeking acceptable outcomes acceptable to stakeholders.
- Worked on Azure Cloud Platform to create Pipelines in ADF that moves the data from db2 to Azure SQL Warehouse.
Consultant - Data Engineering
Core Compete Pvt Ltd.
India
07.2018 - 06.2019
- Get different fixed width files from client in compute engine by a ssh pull to Client server
- Format and transform those files using python modules and then push it to GCS Bucket
- Create a Datalake, Datastage, Datamart layer in BigQuery by using Client API’s transformation
- Create a Semantic Layer on top of the Datamart
- Orchestrate the whole flow through Airflow.
- Worked on Azure Data Factory to collect the data from API sources and move them to Azure Data Lake for consumption by Power BI Report.
Senior Data Engineer
Optum (UHG)
India
05.2017 - 07.2018
- Data mining relevant data from various data sources to gather information related to the requirement using tool such as Sqoop
- Stage the data at Big Data Platform and perform translations using Hadoop tools such as Hive,Spark,HBase.
- Create visualization as end product with tableau.
Software Engineer
Tech Mahindra
India
05.2014 - 05.2017
- Working with structured and unstructured data in MapR Hadoop Distribution across Hive , HDFS and HBase
- Scheduling jobs and monitoring them
- Understanding the requirements for new initiatives and provide a business solution.
Education
Bachelor of Science - Computer Science and Engineering
UCEK - JNTUK
Andhra Pradesh05.2014
Skills
- Spark
- Python
- SQL
- Databricks
- Snowflake
- Dbt
- Azure Data Factory
- Azure Synapse Analytics
- Azure Logic Apps
- Azure Datalake
- Azure Purview
- Azure DevOps
- Unix and Shell Scripting
- Data Warehousing (BigQuery/Azure SQL Data Warehouse)
- Docker
- Kubernetes
- Airflow
Certification
- Snowflake SnowPro Core Certified - 2024
- Databricks Certified Data Engineer Associate - 2023
- Databricks Certified Associate Developer for Apache Spark 3.0 - 2023
- Azure Architect Expert - 2022
- AWS Cloud Architect Associate - 2021
- Certified Kubernetes Application Developer - 2021
- Azure Data Engineer Associate - 2021
- Airflow Astronomer Certification - 2021
- Azure Data Fundamentals - 2021
- Azure ML Fundamentals - 2021
- Azure Fundamentals - 2020
- GCP Professional Architect - 2019
- GCP Professional Data Engineer - 2018
- AWS Certified Developer Associate - 2018
- Cloudera Hadoop and Spark Developer - 2018
Timeline
Manager Projects - Senior Data Engineer
Cognizant Servian
08.2023 - Current
Principal Data Engineer
Neudesic Technologies
08.2021 - 05.2023
Technical Project Manager
Bridgei2i Analytics Solutions
04.2020 - 08.2021
Senior Consultant II - Data Engineering
Neudesic Technologies
06.2019 - 04.2020
Consultant - Data Engineering
Core Compete Pvt Ltd.
07.2018 - 06.2019
Senior Data Engineer
Optum (UHG)
05.2017 - 07.2018
Software Engineer
Tech Mahindra
05.2014 - 05.2017
Bachelor of Science - Computer Science and Engineering
UCEK - JNTUK
English Language Proficiency
PTE Academic : 88/90 equivalent to IELTS 9 - July, 2023
Work Rights
- On 482 Dependent TSS Visa with full working rights.
English Language Proficiency
PTE Academic : 88/90 equivalent to IELTS 9 - July, 2023
Work Rights
- On 482 Dependent TSS Visa with full working rights.
Similar Profiles
Nikhil LakshmishaNikhil Lakshmisha
Associate Consultant at Cognizant Servian (formerly Servian)Associate Consultant at Cognizant Servian (formerly Servian)
Corina NeacsuCorina Neacsu
Associate Consultant - Software Developer at Cognizant ServianAssociate Consultant - Software Developer at Cognizant Servian
Maria Nishanth DMaria Nishanth D
Associate Consultant (DevOps) at ServianAssociate Consultant (DevOps) at Servian