RAHUL KHATEI

Specialist in developing Spark jobs to process data in Spark using spark core and spark SQLAPI’s and pushing to downstream application like Oracle tables and Azure blobstorages.

Extensively worked on Bigdata/Spark framework,AWS S3,SQS, ec2instances,autoloader and AWS Databricks.

Good exposure to ADF(Azure Datafactory),ADB(Azure Databricks),ADLS Zen2 and Azure Synapse .

Worked efficiently of BRS(Baggage Reconciliation System),reading streaming data from Kafka topic and processing into Medallion layer .

Worked on Event processing system on BRS(BagCreatedEventProcess,BagTrackingEventAuditNotofication,BagoffloadNotification process etc)

Generated passenger reports,supporting analysis of volumes,trends and operation planning.

BRS gold layer reports integration with Cognos for Business reporting.

Implementing Upsert logic using SCD1 and SCD2 implementation in spark.

Worked on different file formats like Avro,Parquet,JSON and CSV.

An effective communicator with excellent relationship building,analytical and negotiation skills and an optimistic mindset towards the task .

Specialist in Performance Optimization of Spark Job sand failure in Prod.

• Worked on Retail domain of Nielsen Market analysis with supplies of Kraft-Heinz products across various country market regions like US,UK,Germany and Netherland
• Worked on medallion architecture with Azure Databricks, consuming batch and streaming data from Azure Blob storage and and messaging agents from processing ETL pipelines.
• Performance optimzation of long running jobs in Production environment
• Implemetation of Bussiness logic including incremental load(SCD1 and SCD2) aligned with business requirement using pyspark and DBT core.

• Enhanced system performance with thorough code reviews, debugging, and optimization techniques.
• Improved systems with addition of new features and infrastructure.
• Provided educational expertise and mentoring to junior team members.
• Implementation of PriceDelivery and RevenueManagment pipeline Using spark-scala,Databricks
• Direct interaction with Business on framework enhancement