Summary
Overview
Work History
Education
Skills
Accomplishments
Timeline
Generic

ABC DFG

Sydney,NSW

Summary

Passionate Senior Engineer with 7+ years of experience specializing in assessing, evaluating, and developing highly scalable products related to Big Data technologies. Proficient in architecting and developing optimized distributed processing solutions using Hadoop Ecosystem frameworks, including Spark, Hadoop, SQL, Hive, HBase, and HDFS. Demonstrated hands-on expertise in end-to-end project development, managing 6-7 TB of daily data processing, and overseeing a 10PB data repository. Adept in various Big Data frameworks such as Spark, SQL, Hive, HDFS, and MapReduce. Skilled in configuring CI/CD pipelines on AWS and Azure, showcasing leadership in analyzing, updating, and maintaining technical and operational documentation. Recognized for delivering high-quality solutions within agile methodologies.

Overview

8
8
years of professional experience

Work History

SENIOR BUSSINESS ANALYST TECH

HAYS ( NATIONAL AUSTRALIA BANK)
02.2023 - Current
  • Designed and implemented a comprehensive data warehouse architecture, incorporating cloud-native solutions for optimal scalability.
  • Developed and maintained robust processes for seamless integration of diverse data formats.
  • Implemented scalable solutions to accommodate growing data needs, utilizing serverless computing and workflow orchestration.
  • Engineered and optimized Extract, Transform, Load (ETL) processes for efficient data movement and transformation.
  • Utilized cloud-based data warehousing solutions for effective storage and retrieval of data.
  • Implemented stringent measures for data quality assurance to ensure accuracy and reliability.
  • Monitored and optimized the performance of the data warehouse platform using cloud-native monitoring and logging tools.
  • Collaborated with cross-functional teams to understand and address evolving data requirements.
  • Enforced data governance policies to maintain data integrity and security, aligning with industry best practices.
  • Documented data warehouse processes, configurations, and specifications for clear documentation and knowledge transfer.
  • Evaluated and incorporated emerging technologies to enhance the capabilities of the data warehouse.
  • Provided user support, troubleshooted data-related issues, and optimized queries for efficient data retrieval.
  • Conducted in-depth data analysis to identify trends, patterns, and actionable insights.
  • Implemented robust monitoring tools and strategies to proactively identify and address performance issues.
  • Led continuous improvement initiatives to enhance the overall efficiency and effectiveness of the data warehouse platform.

Senior Engineer II

American Express
07.2016 - 12.2022
  • Accelerated product development cycles by employing Agile methodologies and fostering a collaborative work environment.
  • Maximized resource utilization with effective management of personnel, materials, and equipment for optimal project outcomes.
  • Reduced production downtime by developing comprehensive maintenance plans and training staff on preventive measures.
  • Delivered end to end distributed processing solution to parse complex hierarchical raw data like XML, JSON, mainframe files(fixed width and variable) processing more than 3 Tb of data per day.
  • Analyzed and improved performances of the existing MapReduce applications by 25% by porting them to spark.
  • Designed and solutioned parallel processing capabilities in the enterprise wise data warehouse which processes 6-7 TB of data in a day and houses over 10PB of data.
  • Worked on to solution a process which consume streaming messages from sources such as Solace/Kafka, compute using Spark-SQL and finally publish these messages to real time data sources .
  • Lead QA team of 5 to architect and build automation Framework for big data system which resulted in reduction of time to production for the featured to be achieved.
  • Mentored new joiners and Engineers in the team to provide cross functional knowledge within the system.
  • Practitioner of SAFe Agile with instruments like PI planning, scrum, kanban, CICD, etc.
  • Worked along with product owner to create and refine user stories and acceptance criteria.
  • Improved end to end code deployment process to production using customized build process and Jenkins(CI/CD).
  • Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability.
  • Designed and implemented a solution leveraging apache yarn shell and spark to process and collate large number of files consolidating over 10-15 TB.

Education

Master of Technology - Information Technology

INTERNATIONAL INSTITUTE OF INFORMATION TECHNOLOGY
INDIA
07.2016

Bachelor of Technology - Electronics And Communications Engineering

JNTU
INDIA
04.2013

Skills

  • Amazon Web Services,Azure
  • Data Migration
  • Data Modeling
  • Programming( Java,Python,C, CPP, Bash)
  • Data Warehousing
  • Machine Learning
  • Data Visualization
  • Python
  • Scaling Agile
  • GIT
  • Database(MySQL, NoSQL,Mysql workbench)
  • Jenkins(CI/CD)

Accomplishments

  • Awarded with “ROLE MODEL LEARNER 2017 ” across American Express Technology.
  • Recipient of Talent Scholarship from Societe Generale Global Solution Centre (SG GSC) at IIITB.

Timeline

SENIOR BUSSINESS ANALYST TECH

HAYS ( NATIONAL AUSTRALIA BANK)
02.2023 - Current

Senior Engineer II

American Express
07.2016 - 12.2022

Master of Technology - Information Technology

INTERNATIONAL INSTITUTE OF INFORMATION TECHNOLOGY

Bachelor of Technology - Electronics And Communications Engineering

JNTU

Similar Profiles

CREATE PROFILE
ABC DFG