Anil Kumar Mallikarjunaiah

• Designed and developed generic metadata-driven ingestion pattern for data shared via Snowflake to Snowflake data shares with proper auditing and monitoring in place. The architecture uses a combination of Snowflake, DBT, Airflow with an event driven orchestration upon data share.
• Lead the data platform team in implementing CI/CD best practices, reviewing the branching strategy and designing the optimal workflow for the data engineering pipelines.
• Involved in requirement gathering, architecting and designing various partner related solutions key for Flybuys successful relationship with its partners like Kmart, Bunnings, Onepass , Coles etc. This required numerous workshops with the partners, cross-team collaboration within Flybuys and guiding the data engineering team for solution delivery.
• Championed incorporating data quality checks across the data platform workflows by designing data quality framework within DBT with proper auditing capability.
• Spearheaded security vulnerability remediation across the data platform with the like of resolving vulnerabilities in ECS tasks, EC2 instances, Lambda and Cloudformation templates.
• Got involved in numerous Snowflake related initiatives like users and roles reviews, data archival strategy, data masking approach for sensitive data, explore Snowflake's new features like Streamlit , dynamic tables, containers etc.

• Working in a fast paced devops environment where i was part of the global data platform which is the single source of truth for all of the organization reporting.
• Work in close collaboration with our stakeholders to architect, design, implement, test and manage leading data and analytics solutions.
• Instrumental in building datalake architecture for with extensive use of AWS services like S3, lambda, Redshift, Glue, EMR, Cloudwatch etc.
• Worked extensively in maintaining and enhancing AWS Redshift database including many cluster performance enhancement activities like scheduled vacuum process, auto killing long running queries, cluster query work load management etc.
• Advanced Terraform knowledge which was used as an infrastructure as code component to maintain the code to build the required infrastructure like EC2, EMR, RDS, Redshift, IAM , VPC and other networking components etc.
• Instrumental in developing generic framework to ingest near real time kafka events using Spark on EMR with proper alerting and monitoring for failures.
• Involved in design decisions for any new solutions and guided the junior resources with cloud best practices to follow.
• People management after being promoted as Engineering Manager.
• Involved in developing self serve data pipeline features for the analysts to develop, schedule, monitor their own data pipelines.
• Followed best practices of security using well defined IAM roles for access management between AWS resources. Explored advanced AWS services like SecurityHub, AWS Config, Macie etc.

NBN has an enterprise warehouse – Unified Data Source (UDS) being accessed by many business units. The UDS has many subject modules like conformed order model, work and work force management model, incidents products etc. The UDS conforms data from many different source systems and conforms them to satisfy the business need.

• Model the data and integrate it with different source systems. Apply ETL and business rules and create required facts and dimensions and expose the data to the users using Tableau data sources.
• Make the datawarehouse fit for business by bringing in the data availability to 7 am. For this analysis was done with the existing batch pain points and proper measures were taken like – tuning the long datastage running jobs and Netezza queries, create generic patterns to reduce the CDC process times, decommission unnecessary objects and remove their schedules.
• Designed and built generic spark framework to acquire data from AWS data lake into AWS Redshift.
• Involved in POC related to migrating traditional data warehosue on Netezza to AWS Redshift involving other AWS services like S3,DynamoDB,Athena,Glue,Redshift Spectrum. Other POC include loading data from AWS S3 into netezza/Redhsift using PySpark.

Worked extensively in large scale datawarehousing project in TELSTRA.