Anil Kumar Mallikarjunaiah

Intellihub is a digital energy management company, who provide smart
metering devices and services to over 50 energy retailers across ANZ.
Driving a greenfield data platform initiative on Databricks (AWS),
architecting and implementing scalable ingestion frameworks using
Python/Spark, data transformation pipelines with dbt, and CI/CD
enhancements via Bitbucket, ensuring automation and reliability across
environments.
•
Established enterprise-grade standards for data governance, access
management, and security within Databricks, enabling compliance and
controlled self-service analytics.
•
Leading the design of an AI/ML framework for business use cases,
operationalizing models through MLOps best practices and exploring
GenAI capabilities and foundation models within Databricks to unlock
predictive and generative insights.
•
Defined standardized procedures for the data product lifecycle, including
code promotion strategies, dbt workflow quality checks, and batch
orchestration standards, ensuring consistency and auditability across
deployments.
•
Collaborated with external vendors on advanced data and ML solutions,
overseeing solution design approvals, code reviews, and production
release processes, ensuring alignment with platform architecture and
security standards.
•
Currently leading two specialized teams:Data Engineering squad focused
on building robust data solutions.
Data/MLOps team responsible for platform reliability and automation.
Additionally, managing people leadership responsibilities, including
performance reviews, career development planning, and technical
mentorship to foster growth and innovation.

• Designed and developed generic metadata-driven ingestion pattern for data shared via Snowflake to Snowflake data shares with proper auditing and monitoring in place. The architecture uses a combination of Snowflake, DBT, Airflow with an event driven orchestration upon data share.
• Lead the data platform team in implementing CI/CD best practices, reviewing the branching strategy and designing the optimal workflow for the data engineering pipelines.
• Involved in requirement gathering, architecting and designing various partner related solutions key for Flybuys successful relationship with its partners like Kmart, Bunnings, Onepass , Coles etc. This required numerous workshops with the partners, cross-team collaboration within Flybuys and guiding the data engineering team for solution delivery.
• Championed incorporating data quality checks across the data platform workflows by designing data quality framework within DBT with proper auditing capability.
• Spearheaded security vulnerability remediation across the data platform with the like of resolving vulnerabilities in ECS tasks, EC2 instances, Lambda and Cloudformation templates.
• Got involved in numerous Snowflake related initiatives like users and roles reviews, data archival strategy, data masking approach for sensitive data, explore Snowflake's new features like Streamlit , dynamic tables, containers etc.

• Working in a fast paced devops environment where i was part of the global data platform which is the single source of truth for all of the organization reporting.
• Work in close collaboration with our stakeholders to architect, design, implement, test and manage leading data and analytics solutions.
• Instrumental in building datalake architecture for with extensive use of AWS services like S3, lambda, Redshift, Glue, EMR, Cloudwatch etc.
• Worked extensively in maintaining and enhancing AWS Redshift database including many cluster performance enhancement activities like scheduled vacuum process, auto killing long running queries, cluster query work load management etc.
• Advanced Terraform knowledge which was used as an infrastructure as code component to maintain the code to build the required infrastructure like EC2, EMR, RDS, Redshift, IAM , VPC and other networking components etc.
• Instrumental in developing generic framework to ingest near real time kafka events using Spark on EMR with proper alerting and monitoring for failures.
• Involved in design decisions for any new solutions and guided the junior resources with cloud best practices to follow.
• People management after being promoted as Engineering Manager.
• Involved in developing self serve data pipeline features for the analysts to develop, schedule, monitor their own data pipelines.
• Followed best practices of security using well defined IAM roles for access management between AWS resources. Explored advanced AWS services like SecurityHub, AWS Config, Macie etc.

NBN has an enterprise warehouse – Unified Data Source (UDS) being accessed by many business units. The UDS has many subject modules like conformed order model, work and work force management model, incidents products etc. The UDS conforms data from many different source systems and conforms them to satisfy the business need.

• Model the data and integrate it with different source systems. Apply ETL and business rules and create required facts and dimensions and expose the data to the users using Tableau data sources.
• Make the datawarehouse fit for business by bringing in the data availability to 7 am. For this analysis was done with the existing batch pain points and proper measures were taken like – tuning the long datastage running jobs and Netezza queries, create generic patterns to reduce the CDC process times, decommission unnecessary objects and remove their schedules.
• Designed and built generic spark framework to acquire data from AWS data lake into AWS Redshift.
• Involved in POC related to migrating traditional data warehosue on Netezza to AWS Redshift involving other AWS services like S3,DynamoDB,Athena,Glue,Redshift Spectrum. Other POC include loading data from AWS S3 into netezza/Redhsift using PySpark.

Worked extensively in large scale datawarehousing project in TELSTRA.