Key Responsibilities

Design and implement scalable data architectures including data lakehouse and real-time streaming solutions.
Develop and maintain robust data pipelines using Spark, EMR, Glue, and other big data technologies.
Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to understand data requirements.
Ensure data quality, integrity, and security across all data platforms.
Lead data modeling efforts using Kimball, Inmon, Data Vault, and Medallion architectures.
Establish and enforce best practices for data governance, metadata management, and data lineage.
Mentor junior engineers and contribute to team development and knowledge sharing.
Implement CI/CD pipelines and infrastructure-as-code for data workflows using tools like Terraform and CloudFormation.
Monitor and optimize data workflows and implement alerting mechanisms for pipeline failures.
Ensure compliance with data privacy regulations such as GDPR and HIPAA.

Technical Skills and Core Competencies

Expertise in cloud platforms (AWS & Azure) and services such as S3, Kinesis, Redshift, and Athena.
Azure Blob Storage, Azure Databricks, Azure Data Factory, - Azure Synapse/ Stream Analytics.
Proficiency in big data technologies including Spark, Hadoop, Hive, and Presto.
Strong experience with ETL tools and frameworks.
Knowledge of data lakehouse formats like Delta, Iceberg, and Hudi.
Experience with data visualization tools such as Tableau and Power BI.
Familiarity with enterprise architecture frameworks like TOGAF.
Understanding of microservices architecture and API design for data services.
Experience with data testing frameworks such as Great Expectations.
Strong communication and documentation skills.

Data Architect