Senior Principal Consultant - Data Architect

Data Architect – Databricks (Azure/AWS)

Role Overview:

We are seeking an experienced Data Architect specializing in Databricks to lead the architecture, design, and migration of enterprise data workloads from on-premises systems (e.g., Oracle, Exadata, Hadoop) to Databricks on Azure or AWS. The role involves designing scalable, secure, and high-performing data platforms based on the medallion architecture (bronze, silver, gold layers), supporting large-scale ingestion, transformation, and publishing of data.

Required Skills and Experience:

• 8+ years of experience in data architecture or engineering roles, with at least 3+ years specializing in cloud-based big data solutions.
• Hands-on expertise with Databricks on Azure or AWS.
• Deep understanding of Delta Lake, medallion architecture (bronze/silver/gold zones), and data governance tools (e.g., Unity Catalog, Purview).
• Strong experience migrating large datasets and batch/streaming pipelines from on-prem to Databricks.
• Expertise with Spark (PySpark/Scala) at scale and optimizing Spark jobs.
• Familiarity with ingestion from RDBMS (Oracle, SQL Server) and legacy Hadoop ecosystems.
• Proficiency in orchestration tools (Databricks Workflows, Airflow, Azure Data Factory, AWS Glue Workflows).
• Strong understanding of cloud-native services for storage, compute, security, and networking.

Preferred Qualifications:

• Databricks Certified Data Engineer or Architect.
• Azure/AWS cloud certifications.
• Experience with real-time/streaming ingestion (Kafka, Event Hubs, Kinesis).
• Familiarity with data quality frameworks (e.g., Deequ, Great Expectations).