Data Architect

Job Title: Databricks Architect

Experience: 10+ Years

Location: Hyderabad

Employment Type: Full-Time

Job Summary

We are looking for an experienced Databricks Architect to design, develop, and implement scalable data platforms and modern data solutions on Databricks. The ideal candidate should have strong expertise in data architecture, cloud technologies, big data processing, and data engineering best practices. The candidate will work closely with business stakeholders, data engineers, and leadership teams to define and implement enterprise-wide data strategies.

Key Responsibilities

• Design and architect scalable data platforms using Databricks and cloud technologies.
• Define end-to-end data architecture, including ingestion, transformation, storage, and consumption layers.
• Lead the implementation of Lakehouse architecture using Databricks.
• Design and optimize ETL/ELT pipelines for large-scale data processing.
• Establish data governance, security, and compliance standards.
• Work with business stakeholders to understand requirements and translate them into technical solutions.
• Provide architectural guidance to development teams and review solution designs.
• Optimize Databricks workloads for performance, scalability, and cost efficiency.
• Implement CI/CD pipelines and DevOps best practices for data platforms.
• Mentor data engineers and technical teams on Databricks best practices.
• Collaborate with cross-functional teams including Data Science, Analytics, and Business Intelligence.

Required Skills

• Strong experience in Databricks Architecture and Administration.
• Expertise in Apache Spark (PySpark/Scala).
• Experience with Delta Lake, Unity Catalog, and Databricks Workflows.
• Strong understanding of Lakehouse Architecture.
• Hands-on experience with cloud platforms such as Azure, AWS, or GCP.
• Experience designing and implementing enterprise data warehouses and data lakes.
• Strong SQL and data modeling expertise.
• Experience with data integration and orchestration tools.
• Knowledge of performance tuning and optimization techniques.
• Experience implementing data security and governance frameworks.
• Strong understanding of CI/CD and DevOps practices.
• Excellent communication and stakeholder management skills.

Preferred Skills

• Experience with Azure Data Factory (ADF), Azure Synapse, or equivalent cloud-native services.
• Experience with streaming technologies such as Kafka and Structured Streaming.
• Knowledge of Data Mesh and Modern Data Architecture concepts.
• Exposure to AI/ML workloads on Databricks.
• Databricks certifications are highly preferred.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• 10+ years of overall IT experience with at least 4+ years of hands-on Databricks architecture experience.
• Proven experience leading enterprise-scale data transformation projects.

Nice to Have

• Databricks Certified Data Engineer Professional.
• Databricks Certified Solutions Architect.
• Experience working in Agile/Scrum environments.
• Experience with multi-cloud data platform implementations.

Keywords: Databricks, Spark, PySpark, Delta Lake, Unity Catalog, Lakehouse, Azure, AWS, GCP, Data Architecture, Data Engineering, Data Lake, ETL, ELT, Data Governance, Kafka, ADF, CI/CD.