AIA Data Engineering - Director
Location: Hyderabad
Summary: with 20+ years of experience, specializing in Spark, Java, AWS, and other cloud technologies, typically holds a leadership role focused on strategic data architecture, platform development, and team management.
Key Responsibilities:
- Strategic Leadership & Architecture:
Define and execute the organization's data strategy and roadmap. Design and implement scalable, robust, and secure data architectures, leveraging cloud platforms (AWS, Azure, GCP) and big data technologies (Spark, Hadoop).
- Platform Development & Optimization:
Oversee the design, development, and maintenance of enterprise-level data platforms, including data lakes, data warehouses, and real-time streaming solutions. Optimize data pipelines, ETL processes, and data storage for performance and cost-efficiency.
- Technical Expertise & Innovation:
Provide deep technical expertise in distributed data processing (Spark), programming languages (Java, Scala, Python), and cloud services (AWS Glue, Kinesis, Redshift, S3, Lambda). Evaluate and integrate emerging data technologies and best practices.
- Team Leadership & Mentorship:
Lead and mentor a team of data engineers, fostering a culture of technical excellence, collaboration, and continuous learning. Drive talent development, performance management, and resource allocation within the data engineering function.
- Stakeholder Collaboration:
Partner with executive leadership, product teams, data scientists, and other stakeholders to understand business requirements, translate them into technical solutions, and ensure successful delivery of data initiatives.
- Data Governance & Security:
Establish and enforce data governance policies, security protocols, and compliance standards across all data platforms and pipelines.
- Budget Management & Resource Planning:
Manage budgets, resources, and vendor relationships related to data engineering initiatives and cloud infrastructure.
Qualifications:
- 20+ years of progressive experience in data engineering, with a significant portion in leadership roles.
- Proven expertise in designing, building, and managing large-scale data platforms in cloud environments (AWS, Azure, or GCP).
- Deep hands-on experience with Apache Spark for distributed data processing.
- Strong proficiency in Java and other relevant programming languages (Scala, Python).
- Extensive experience with AWS services for data engineering (e.g., S3, Redshift, Kinesis, Glue, Lambda).
- Demonstrated experience in leading and mentoring high-performing data engineering teams.
- Strong understanding of data architecture principles, data modeling, ETL/ELT processes, and data warehousing concepts.
- Excellent communication, interpersonal, and stakeholder management skills.
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.