Python Technical Lead

Role Summary

We are seeking a Hands-On Python Lead Engineer to lead the design and implementation of enterprise-grade big data pipelines. This is a technical leadership role requiring significant daily coding. The successful candidate will architect scalable ETL frameworks using Python, PySpark, and Apache Airflow, serving as both a lead developer and a system designer. Experience integrating with Salesforce ecosystems is strongly preferred.

Core Responsibilities

Architectural Design & Framework Development

• ETL Framework Design: Architect modular, metadata-driven ETL frameworks to standardize data ingestion and processing across the organization.
• Airflow Orchestration: Design complex workflow orchestration strategies. Customize Airflow environments, including developing custom operators, sensors, and plugins to support business logic.
• Scalability & Performance: Engineer high-performance pipelines capable of handling large-scale datasets; optimize DAG execution, memory management, and concurrency settings.

Hands-On Implementation

• Code Contribution: Actively write, test, and deploy production-quality code in Python and PySpark (60%+ of role focus on hands-on development).
• Code Quality Assurance: Enforce strict coding standards, design patterns, and unit testing coverage through rigorous code reviews.

Integration Strategy

• Data Integration: Architect robust patterns for consuming data from Databases, Data Lakes, and APIs.
• CRM Data Integration: Oversee the strategy for ingesting and synchronizing data from Salesforce, ensuring adherence to API limits and data integrity.

Required Qualifications

• Python & PySpark Expertise: 8+ years of software development experience. Mastery of Python and deep proficiency in PySpark for distributed data processing.
• Airflow Proficiency: Extensive experience setting up, managing, and customizing Apache Airflow in a production environment.
• ETL Architecture: Proven track record of building custom ETL frameworks from scratch, moving beyond basic scripting to software engineering-based data solutions.
• Database Skills: Advanced SQL knowledge and experience with cloud data warehouses.

Preferred Qualifications

• Salesforce Knowledge: Experience with Salesforce data models, SOQL, and bulk data extraction APIs.
• CI/CD: Experience implementing CI/CD pipelines for data engineering (GitHub Actions, Jenkins).