Data Engineer

About the Company

We are a leading financial services firm dedicated to providing innovative solutions and exceptional service to our clients. Our mission is to empower businesses with data-driven insights while fostering a culture of inclusivity and collaboration.

About the Role

The Data Engineer PySpark & Azure Databricks will be responsible for designing and developing scalable data pipelines, processing financial data, and ensuring data quality and compliance.

Responsibilities

• Design and develop scalable data pipelines using PySpark on Azure Databricks
• Process structured and unstructured financial data (transactions, customer data, risk data)
• Build ETL/ELT pipelines using Azure services:
• Azure Data Factory (ADF)
• Azure Data Lake Storage (ADLS)
• Implement data transformation, cleansing, and aggregation for:
• Risk analytics
• Fraud detection
• Regulatory reporting (Basel, AML, KYC)
• Optimize Spark jobs for performance (partitioning, caching, tuning)
• Ensure data quality, governance, and security compliance
• Integrate data pipelines with downstream tools (Power BI, SQL, ML models)
• Work closely with business stakeholders (risk, compliance, finance teams)

Qualifications

Experience: 410 Years

Required Skills

• Strong hands-on in PySpark & Spark SQL
• Expertise in Azure Databricks
• Experience with:Azure Data Factory (ADF)
• ADLS / Blob Storage
• Good understanding of:Data warehousing concepts
• Financial data models
• SQL & Python proficiency
• Knowledge of Git, CI/CD pipelines
• Experience with Delta Lake (highly preferred)

Preferred Skills

• Experience with data governance tools
• Familiarity with machine learning concepts

Pay range and compensation package

Competitive salary based on experience and qualifications.

Equal Opportunity Statement

We are an equal opportunity employer and are committed to fostering a diverse and inclusive workplace. We encourage applications from all qualified individuals regardless of race, gender, age, sexual orientation, disability, or any other characteristic.