Data Engineer

Required Skills:

- 4-7 years of working experience and Strong expertise in SQL and Python, including pandas and PySpark, with a focus on analytical and largescale data engineering.

- Deep handson experience with Azure Databricks, including PySpark, Delta Lake, notebooks, cluster configuration, performance tuning, and cost optimization.

- Experience with Azure Synapse, ADF, Azure Functions

- Knowledge of Databricks workflows / job orchestration

- Strong working knowledge of Unity Catalog, including data governance, access control, lineage, and security best practices.

- Advanced PySpark skills covering distributed processing, optimization strategies, handling large and small files, and performance tuning.

- Advanced SQL skills (complex joins, window functions, query optimization, analytical patterns).

- Strong understanding of ETL / ELT architecture, data modeling, data quality frameworks, and governance principles.

- Solid understanding of Risk and Regulatory data domains, including Credit Risk, Capital, Stress Testing, and Regulatory Reporting.

- Proven ability to design solutions independently, make architectural tradeoffs, and deliver productiongrade systems.

- Ability to clearly articulate design decisions and collaborate effectively with architects, stakeholders, and downstream consumers.Understanding of AI/ML concepts (regression, classification, model validation, monitoring).

Good to have:

- Familiarity with model validation, backtesting, monitoring dashboards

- Exposure to DevOps tools (Git, CI/CD)

- Experience with large financial institutions