Senior Python Developer + Spark

About the Role:

As a Python Engineer, you'll be a key contributor to building robust and scalable data pipelines that integrate with diverse internal and external systems. You'll be involved in architectural decisions, ensuring the platform's health through monitoring, infrastructure setup, and CI/CD optimization. This platform processes vast datasets, running sophisticated models to detect potential market abuse, making your contribution crucial to our success.

You'll collaborate closely with data scientists and business stakeholders, translating their needs into effective technical solutions. You'll also champion development and deployment best practices within the team, driving continuous improvement.

This is a high-impact role with significant visibility

Responsibilities:

• Develop and maintain Python micro-services, encompassing numerous data pipelines and algorithms designed to identify market abuse.
• Expand ETL processes to incorporate new data sources seamlessly.
• Collaborate with quantitative analysts and data scientists to understand and implement requirements for new algorithms, data onboarding, quality checks, and timeliness.
• Build strong relationships with clients and stakeholders to understand their needs and prioritize work effectively.
• Thrive in a multidisciplinary team environment, working closely with fellow developers, quants, data scientists, and production support teams.

Qualifications:

• 5-8 years of relevant experience
• Proven experience designing and implementing Python-based backend services.
• Expertise in building data pipelines leveraging Big Data technologies, preferably Spark and Python.
• Experience with Angular, React or other front-end frameworks is a plus, as is a general understanding of full-stack development principles.
• Strong database skills, with experience using SQL and NoSQL technologies (e.g., SQL Server, MongoDB).
• Experience collaborating with data scientists and developing pipelines that support statistical algorithms.
• Demonstrated experience working in a DevOps environment, including CI/CD tools, monitoring tools, and log aggregation tools. Experience with Docker/Kubernetes is mandatory.
• Ability to automate and streamline the build, test and deployment of data pipelines.
• Deep experience with software engineering best practices (unit testing, automation, design patterns, peer review, etc.) 
• Proven experience of providing technical vision and guidance to a data team.
• Thrives in a dynamic environment, capable of managing multiple tasks simultaneously while maintaining a high standard of work.
  

Education:

• Bachelor’s degree/University degree or equivalent experience