Lead Software Engineer

About BU

The Product team forms the crux of our powerful platforms and connects millions of customers to the product magic. This team of innovative thinkers develop and build products that help Epsilon be a market differentiator. They map the future and set new standards for our products, empowered with industry best practices, ML and AI capabilities. The team passionately delivers intelligent end-to-end solutions and plays a key role in Epsilons success story.

Why we are looking for you?

At Epsilon, we run on our peoples ideas. Its how we solve problems and exceed expectations. Our team is now growing, and we are on the lookout for dedicated individuals who always set the standard by constantly challenging themselves and are experts in building customized solutions in the digital marketing space.

What will you enjoy in this Role?

So, are you someone who wants to work with innovative technology and enable marketers to create data-driven, omnichannel consumer experiences through data platforms? Then you could be exactly who we are looking for.

Apply today and be part of a creative, innovative, and dedicated team thats not afraid to push boundaries or take risks.

Responsibilities:

What will you do?

As a Lead Software Engineer in the Epsilon Product Development team, you will design and implement scalable data processing solutions using Scala and Spark, working on cloud infrastructure. You will drive the development and optimization of data workflows.

In this role, you will lead technical discussions, mentor engineers, ensure high-performance system design, and promote engineering standard methodologies across the team.

Strong written and verbal communication skills (in English) are required to facilitate work across multiple countries and time zones. Good understanding of Agile Methodologies SCRUM.

Qualifications:

• Strong experience (8-10 years) in Scala programming language and extensive experience with Apache Spark for Big Data processing for design, developing and maintaining scalable on-prem and cloud environments experience.
• Proficiency in performance tuning of Spark jobs, optimizing resource usage, shuffling, partitioning, and caching for maximum efficiency in Big Data environments.
• Experience in designing and implementing scalable, fault-tolerant data pipelines with end-to-end monitoring and alerting.
• Hands on experience in AWS (or any other cloud environment)for 2+ years. However, AWS EMR experience is preferred
• Hands-on experience with Python.
• Solid grasp of database systems and SQLs for writing efficient SQLs (RDBMS/Warehouse) to handle TBS of data.
• Familiarity with design patterns and standard processes for efficient data modelling, partitioning strategies, and sharding for distributed systems
• Experience in working on GIT (or equivalent source control) and solid understanding of Unit and integration test frameworks.
• Must have the ability to collaborate with partners/Product Owners/teams to understand requirements and develop a working solution, and the ability to work within tight deadlines and optimally prioritize and complete tasks in a high-pressure environment.
• Able to mentor engineers, review code, and contribute to setting engineering standard methodologies.

Advantageous to have experience on below:

• Databricks experience including Databricks Notebooks, Delta Lake, and Catalogues.
• Proficiency in using the ELK (Elasticsearch, Logstash, Kibana) stack for real-time search, log analysis, and visualization.
• Strong background in analytics, including the ability to derive actionable insights from large datasets and support data-driven decision-making.
• Experience with data visualization tools like Tableau, Power BI, or Grafana.
• Familiarity with Docker for containerization and Kubernetes for orchestration.
• It's good to have AWS or any other cloud certifications.