Lead Software Engineer

Job Title: Lead Software Engineer (Java, Kafka Streaming, AWS, Distributed Data Systems)

Experience: 812+ years

Role Type: Lead Engineer

Role Overview

We are looking for a highly skilled Lead Software Engineer with strong expertise in Java , real-time streaming systems (Kafka) , distributed databases , and AWS cloud technologies . The ideal candidate will also have exposure to large-scale data pipelines and experience working with NoSQL technologies such as HBase or Aerospike (nice-to-have).

In this role, you will provide technical leadership, design scalable architectures, mentor engineers, and drive high-quality delivery across mission-critical data and streaming platforms.

Key Responsibilities

Technical Leadership & Architecture

• Lead the design and development of scalable, resilient, and high-performance backend services using Java/Spring Boot.
• Architect real-time streaming solutions based on Apache Kafka , including producers, consumers, stream processing, and event-driven microservices.
• Ensure strong data modeling and optimal usage of relational and NoSQL databases.
• Drive engineering best practices: code quality, design reviews, CI/CD, observability, and performance optimization.

Backend & Streaming System Development

• Build microservices and distributed systems with strong APIs (REST or gRPC).
• Implement event-driven architectures, streaming ETL, and stateful stream processing.
• Design and maintain Kafka infrastructure (topics, partitions, schemas, offset management).
• Ensure data consistency, idempotency, and scalability across services.

Data Engineering & Storage

• Collaborate with data engineering teams to design and optimize pipelines.
• Work with large datasets using relational & NoSQL stores (MySQL, Postgres, DynamoDB, HBase, Aerospike, etc.).
• Integrate data pipelines with backend services, ensuring high throughput and reliability.

Cloud & DevOps

• Build and deploy services on AWS (EC2, ECS/EKS, Lambda, S3, DynamoDB, RDS).
• Implement observability using tools like CloudWatch, Prometheus, ELK, Grafana.
• Work with Docker/Kubernetes for containerization and orchestration.
• Optimize infrastructure cost, performance, resilience, and scalability.

Team Leadership & Collaboration

• Mentor and guide engineers in architecture, design patterns, and scalable system design.
• Facilitate technical discussions, backlog grooming, sprint planning, and cross-team coordination.
• Work closely with Product, Data, and DevOps teams to translate requirements into system designs.
• Take end-to-end ownership from design development deployment monitoring.

Required Skills & Experience

Core Skills

• 812+ years of hands-on backend engineering experience.
• Strong proficiency in Java (Collections, Concurrency, Streams, Spring Boot).
• Deep experience with Apache Kafka (core, Streams, Connect).
• Strong understanding of distributed systems, event-driven architecture, and streaming guarantees.
• Experience with AWS cloud services (EC2, ECS/EKS, S3, Lambda, CloudWatch, RDS).
• Solid experience in SQL and relational databases (MySQL, PostgreSQL).
• Familiarity with microservices design , API best practices, and system decomposition.
• Experience in performance tuning, memory optimization, and high-throughput system design.

Nice-to-Have Skills

• Experience with HBase , Aerospike , or similar NoSQL key-value/columnar stores.
• Exposure to Spark, Flink, or streaming ETL frameworks .
• Knowledge of Terraform or IaC tools .
• Experience with monitoring and alerting systems .