Data Analytics - Java/Spring Boot, Kafka, NO SQL/SQL

Discover your future at Citi

Working at Citi is far more than just a job. A career with us means joining a team of more than 230,000 dedicated people from around the globe. At Citi, you’ll have the opportunity to grow your career, give back to your community and make a real impact.

Job Overview

We are seeking a highly motivated and skilled Kafka Developer to join our engineering team. The Kafka Developer will be responsible for designing, developing, and maintaining robust and scalable applications that interact with our Apache Kafka streaming platform. This role requires a strong understanding of Kafka's core concepts, client APIs, and best practices for building reliable data pipelines and stream-processing solutions.

Roles and Responsibilities

• Design, develop, and maintain high-performance and fault-tolerant Kafka producers and consumers using Java, Scala, Python, Go, or other relevant programming languages.
• Implement data streaming solutions, including real-time data ingestion, processing, and distribution.
• Develop and optimize Kafka Streams or ksqlDB applications for complex event processing, data transformations, and aggregations.
• Write clean, well-tested, and maintainable code following best practices and coding standards.
• Integrate Kafka-based solutions with various upstream and downstream systems, including databases, APIs, and other micro-services.

Data Pipeline Management

• Work with Kafka Connect to configure and manage connectors for integrating with diverse data sources (e.g., CDC from databases, logs, external APIs) and sinks (e.g., data lakes, warehouses, other services).
• Ensure data quality, consistency, and integrity across Kafka-based data pipelines.
• Collaborate with data engineers and DevOps teams to establish and enforce data governance policies, including schema evolution using Schema Registry.
• Develop and execute unit, integration, and end-to-end tests for Kafka applications and data pipelines.
• Perform performance testing and tuning of Kafka clients and streaming applications to meet throughput and latency requirements.
• Implement monitoring and alerting for Kafka applications to ensure their health and performance in production.

Collaboration and Documentation

• Collaborate closely with product owners, architects, and other development teams to gather requirements, define scope, and deliver solutions.
• Participate in code reviews, technical discussions, and knowledge-sharing sessions.
• Create and maintain clear and concise technical documentation for Kafka applications, data flows, and operational procedures.
• Assist in troubleshooting and debugging production issues related to Kafka applications.

Continuous Improvement

• Stay up-to-date with the latest developments in the Kafka ecosystem and streaming technologies.
• Proactively identify opportunities for performance improvements, optimization, and automation in Kafka applications and development processes.
• Contribute to the continuous improvement of our data streaming platform and development practices.

• Experience with cloud platforms (AWS, Azure, GCP) and containerization technologies (Docker, Kubernetes).
• Familiarity with monitoring tools (e.g., Prometheus, Grafana, ELK stack).
• Knowledge of schema management with Schema Registry (e.g., Avro, Protobuf).
• Experience with unit testing frameworks and practices for streaming applications.
• Understanding of data modeling principles and database concepts.