Java Ful stack developer

Java Developer

Key Responsibilities:

Design and Development:

Design, develop, and implement scalable and robust data pipelines and applications using Apache Spark with Java.

Write efficient and optimized code for data processing, transformation, and analysis.

Develop and maintain APIs and microservices using Java or Python for integrating with various systems.

Data Management & Analysis:

Work extensively with SQL for querying, manipulating, and optimizing relational databases (e.g., PostgreSQL, MySQL, Oracle, MS SQL Server).

Design and implement database schemas, stored procedures, and complex queries.

Perform data analysis and develop insights from large datasets using Spark and relevant libraries. Big Data Ecosystem:

Utilize and manage components within the Hadoop ecosystem (HDFS, YARN) for distributed storage and processing.

Implement and manage data ingestion and orchestration workflows (e.g., Apache Airflow, Oozie). System Administration & Scripting:

Work proficiently in Linux/Unix environments, including shell scripting for automation, system monitoring, and deployment tasks.

Manage and configure servers and cloud resources (e.g., AWS EC2, EMR).

Collaboration & Best Practices:

Collaborate with data scientists, analysts, and other engineers to understand requirements and deliver solutions.

Participate in code reviews, promote best practices, and ensure code quality and maintainability. Adhere to Agile/Scrum methodologies for software development.

Performance Optimization & Troubleshooting:

Identify and resolve performance bottlenecks in Spark jobs and data pipelines.

Troubleshoot issues related to data processing, system stability, and application functionality.