Big Data Engineer

Job Opportunity: Big Data Engineer

Location: Bengaluru, India (On-site, Full-time)

Company: BrandVelocity.ai | Bravelo Tech Solutions Pvt Ltd

Brandvelocity.ai is Looking for a Data Pipeline and Processing Engineer to design, develop, and maintain large-scale data pipelines using Java, Kafka, Spark, and Scala. The successful candidate will be responsible for building efficient, scalable, and reliable data processing systems that extract insights from vast amounts of data.

Responsibilities:

1. Design and Develop Data Pipelines: Design, develop, test, and deploy data pipelines using Java, Kafka, Spark, and Scala to process large datasets.

2. Data Ingestion and Processing: Ensure seamless data ingestion from various sources into the pipeline, perform data cleaning, transformation, and validation as needed.

3. Kafka Integration: Integrate Kafka with other components of the data pipeline, ensuring high-throughput, low-latency data processing.

4. Spark Development: Develop Spark-based applications for complex data processing tasks, such as ETL, data aggregation, and machine learning algorithms.

5. Scala Programming : Write efficient Scala code to leverage the strengths of functional programming languages in big data processing.

6. Data Quality and Validation : Implement data quality checks and validation mechanisms to ensure accurate and reliable data processing results.

7. Monitoring and Maintenance: Monitor pipeline performance, identify bottlenecks, and perform maintenance tasks to ensure optimal system uptime and efficiency.

Requirements:

1. Education:Bachelor's or Master's degree in Computer Science, Information Technology, or related field.

2. Experience: 3 to 5 years of experience in designing and developing large-scale data pipelines using Java, Kafka, Spark, and Scala.

3. Skills:

1. Proficiency in Java, Kafka, Spark, and Scala programming languages.
2. Experience with data processing frameworks like Apache Beam or AWS Glue.
3. Strong understanding of big data processing concepts, including data ingestion, processing, storage, and retrieval.
4. Knowledge of distributed computing paradigms, such as MapReduce and Spark Resilient Distributed Datasets (RDDs).

Nice to Have:

1. Cloud Computing Experience: Experience with cloud platforms like AWS, GCP, or Azure is a plus.

2. Data Science Experience:Knowledge of data science concepts, including machine learning algorithms and statistical modeling, would be beneficial.

How to Apply:

Send your profile/portfolio/GitHub to

with the subject line: "Big Data Engineer"