Data Engineer

Job Title: Senior Data Engineer -Vertica+Copilot

Experience: 58 Years

Location: Bangalore (Hybrid)

Notice Period: Immediate to 20 Days (030 days preferred)

Role Overview

We are looking for a highly skilled Senior Data Engineer / AI Developer Copilot with strong expertise in Vertica and distributed data engineering technologies. The ideal candidate will be responsible for designing, developing, and optimizing high-volume data pipelines while supporting AI-driven data platforms. This role requires hands-on experience with large-scale data processing, performance tuning, and distributed systems.

Key Responsibilities

• Design, develop, and optimize high-volume, high-performance data pipelines.
• Work extensively on Vertica database design, SQL optimization, and performance tuning.
• Develop and maintain efficient ETL workflows for structured and unstructured datasets.
• Implement scalable data solutions using distributed programming concepts.
• Write production-grade code using Python and Shell scripting.
• Integrate and manage NoSQL and caching systems such as Redis and Cassandra.
• Perform database performance analysis and query optimization.
• Collaborate with cross-functional teams including Data Scientists, AI Engineers, and stakeholders.
• Ensure data quality, integrity, and reliability across platforms.
• Participate in architecture discussions and recommend best practices.

Mandatory Skills

• Strong expertise in Vertica (database architecture, performance tuning, query optimization).
• Hands-on experience in Python programming for data engineering.
• Advanced SQL skills.
• Experience in Shell scripting.
• Good understanding of ETL processes and tools.
• Experience with Redis (caching mechanisms).
• Experience with Cassandra (large-scale distributed storage).
• Knowledge of Scala is preferred.
• Strong understanding of distributed systems and big data architecture.

Preferred Qualifications

• Experience working in enterprise-level data environments.
• Exposure to AI/ML data pipelines and Copilot-based solutions.
• Familiarity with CI/CD pipelines and DevOps practices.
• Ability to troubleshoot performance bottlenecks in large data systems.
• Strong analytical and problem-solving skills.

Soft Skills

• Excellent written and verbal communication skills.
• Strong analytical and logical thinking abilities.
• Ability to work independently in a hybrid model.
• Good stakeholder management skills.

Experience Level

• 58 years of relevant experience in Data Engineering / Data Platforms.
• Proven experience in handling large datasets and distributed environments.