Big Data Engineer - C11 - BANGALORE

Bigdata developer responsible for Designing and Developing ETL integration patterns. Strong experience in understanding functional requirements in analyzing the large using Py Spark scripts and Hive queries. Must be good at Unix scripting , configuring scheduling tools like AutoSys etc. Also should be well versed with Agile technology like JIRA etc.

Responsibilities:

• Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements, including using script tools and analyzing/interpreting code

• Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems

• Apply fundamental knowledge of programming languages for design specifications.

• Analyze applications to identify vulnerabilities and security issues, as well as conduct testing and debugging

• Serve as advisor or coach to new or lower level analysts

• Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions

• Resolve issues by identifying and selecting solutions through the applications of acquired technical experience and guided by precedents

• Has the ability to operate with a limited level of direct supervision.

• Can exercise independence of judgement and autonomy.

• Acts as SME to senior stakeholders and /or other team members.

• Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.

Qualifications:

• 4+ years of experience as a Python/PySpark developer with expertise in automation testing to design, develop, and automate robust software solutions and testing frameworks like Pytest, Behave

  • 4+ years of experience as Big Data Engineer to develop, optimize, and manage large-scale data processing systems and analytics platforms with exposure to Snowflake.

  • 4+ years of experience in distributed data processing & near real-time data analytics using PySpark. Strong understanding of PySpark execution plans, partitioning & optimization techniques.

  • 4+ years of experience in developing, and implementing data pipelines for data ingestion, transformation, and processing.

  • 4+ years of experience in tools such as Selenium, Cucumber BDD, Rest Assured and programming languages such as Java and python for Web application automation and Rest API automation.

  • Familiar with CI/CD pipelines, version control systems (e.g., Git), and DevOps practices.

  • Familiar with Agile Methodologies, Scheduling tools, Cloud Computing, SQL, UNIX

  • Well versed with CI/CD practices and automation tools to streamline development, test automation and deployment processes.

Education:

• Bachelor’s degree/University degree or equivalent experience

Good to have 3-5 years of experience in tools such as Selenium, Cucumber BDD, Rest Assured and programming languages such as Java and python for Web application automation and Rest API automation