Data Engineer - C10

The Data Science Analyst 2 is a developing professional role. Applies specialty area knowledge in monitoring, assessing, analyzing and/or evaluating processes and data. Identifies policy gaps and formulates policies. Interprets data and makes recommendations. Researches and interprets factual information. Identifies inconsistencies in data or results, defines business issues and formulates recommendations on policies, procedures or practices. Integrates established disciplinary knowledge within own specialty area with basic understanding of related industry practices. Good understanding of how the team interacts with others in accomplishing the objectives of the area. Develops working knowledge of industry practices and standards. Limited but direct impact on the business through the quality of the tasks/services provided. Impact of the job holder is restricted to own team.

Responsibilities:

• The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working with tech leads, product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.

• Responsible for design and development of big data solutions. Partner with domain experts, product managers, analyst, and data scientists to develop Big Data pipelines in Hadoop
• Responsible for moving all legacy workloads to cloud platform
• Work with data scientist to build Client pipelines using heterogeneous sources and provide engineering services for data science applications
• Ensure automation through CI/CD across platforms both in cloud and on-premises
• Define needs around maintainability, testability, performance, security, quality and usability for data platform
• Drive implementation, consistent patterns, reusable components, and coding standards for data engineering processes
• Convert SAS based pipelines into languages like PySpark, Scala to execute on Hadoop, Snowflake and non-Hadoop ecosystems
• Tune Big data applications on Hadoop, Cloud and non-Hadoop platforms for optimal performance
• Applies in-depth understanding of how data analytics collectively integrate within the sub-function as well as coordinates and contributes to the objectives of the entire function.
• Produces detailed analysis of issues where the best course of action is not evident from the information available, but actions must be recommended/taken.
• Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.

Qualifications:

• 2-4 years of total IT experience
• Experience with Hadoop (Cloudera)/big data technologies /Cloud/AI tools
• Hands-on experience with HDFS, MapReduce, Hive, Impala, Spark, Kafka, Kudu, Kubernetes, Dashboard tools, Snowflake builts, AWS tools, AI/ML libraries and tools, etc) 
• Experience on designing and developing Data Pipelines for Data Ingestion or Transformation.
• System level understanding - Data structures, algorithms, distributed storage & compute tools, SQL expertise, Shell scripting, Schedule tools, Scrum/Agile methodologies.
• Can-do attitude on solving complex business problems, good interpersonal and teamwork skills

Education:

• Bachelor’s/University degree or equivalent experience