Data AEP Senior Engineer

Senior Data Engineer- AEP

We are combining data, programmatic decisioning, and real-time delivery to build Customer Data Platforms (CDPs), complimented by a suite of client-specific managed services.

Adobe Experience Platform, is powerful, flexible, open system for bulding and managing complete solutions that drive customer experience. Experience platform enables organizations to centralize and standardize customer data and content from any system and apply data science and machine learning to deliver consistent interactions at scale across several orchestration layers within any marketing stack.

Our engineering team is responsible for building the Adobe Experience Platform. We help thousands of customers collect, manage and synthesize petabytes of data with high fidelity and velocity. We help them make sense of that data, glean insights and run algorithms, so they can deliver delightful experiences in real-time, every time. If building new secure micro-services in the cloud based on open-source technologies at large-scale is something that might interest you, lets share some thoughts, we would like to meet!

Work youll do

We rely on powerfully insightful data to inform our systems and solutionsand were seeking an experienced pipeline-centric Data Engineer to put it to use. Our ideal hire will have the mathematical and statistical expertise youd expect, combined with a rare curiosity and creativity. Youll wear many hats in this role, but much of your focus will be building out our Python ETL processes and writing superb SQL. Beyond technical prowess, youll need the soft skills it takes to communicate highly complex data trends to organizational leaders in a way thats easy to understand. Were looking for someone willing to jump right in to help the company get the most out of our data.

Your responsibilities will include:

• Analyze customer use case, available data to prepare for automated & ongoing data ingestion to meet customer requirements
• Design technical solution with all data ingestion, transformation & scheduling to review & sign-off with customer
• Identifying AEP enhancements, extending frameworks and incorporating new ideas
• Closely collaborating with other AEP team members (sales teams, engineers, consultants) and onshore teams for delivering projects
• Enterprise level software development leveraging Big Data, Cloud technologies & Python
• Building/operating highly scalable, fault tolerant, distributed systems for extraction, ingestion to process large data sets
• Experience with data analysis, modeling and mapping to coordinate closely with Data Architect(s)
• Build the necessary schemas, workflows to ingest customers data into AEP successfully
• Create necessary identity namespaces, privacy settings and merge poilicys required to build the solution
• Build Audiences (segmentations) and create necessary pipeline for Destination activation
• Deploying the final solution to a production environment (or end-state environment)
• Post-Deployment, provide ongoing maintenance & support of solution & knowledge transfer to offshore support team

Our team

Youll join a team of passionate. As an engineer in digital marketing and digital media, youll get an unprecedented opportunity to work at the intersection of creative content, data science, and experience delivery.  If youre intellectually curious, hardworking and solution-oriented, youll fit right into our fast-paced, collaborative environment. In addition to working daily with our DataScience Team and DevOps Team to deliver production-level pipelines that will run unattended for weeks and months. youll also work closely with our Project Management Team to understand our clients needs and any change requests that drive our development efforts.

Qualifications

Required:

• Graduate/ Postgraduate in Computer Science/ Engineering or related fields from a renowned University
• 5+ years experience ETL/Data warehousing Solutions(Talend, Informatica)
• 5+ years experience with Real-time processing Framework (Apache Spark)
• 2 to 3 year of experience with Adobe Experience Platform (Project or POC),
• Database architectures

• 5+ years experience SQL-based technologies (e.g. MySQL/ Oracle DB)
• 3+ years experience NoSQL technologies (e.g. Cassandra and MongoDB)
• Programming Languages: Python/R/Java
• Knowledge of different Operating System UNIX, Linux, Solaris or MS Windows
• Knowledge of SOA, Graph Database, CI/CD Pipeline, monitoring & alert would be a plus point.

       Passionate about testing, and with extensive experience in Agile teams using SCRUM; you consider automated build-and-test to be the norm.

       Proven ability to communicate in both verbal and writing in a high performance, collaborative environment.

       Follows data development best practices, and enjoy helping others learn to do the same.

       An independent thinker who considers the operating context of what he/she is developing.

       Believes that the best data pipelines run unattended for weeks and months on end.

       Familiar with version control, you believe that code reviews help to catch bugs, improves code base and spread knowledge.

       Ability to travel 5-10% of the time

Helpful, but not required:

       Knowledge in:

o   Experience with large consumer data sets used in performance marketing is a major advantage.

o   Familiarity with machine learning libraries is a plus

o   Well-versed in (or contributes to) data-centric open source projects.

o   Reads Hacker News, blogs, or stays on top of emerging tools in some other way

o   Data visualization

o   Experience with stream-processing systems: Apache Spark-Streaming, Apache Storm etc.

o   Experience working in open table / in-memory table formats for huge analytics dataset: Iceberg, Parquet, Arrow, AVRO etc.

       Adobe Qualified Adobe Experience Platform (AEP) Certification

       Technologies of Interest:

o   Languages/Libraries Python, Java, Scala, Spark, Kafka, Hadoop, HDFS, Parquet.

o   Cloud AWS, Azure, Google

About Company

Deloitte is a global professional services firm that provides a wide range of services, including audit and assurance, consulting, tax, risk management, and financial advisory. With a presence in over 150 countries and a network of member firms, Deloitte serves clients across various industries, helping them solve complex business challenges, improve operations, and innovate. Known for its expertise in management consulting, technology solutions, and strategy, Deloitte is one of the Big Four accounting firms and is recognized for its commitment to quality, integrity, and making an impact in the marketplace.