IN-Senior Associate _Azure DE + Databricks Developer_Data & Analytics _Advisory _PAN India

Line of Service

Industry/Sector

Specialism

Management Level

Job Description & Summary

*Why PWC

At PwC, you will be part of a vibrant community of solvers that leads with trust and creates distinctive outcomes for our clients and communities. This purpose-led and values-driven work, powered by technology in an environment that drives innovation, will enable you to make a tangible impact in the real world. We reward your contributions, support your wellbeing, and offer inclusive benefits, flexibility programmes and mentorship that will help you thrive in work and life. Together, we grow, learn, care, collaborate, and create a future of infinite experiences for each other. Learn more about us.

At PwC, we believe in providing equal employment opportunities, without any discrimination on the grounds of gender, ethnic background, age, disability, marital status, sexual orientation, pregnancy, gender identity or expression, religion or other beliefs, perceived differences and status protected by law. We strive to create an environment where each one of our people can bring their true selves and contribute to their personal growth and the firm’s growth. To enable this, we have zero tolerance for any discrimination and harassment based on the above considerations. "

Responsibilities:

1. Data Lake and Lakehouse Implementation:

o Design, implement, and manage Data Lake and Lakehouse architectures. (Must have)

o Develop and maintain scalable data pipelines and workflows. (Must have)

o Utilize Azure Data Lake Services (ADLS) for data storage and management. (Must have)

o Knowledge on Medalion Architecture, Delta Format. (Must have)

2. Data Processing and Transformation:

o Use PySpark for data processing and transformations. (Must have)

o Implement Delta Live Tables for real-time data processing and analytics. (Good to have)

o Ensure data quality and consistency across all stages of the data lifecycle. (Must have)

3. Data Management and Governance:

o Employ Unity Catalog for data governance and metadata management. (Good to have)

o Ensure robust data security and compliance with industry standards. (Must have)

4. Data Integration:

o Extract, transform, and load (ETL) data from multiple sources (Must have) including SAP (Good to have), Dynamics 365 (Good to have), and other systems.

o Utilize Azure Data Factory (ADF) and Synapse Analytics for data integration and orchestration. (Must have)

o Performance Optimization of the Jobs. (Must have)

5. Data Storage and Access:

o Implement and manage Azure Data Lake Storage (ADLS) for large-scale data storage. (Must have)

o Optimize data storage and retrieval processes for performance and cost-efficiency. (Must have)

6. Collaboration and Communication:

o Work closely with data scientists, analysts, and other stakeholders to understand data requirements. (Must have)

o Provide technical guidance and mentorship to junior team members. (Good to have)

7. Continuous Improvement:

o Stay updated with the latest industry trends and technologies in data engineering and cloud computing. (Good to have)

o Continuously improve data processes and infrastructure for efficiency and scalability. (Must have)

Required Skills and Qualifications:

· Technical Skills:

o Proficient in PySpark and Python for data processing and analysis.

o Strong experience with Azure Data Lake Services (ADLS) and Data Lake architecture.

o Hands-on experience with Databricks for data engineering and analytics.

o Knowledge of Unity Catalog for data governance.

o Expertise in Delta Live Tables for real-time data processing.

o Familiarity with Azure Fabric for data integration and orchestration.

o Proficient in Azure Data Factory (ADF) and Synapse Analytics for ETL and data warehousing.

o Experience in pulling data from multiple sources like SAP, Dynamics 365, and others.

· Soft Skills:

o Excellent problem-solving and analytical skills.

o Strong communication and collaboration abilities.

o Ability to work independently and as part of a team.

o Attention to detail and commitment to data accuracy and quality.

Certifications required:

· Certification in Azure Data Engineering or relevant Azure certifications.

o DP203 (Must have)

· Certification in Databricks.

o Databricks certified Data Engineer Associate (Must have)

o Databricks certified Data Engineer Professional (Good Have)

Mandatory skill sets:

Azure DE, Pyspark, Databricks

Preferred skill sets:

Azure DE, Pyspark, Databricks

Years of experience required:

5-10 Years

Education Qualification

BE, B.Tech, MCA, M.Tech

Education (if blank, degree and/or field of study not specified)

Certifications (if blank, certifications not specified)

Required Skills

Optional Skills

Desired Languages (If blank, desired languages not specified)

Travel Requirements

Available for Work Visa Sponsorship?

Government Clearance Required?

Job Posting End Date