Azure data engineer

Role : Sr Azure Data Engineer/ Lead

Exp Range : 5-12 years

Location : PAN India

Must have :

• Deep Microsoft Azure product knowledge and understanding of product features, with exposure to other cloud solutions (Google, AWS) a plus.
• Proven track record in designing, developing and implementing data modelling, at a conceptual, logical and physical level.
• Experience with accountability for E2E Data Architecture and Lifecycle (including non-functional requirements and operations) management.
• Understands industry recognised data modelling patterns and standards
• Understands overall IT system design, in particular networking, authorization and authentication protocols, data security, disaster recovery.
• Knowledge of the latest technologies to process large amount of data (Apache Spark; Databricks is a plus)
• Experience in MDM, Metadata Management, Data Quality and Data Lineage tools.
• Expertise in cloud security and identity best practices
• A deep understanding of ETL and ELT tools and techniques
• Strong experience in architecting fit-for-purpose and fit-for-use complex solutions on the Microsoft Azure cloud stack (IaaS, PaaS and SaaS)
• Very strong hands-on experience in using Microsoft Azure services like Azure Data Factory, Data brick services (like processing streaming data using Spark clusters), usage of Blob containers, ESB, Event Grid, Azure SQL server, Cosmos DB, Azure functions, Analytics (like Power BI) is a mandatory requirement.
• Excellent understanding of architecture patterns and best practices with proven innovative and leading-edge thinking to build cloud-ready systems (e.g., Domain Driven Architecture, multi-tenancy, building for resilience, scalability, performance, and federation, microservices etc.)
• Hands-on work experience on application platforms like .Net, Python, Java etc. preferred
• Understanding of big data platforms concepts and NoSQL technologies, and ability to map such
• The candidate should have experience in modeling data for traditional structured data, as well as information design in Data Lake and NoSQL database engines.
• Fluent with modelling, development and project execution processes around star, snowflake, 3NF, and Data Vault approaches.
• Good experience in working with data using SQLs, Python and Scala in Microsoft based resources like Azure Synapse and Data Lake.
• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.

Good to have :

• A great team player, with the ability to perfectly integrate and play a pivotal role in a team that includes also data scientists, data engineers and data analysts
• Able to appreciate short term vs. long term goals and take both tactical and strategic decisions
• Great communication skills, ability to communicate complex technical concepts to a non-technical audience
• Strong organizational skills, the ideal candidate has the ability to work in a fast-paced environment and has the ability to quickly adapt to changing priorities
• Stays on top of the latest trends and develops expertise in emerging cloud technologies
• Works well as a technical leader and individual contributor.
• Build processes supporting data transformation, data structures, metadata, dependency, and workload management.
• Guide the development of an Agile data development experience, when required using DW automation tools.
• Assist the project team in planning and execution to achieve daily and sprint goals.
• Articulate documentation of project artefacts and deliverables.
• Identifies and communicates risks clearly to the Product owner and the team.
• Education of customer on how the technology recommendation/solution planned will fulfil the customer expressed needs