Data Engineer

We are on the lookout of Senior Data Engineers ( 4 Headcount), where you will be a part of a highly qualified team, leading the delivery of modern data platforms using Azure Services, Databricks, Synapse

Role:

• Define and design scalable scraping architecture, data ingestion pipelines, storage strategies, and monitoring frameworks.
• Establish engineering standards for resilience, observability, error handling, and performance optimization.
• Review and validate production-grade code, scraping logic, system design, and CI/CD practices. Implement monitoring and alerting frameworks to ensure data reliability and SLA adherence. Ensure structured delivery planning, sprint governance, and milestone alignment.
• Designing and delivering software using an agile and iterative approach based on Scrum or Kanban
• Supporting the Lead Data Engineer in technical architecture and design
• Owning their own learning so that they remain a technical subject matter expert.

Technical Skills:

• Having solid experience in data modeling, data warehousing, SQL Server database design and development (SQL, relational and dimensional modelling) Analytical querying using technologies such as SSAS cubes or columnar data stores, Python.
• ETL tools such as SSIS, Alteryx, Azure Data Factory
• Building data pipelines using code e.g. Pyspark
• Modern cloud-based data architectures and technologies such as Data Lake, Delta Lake, Data Lakehouse e.g. Azure Data Lake Gen 2, Azure Delta Lake Lakehouse architecture using Databricks, Hudi or Iceberg
• Python scripting, including data engineering and machine learning libraries
• Scripting languages such as Powershell, bash, DAX, M-language etc. (Note - We primarily use SQL, Pyspark and Python in our day-to-day work along with some coding around DAX, Bash, Powershell and other scripting languages. )
• Distributed computing mainly around Spark, Big data files formats e.g. Parquet, Avro, ORC etc.
• Big data technologies such as Hadoop, Hive, Map-Reduce, Partition design, performance tuning in Spark etc.
• Reporting and visualisation tools such as SSRS, Tableau, Power BI or Excel.
• Work Location: Bengaluru, Neurones IT