Senior Data Engineer

Job Title: Senior Data Engineer

Location: Bengaluru

Job Type: Full-Time

Immediate joiners or notice period of less than 10days are needed.

Job Role:

As a Senior Data Engineer at LogixHealth, you will work with a globally distributed team of engineers to design and build cutting-edge solutions that directly improve the healthcare industry. Youll contribute to our fast-paced, collaborative environment and bring your expertise to continue delivering innovative technology solutions, while mentoring others.

Duties and Responsibilities:

1. Lead and contribute to the creation of a self-service data platform for reporting and analytics
2. Design and build data solutions using Databricks, SQL, Python, Spark, and Delta Lake in the Azure ecosystem (Blob Storage, Data Factory, Event Hubs)
3. Ensure best practices for ETL / ELT processes (data quality management, data processing, data partitioning, maintainability, and reusability)
4. Collaborate with engineers, product, and business leaders to ensure data platform is integrated with other systems and technologies (Tableau, Power BI, APIs, custom applications)
5. Establish CI/CD processes, test frameworks, infrastructure-as-code tools, and monitoring/alerting (Git, Terraform, Azure DevOps / GitHub Actions / Jenkins, Azure Monitor / Datadog)
6. Adhere to the Code of Conduct and be familiar with all compliance policies and procedures stored in LogixGarden relevant to this position

Qualifications:

To perform this job successfully, an individual must be able to perform each duty satisfactorily. The requirements listed below are representative of the knowledge, skills, and/or ability required. Reasonable accommodation may be made to enable individuals with disabilities to perform the duties.

Education (Degrees, Certificates, Licenses, Etc.)

BE (or higher, MS/PhD) degree in computer science/related field, or equivalent technical experience.

Experience:

1. Experience with native and third-party Databricks integrations (Delta Live Tables, Auto Loader, Databricks Workflows/Apache Airflow, Unity Catalog)
2. 8+ years data engineering experience
3. 3+ years in a Senior, Staff or Principal Engineer role
4. Experience designing scalable data pipelines
5. Experience leading projects within a team and across teams
6. Azure experience preferred
7. Azure Databricks implementation experience preferred
8. Experience designing and implementing data security and governance platform adhering to compliance standards (HIPPA, SOC 2) preferred

Specific Job Knowledge, Skill and Ability:

1. Strong programming skills in pySpark/Scala
2. Strong Dataframe programming skills, such as Spark, Pandas, NumPy
3. Passion for mentoring and guiding others
4. Strong written and verbal communication skills
5. Expert knowledge in architecting, designing, and implementing data solutions to serve the needs of our business processes and software products
6. Ability to keep security, maintainability, and scalability in mind with the solutions built
7. Possess excellent interpersonal communication skills and an aptitude for continued learning