Data Engineer

Company Description Yantriki AI Technology is an emerging technology company focused on building data-driven and AI-powered solutions for modern businesses. The organization combines advanced engineering practices with practical industry experience to help clients unlock value from their data. Teams at Yantriki AI work with contemporary data platforms, cloud technologies, and analytics tools to deliver scalable solutions. The company values curiosity, collaboration, and continuous learning, offering contributors the opportunity to work on impactful, high-performance systems in a flexible environment.

Role Description This is a remote contract role for a Data Engineer for our esteemed clients. The Data Engineer will design, build, and maintain reliable data pipelines, with a focus on Extract, Transform, Load (ETL) processes and automation. Daily responsibilities include implementing and optimizing data models, integrating data from multiple sources, and ensuring data quality, reliability, and scalability. The role involves collaborating with data analysts, data scientists, and software engineers to support reporting, analytics, and machine learning use cases. The Data Engineer will also monitor data workflows, troubleshoot issues in production environments, and contribute to documentation and best practices for data infrastructure.

3-5 years of Data Engineering experience

2-5 years of cloud-based ETL/ELT pipeline experience (atleast one of - AWS, GCP, Azure )

Key Responsibilities

• Design and develop batch and streaming data pipelines using PySpark.
• Build scalable ETL/ELT pipelines and data infrastructure
• Enable clean, structured data layers for AI/ML systems
• Implement data validation and data quality checks.
• Monitor pipelines using logging, alerting, and observability tools.
• Collaborate with cross-functional teams and document data pipeline architecture.
• Write optimized PySpark code to process large datasets.
• Build and manage CI/CD pipelines for data workflows using GitHub Actions and other tools.

What Were Looking For:

• Strong hands-on engineer with real production deployment experience
• High proficiency in Python (and SQL for data track)
• Experience building APIs, ETL/ELT & data pipelines, and distributed systems

Tech Skills and Tech Stack :-

Understanding and experience of cloud-based data pipelines and query processing solutions is preferred

Data Warehousing / Lakehouse Architecture concepts

Experience in Data Governance & Data Modeling

Hadoop and Spark ecosystem ( Databricks or Cloudera or opensource)

SnowFlake

PySpark

Apache Kafka

Snowflake

Postgres or other SQL database

MongoDB or other NoSQL databases

SQL programming

GitHub (branching, pull requests, code reviews)

Data Validation / Data Quality Frameworks

Pipeline Monitoring (Logging, Alerting, SLA tracking)

Docker / Kubernetes

The role can be taken up as part-time with commitment of 20-25 hours per week. Interested candidates apply for the position.