Data Engineer

About Catalytics Datum

Catalytics Datum is the Next-Gen Enterprise that amalgamates Data Science, Big Data, Cloud Computing & Business Intelligence to solve complex business problems for enterprises through user experience and faster decision-making. Recognized by Microsoft BizSpark, Catalytics is present across the globe to become your partner in Digital Transformation.

Catalytics Datum offers Platform as a Service, which is One Stop Solution. The complete process; starting from Requirement Gathering to the Final Deployment, is data-driven, processed by collaborative and different Predictive modeling tools which leave clients overhead free. We provide up to 99.9% accurate results in order to increase profitability by providing the deepest insights of your brands.

Role Summary

We are looking for a Data Engineer (1-4 years experience) to design, build, and maintain scalable data pipelines, data warehouses, and data lakes that power analytics, machine learning, and reporting. Youll work closely with analytics, data science, and business teams to ensure data is reliable, accessible, and production-ready.

Skills & Competencies

Role Overview:

Design, build, and maintain scalable data pipelines, warehouses, and lakes. Enable data accessibility for analytics, ML, and reporting. Mapped Skills / Tools: Cloud Data Platforms (Azure, AWS, GCP), Data Engineering Fundamentals.

Core Responsibilities:

Develop & manage ETL/ELT pipelines; Build and optimize data warehouses & lakes; Ensure data quality & governance; Enable real-time & batch processing; Support AI/ML model data needs. Mapped Skills / Tools: SQL, Python, Scala, Databricks, Spark, Hadoop, Kafka, Azure Data Factory, dbt, Airflow.

Data Modeling:

Design schemas & data models; Optimize queries & storage; Implement star/snowflake schemas. Mapped Skills / Tools: SQL, Relational DBs, NoSQL, Data Vault, Dimensional Modeling.

Data Governance:

Implement access controls, lineage, cataloging; Ensure compliance with GDPR, HIPAA, SOC2 (awareness level). Mapped Skills / Tools: Azure Purview, Unity Catalog, Collibra, Data Quality/Observability Tools.

Infrastructure & Ops:

Build CI/CD for data pipelines; Monitor cost, performance & reliability. Mapped Skills / Tools: DevOps, GitHub/GitLab, Docker, Kubernetes (basic), Terraform (basic), Azure DevOps.

Collaboration:

Work with analysts, data scientists, and business teams; Translate requirements into data solutions. Mapped Skills / Tools: Agile/Scrum, JIRA, Confluence, Business Communication.

Required Skills:

Strong SQL, Python, and/or Scala; Hands-on cloud data experience; Knowledge of big data frameworks; ETL/ELT pipeline development. Mapped Skills / Tools: SQL, Python, Scala, Spark, Hadoop, Airflow, Azure Data Factory, AWS Glue, dbt.

Preferred Skills:

Experience with Databricks & Delta Lake; Exposure to ML pipelines; Familiarity with APIs & microservices; Knowledge of observability tools. Mapped Skills / Tools: Databricks, Delta Lake, MLflow, Ragas, Great Expectations, REST APIs, GraphQL.

Soft Skills:

Strong communication & teamwork; Analytical problem solving; Ownership & accountability; Ability to explain the data journey (raw curated consumed) to technical & non-technical stakeholders using PPT. Mapped Skills / Tools: Collaboration, Agile mindset, Documentation, Critical Thinking.

Key Responsibilities

• Build and maintain ETL/ELT pipelines for batch and near real-time data
• Develop and optimize data warehouse and data lake structures
• Ensure data quality, reliability, and basic governance controls
• Support data consumption for analytics and ML use cases
• Monitor pipeline health, performance, and costs
• Collaborate with analysts and data scientists to deliver trusted datasets
• Contribute to CI/CD pipelines and basic DevOps practices for data

Minimum Qualifications

• 12 years of hands-on experience in Data Engineering or Analytics Engineering
• Strong SQL and working knowledge of Python (or Scala)
• Experience working on at least one cloud platform (Azure/AWS/GCP)
• Exposure to building and maintaining ETL/ELT pipelines
• Familiarity with data warehouses/lakes and dimensional modeling basics

Nice-to-Have Qualifications

• Hands-on exposure to Databricks & Delta Lake
• Familiarity with Airflow/dbt for orchestration and transformations
• Exposure to ML data pipelines or feature engineering
• Understanding of data observability and quality frameworks

What Success Looks Like (First 6 Months)

• Reliable data pipelines running in production with low failure rates
• Clean, well-documented data models adopted by analytics teams
• Improved data freshness and quality metrics
• Positive feedback from analysts/data scientists on dataset usability
• Increased ownership of one end-to-end data pipeline