Senior Data Engineer

Company Description

Texplorers Inc. is a leader in digital transformation for global enterprises, with over 15 years of experience in cloud technology and AI expertise. The company focuses on empowering businesses to scale efficiently and continuously improve through their innovation ecosystem. Texplorers Inc. is committed to innovation, sustainability, and inclusion, ensuring perpetual learning and growth for their clients.

We are seeking candidates who can join us immediately.

We are seeking multiple experienced Data Engineers to design, build, and optimize scalable big data solutions. This role involves developing high-performance ETL pipelines using Apache Spark across major cloud platforms (Azure, AWS, or GCP), with a strong emphasis on data engineering best practices and system performance.

Key Responsibilities:

• Design, develop, and optimize ETL pipelines in Databricks.
• Build data ingestion, transformation, and processing workflows using Apache Spark.
• Work with structured and unstructured data from various sources.
• Optimize Spark jobs for performance, cost efficiency, and scalability.
• Implement CI/CD for Databricks workflows and manage version control.
• Work closely with Data Engineers, Data Scientists, and Business Analysts to support analytics and reporting.
• Implement security best practices and manage access controls in Databricks.
• Troubleshoot performance and scalability issues in big data processing.

Required Skills:

• Strong hands-on experience with Databricks and Apache Spark (PySpark/Scala/Java).
• Proficiency in SQL for data analysis and transformation.
• Experience in Azure cloud platforms
• Hands-on experience with ETL tools, Data Pipelines, and Data Warehousing.
• Strong knowledge of CI/CD, Git, and DevOps tools for data workflows.
• Knowledge of Delta Lake, Lakehouse architecture, and performance tuning.
• Experience in handling large-scale datasets and distributed computing.
• Experience with Azure synapse

Preferred Skills:

• Experience with Databricks MLflow and machine learning pipelines.
• Knowledge of Kafka, Airflow, or Snowflake.
• Understanding of data governance, compliance, and security policies.

Education & Experience:

• Bachelors or Masters degree in Computer Science, Data Engineering, or a related field.
• 5-7 years of Experience in Databricks development
• Experience in Retail industry is mandatory