Senior Data Specialist

Career Area:

Job Description:

Your Work Shapes the World at Caterpillar Inc.

When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other.  We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't just talk about progress and innovation here – we make it happen, with our customers, where we work and live. Together, we are building a better world, so we can all enjoy living in it.

Position Overview

Highly skilled Senior Microsoft Fabric Developer with strong expertise in ETL development, data engineering, and modern data platform solutions. The ideal candidate will design, build, and optimize enterprise-grade data pipelines, Lakehouses, and data integration solutions using Microsoft Fabric, along with a deep understanding of Azure data services and end-to-end data lifecycle management. This role requires hands-on experience in Microsoft Fabric Data Factory (preferred-must have), Azure Data Factory, Azure Data Lake, Dataflows Gen2, Lakehouses, Delta tables, and robust data modelling techniques. The candidate should be capable of architecting scalable data solutions, driving data quality, and supporting analytics and BI workloads across the organization.

Key Responsibilities

Microsoft Fabric & Data Engineering

• Design, build, and maintain ETL/ELT pipelines using Microsoft Fabric Data Factory (preferred), Azure Data Factory, Azure Data Lake Dataflows Gen2, Lakehouses, Warehouses, and Notebooks (PySpark / Spark SQL).
• Develop and optimize end-to-end data ingestion frameworks from various structured & unstructured sources.
• Implement Delta Lake best practices including versioning, schema evolution, and ACID transactions.
• Build scalable data models supporting analytics, BI, and downstream applications.
• Manage data transformation logic, metadata, lineage, and governance per enterprise standards.
• Optimize data pipelines for performance, reliability, and cost efficiency.

Azure Data Platform

• Build integrations with Azure Storage, Azure SQL, Azure Data Lake Storage, Key Vault, and event-based ingestion.
• Deploy and manage solutions in Azure using DevOps pipelines (CI/CD).
• Monitor workloads using Fabric Monitoring Hub, Azure Monitor, and Log Analytics.

Collaboration & Delivery

• Work closely with business stakeholders, BI teams, and data consumers to understand data needs.
• Provide technical leadership on Fabric adoption, architecture, and engineering best practices.
• Participate in Agile ceremonies, sprint planning, and technical design discussions.

Required Skills & Qualifications

• Bachelor’s degree in Computer Science, Data Engineering, Information Technology, or related field.
• 6–10 years of experience in data engineering, ETL development, or cloud data platforms.
• Strong practical experience with Microsoft Fabric Data Factory, including Azure Data Factory, Azure Data lake, Dataflows Gen2, Lakehouses, Warehouses, and Notebooks (PySpark).
• Deep knowledge of ETL/ELT patterns, data modelling, SQL optimization, and data warehousing.
• Expert-level SQL skills (T-SQL, Spark SQL).
• Hands-on experience with Azure Data Lake, Azure SQL / Synapse, Azure Storage, Azure Active Directory, Key Vault.
• Strong understanding of modern data architectures (Lakehouse, Medallion Architecture).
• Experience with Azure DevOps (Repos, Pipelines).
• Solid understanding of data quality, governance, lineage, and security practices.

Preferred Skills

• Experience in semantic models, and Fabric warehouse connectivity.
• Experience implementing CI/CD for data pipelines.
• Familiarity with Infrastructure-as-Code (Terraform, Bicep, ARM templates).
• Knowledge of real-time ingestion tools (Event Hub, Stream Analytics).
• Exposure to Python, .NET, or APIs for integration purposes.
• Strong communication and documentation abilities.

Soft Skills

• Strong analytical thinking and problem-solving skills.
• Ability to work independently and lead technical discussions.
• Excellent interpersonal, cross-team collaboration, and communication skills.

Posting Dates:

Caterpillar is an Equal Opportunity Employer.  Qualified applicants of any age are encouraged to apply

Not ready to apply? Join our Talent Community.