Data Engineer

Insight Global is seeking a highly experienced Senior Data Engineer to design, build, and optimize data pipelines that integrate metadata and lineage between Snowflake and Collibra. This engineer will play a key role in enabling enterprise-grade data governance by automating metadata ingestion, synchronization, and lineage flows across cloud data platforms and governance tools. This is a hands-on engineering role focused on building metadata pipelines, API-based integrations, and scalable orchestration workflows that ensure Collibra always has accurate, uptodate information from Snowflake.

Must Haves:

• Strong Data Engineering experience
• Expert-level Python for API integration, metadata processing, and automation
• Strong SQL experience with Snowflake (SnowSQL, query optimization, access patterns)
• Experience working with Collibra APIs, import/export workflows, and governance operating models
• Proficiency with REST APIs, JSON, and metadata transformation
• Strong understanding of ETL/ELT patterns, data warehousing concepts, and cloud-native architectures
• Experience with version control, CI/CD, and DevOps tooling (GitHub Actions, Azure DevOps, Jenkins, etc.)
• Familiarity with metadata concepts: business glossary, data domains, assets, schemas, lineage
• Understanding of data quality frameworks and standards

Plusses:

• Experience with Snowflake metadata tables, INFORMATION_SCHEMA, and ACCOUNT_USAGE views
• Hands-on work with Collibra Workflow Designer, Collibra import jobs, or custom integration scripts
• Familiarity with DBT, Databricks, or cloud data engineering (AWS, Azure, GCP)
• Background in highly regulated industries (financial services, energy, government, healthcare)