Digital Cloud Engineer

Core Skills

• PySpark – mid-level knowledge (DataFrames, Spark SQL, basic transformations)
• Notebook creation – organizing and collaborating in Databricks
• Data ingestion – loading data from multiple sources
• Data modeling – designing tables and schemas
• Medallion architecture – Bronze → Silver → Gold layers

Databricks Platform

• Workspace & notebooks – collaborative development
• Jobs & workflows – scheduling and automation
• Delta Lake – ACID-compliant storage
• Unity Catalog – data governance and access control

DevOps / Automation

• CI/CD pipelines – Git, GitHub Actions, Azure DevOps

ADF - Azure Data Factory  – pipeline creation, scheduling, and monitoring