Databricks Engineer

About the role

We are looking for a skilled Databricks Engineer to design, develop, and maintain robust data pipelines on the Databricks platform. You will build enterprise‑grade data lakehouses, ensure data quality, and support machine‑learning model deployment.

Key responsibilities

• Design, develop, deploy, and maintain data pipelines on Databricks, ensuring efficiency and reliability.
• Build and optimize lakehouses using Delta Lake technology, handling data modeling, cleansing, transformation, and integration.
• Utilize Databricks Workflows, Apache Airflow, and related tools for scheduling, monitoring, and alerting of data tasks.
• Collaborate with data scientists and analysts to provide trustworthy data assets and support ML model lifecycle.
• Apply data‑governance best practices with Unity Catalog for metadata management, access control, and lineage tracking.

Required profile

• Bachelor’s degree or higher in Computer Science, Information Technology, or a related field.
• Hands‑on experience with Databricks in commercial projects, including workspace management, cluster configuration, notebook development, and job scheduling.
• Strong proficiency in PySpark and Spark SQL for complex data transformations.
• Deep understanding of Delta Lake operations such as MERGE INTO, OPTIMIZE, and Z‑ORDER BY.
• Excellent SQL skills and the ability to write and optimize complex queries.
• Good English reading and writing ability for technical documentation and team collaboration.

Required skills

• Databricks platform
• PySpark
• Spark SQL
• Delta Lake (MERGE INTO, OPTIMIZE, Z‑ORDER BY)
• SQL
• Databricks Workflows
• Apache Airflow
• Unity Catalog
• MLflow (optional)
• Azure, AWS, or GCP (optional)