Databricks Data Engineer

About the role

The team is looking for a Databricks Data Engineer to build and operate enterprise‑grade data lakehouses. You will work closely with data scientists and analysts to deliver reliable, high‑quality data assets that power analytics and machine‑learning workloads.

Key responsibilities

• Design, develop, deploy and maintain scalable data pipelines on the Databricks platform.
• Build and optimise Delta Lake‑based lakehouses, including data modelling, cleansing, transformation and integration.
• Implement performance enhancements such as file indexing, small‑file compaction, OPTIMIZE and Z‑ORDER.
• Use Databricks Workflows, Apache Airflow and related tools for scheduling, monitoring and alerting of data jobs.
• Collaborate with data scientists and analysts to provide trustworthy data sets and support ML model deployment.
• Apply data‑governance best practices with Unity Catalog for metadata management, access control and lineage tracking.

Required profile

• Bachelor’s degree or higher in Computer Science, Information Technology or a related discipline.
• Proven experience in big‑data development and commercial projects using Databricks.
• Strong ability to write complex SQL queries and develop data transformation logic.
• Good English reading and writing skills for technical documentation and team collaboration.

Required skills

• Databricks platform (workspace management, cluster configuration, notebook development, job scheduling).
• PySpark and Spark SQL for data processing.
• Delta Lake operations (MERGE INTO, OPTIMIZE, Z‑ORDER, transactional features).
• SQL proficiency.
• Unity Catalog for metadata and access control.
• Databricks Workflows and Apache Airflow for orchestration.
• Familiarity with MLflow, Azure, AWS or GCP is a plus.