Databricks Lead – Delivery & Engineering

About this role

Databricks Lead – Delivery & Engineering1

Lead hands-on Databricks delivery, ensuring adherence to CoE standards and best practices.

KEY RESPONSIBILITIES

• Lead Databricks engineering teams

• Build and optimize Spark & Delta Lake pipelines

• Implement Databricks Jobs, Workflows, CI/CD

• Optimize performance and DBU usage

• Contribute to Databricks accelerators and standards

• Design and implement end-to-end data platforms using Databricks Lakehouse

• Lead development of:

• Data ingestion (batch & streaming)

• Delta Lake data modeling (Bronze/Silver/Gold)

• Scalable ETL/ELT pipelines

• Optimize Spark workloads for performance and cost

• Ensure reliability, scalability, and observability of data pipelines

EXPERIENCE

• 10–12 years in data engineering

• 3–5 years hands-on Databricks experience

SKILLS

• Deep expertise in Databricks, Apache Spark, Delta Lake

• Strong hands-on experience with PySpark / Spark SQL  (core, SQL, streaming)

• Experience designing Lakehouse architectures

• Strong understanding of data modeling, data quality, and lineage

• Databricks platform features

• Cloud-native data engineering

• Performance tuning & troubleshooting

Certifications (Preferred)

• Databricks Data Engineer Associate and Professional