Senior Data Engineer - Databricks

Role: Databricks Lead Engineer

Location: London (Hybrid / On-site requirement: 3 days per week in London )

Rate: Up to £575/day (Inside IR35)

Contract Length: 6 months with potential extension

Start Date: ASAP

Role Summary We are seeking an experienced Databricks Lead Engineer to drive the design, development, and optimization of enterprise-scale data platforms. The ideal candidate will have strong technical leadership capabilities, deep expertise in the Databricks Lakehouse platform, and a proven track record of delivering robust data pipelines and architecture in complex environments.

Key Responsibilities Lead the design and implementation of scalable data architectures on the Databricks Lakehouse platform.

Oversee the development of efficient ETL/ELT pipelines using PySpark, SQL, and Delta Lake.

Mentor and guide junior data engineers, promoting best practices in code quality, CI/CD, and performance tuning.

Collaborate with data architects, analysts, and stakeholders to translate business requirements into technical data solutions.

Ensure data security, governance, and quality standards are maintained across all data workflows.

Attend the London office for 3 days per week for collaborative working and stakeholder engagement.

Essential Requirements & Skills Databricks Expertise: Extensive hands-on experience with Databricks, Delta Lake, Unity Catalog, and Spark optimization.

Programming Languages: Advanced proficiency in Python/PySpark and SQL.

Cloud & Infrastructure: Strong background in cloud data platforms (Azure, AWS, or GCP) and cloud-native architectures.

Engineering Best Practices: Experience with CI/CD pipelines, version control (Git), and infrastructure-as-code.

Leadership: Demonstrated experience leading engineering teams, managing workstreams, and driving technical delivery.

Contract & Commercial Terms Rate: Up to £575 per day.

Tax Status: Inside IR35.

Duration: 6 months (with scope for extension).

AMRT1_UKTJ

...