Azure Databricks Data Engineer

Azure Databricks Data Engineer (Contract) | International Digital Infrastructure Organisation | London (Hybrid)

6-Month Initial Contract | Inside IR35 | 4 Days Onsite, 1 Day Remote

We're partnering with a leading international digital infrastructure organisation undergoing a significant data transformation programme. Operating across multiple markets and supporting critical communications infrastructure, the business is investing heavily in its cloud, data, and analytics capabilities to enable more informed decision-making and unlock greater value from its enterprise data estate.

This is an opportunity to join a high-performing Digital Innovations team at a pivotal stage of that journey. You'll play a key role in building and enhancing a modern Azure Databricks platform that supports business-critical reporting, analytics, and future AI initiatives. Working closely with architects, engineers, and senior stakeholders, you'll help shape the direction of the data platform while delivering scalable, secure, and high-quality data solutions.

What You'll Do

Design, develop, and optimise Azure Databricks solutions using PySpark, Python, SQL, and Azure-native technologies.

Build and maintain enterprise-grade ETL/ELT pipelines supporting ingestion, transformation, CDC, validation, and data delivery across multiple business functions.

Develop and support Lakeflow pipelines, Delta Lake implementations, and modern data engineering frameworks.

Drive best practices across performance optimisation, security, governance, monitoring, and platform reliability.

Collaborate with business, data, and technology teams to deliver high-quality data products that support organisational growth and strategic decision-making.

What You'll Bring

Proven experience designing and delivering solutions within Azure Databricks environments.

Strong Python, PySpark, SQL, and T-SQL development skills.

Expertise building and supporting large-scale ETL/ELT pipelines and cloud-based data platforms.

Strong understanding of Delta Lake, Unity Catalog, data governance principles, and medallion architecture.

Experience integrating REST APIs, JSON data sources, Azure Data Factory, ADLS, Key Vault, and other Azure services.

Hands-on experience w...