Databricks Platform Engineer

Platform Architecture & Engineering responsibilities:

Design and implement scalable Databricks Lakehouse platforms on AWS and/or Azure aligned to client requirements

Architect end-to-end data platforms including ingestion, storage (Delta Lake), processing, and consumption layers

Build and configure cloud infrastructure using infrastructure-as-code (e.g. Terraform & Declarative Automation Bundles(DAB's))

Establish secure, compliant environments including networking (VNet/VPC, Private Link), identity (IAM/Entra ID), data governance (Unity Catalog), and access controls

Define environment strategies (dev/test/prod), CI/CD pipelines, and release processes for Databricks deployments

Implement monitoring, logging, cost optimisation, and performance tuning across the platform

Design and implement data pipelines using Delta Live Tables, Auto Loader, and Databricks Workflows for both batch and streaming workloads

Client Delivery & Enablement responsibilities:

Work directly with clients to translate business and technical requirements into scalable platform designs

Lead technical workshops, architecture sessions, and whiteboarding engagements with client stakeholders

Support rapid prototyping and proof-of-concept builds within Databricks to demonstrate platform capabilities and accelerate client adoption

Provide best practice guidance on Lakehouse architecture, data modelling, workload optimisation, and cost management

Produce high-quality technical documentation including architecture diagrams, architecture decision records (ADRs), runbooks, and deployment guides

Enable client teams through structured knowledge transfer, training, and platform handover

Collaborate with data engineers, data scientists, and product teams to ensure successful delivery outcomes

Governance & Security:

Implement Unity Catalog for centralised data governance, including access control (RBAC/ABAC), data lineage, audit logging, and compliance enforcement

Apply security best practices across platform design: network isolation, secret management, encryption at rest and in transit, and identity federation

Ensure platform designs meet client regulatory and compliance requirements (e.g. GD...