Senior Data Engineer

Exciting opportunity for a Senior Data Engineer to own the maintenance, expansion, and performance of Databricks-based data platform. This is a hands-on role, you'll be writing pipeline code, building jobs, and modelling as the main part of your day to day.

The Role

You will be the technical expert for the data solution, to work alongside the engineering lead to manage the data aspects of the digital solution. You will be designing and building data pipelines, working with source systems, developing Databricks jobs, manipulating and modelling data, and making technical decisions about how data should be managed and structured. You’ll work closely with the wider engineering team to understand what the business needs from its data, what the end goal is, and how best to get there.

Key responsibilities will include:

Designing, building, and operating Databricks Pipelines and Jobs

Designing and building Delta Lake pipelines

Owning Databricks deployment bundles, packaging, versioning, and promoting pipeline code and job definitions

Building and maintaining environment-variable-driven Azure Data Factory pipelines

Building the data layer that feeds Microsoft Fabric/OneLake

Monitoring and troubleshooting data pipelines for performance, reliability and cost.

Taking ownership of data solutions from initial source-system investigation through to the final business outcome.

About You

We’re looking for someone who can demonstrate real-world, hands-on data engineering experience.

You’ll ideally have:

Strong, demonstrable experience working with Azure data platforms in production.

Hands-on experience building and running Databricks Pipelines and Jobs in a production environment. Candidates with a strong background in Snowflake and drive to learn Databricks also encouraged to apply.

Strong data modelling and data manipulation skills, with the ability to take raw source data and turn it into sensible, structured datasets.

Strong data modelling ability, able to look at raw source data and design sensible, well-structured silver/gold tables

Strong understanding of medallion (bronze/silver/gold) design patterns, and data architecture right at table level

Solid Delta Lake knowledge, deletion vectors, time travel, optimization, compatibility considerations

Experience with Databricks Asset Bundles or equivalent deployment/CI practices for data pipelines

Practical experience building environment-variable/parameter-driven Azure Data Factory pipelines

Proven experience of managing large data architectures

Strong SQL, and comfort in at least one of Python/Scala for pipeline logic

If you’re a Senior Data Engineer who enjoys getting into the detail, solving complex data problems and having genuine ownership of the solution, we’d be keen to hear from you.

FPSG is committed to equal opportunities regardless of gender, race, disability, sexual orientation, religion or belief and age.

#J-18808-Ljbffr