Senior Data Engineer

Our client is a public sector technology consultancy.

Description Our Senior Data Engineers enable public sector organisations to embrace a data-driven approach by providing data platforms and services that are high-quality, cost-efficient, and tailored to clients' needs. They develop, operate, and maintain these services. They make sure they provide maximum value to data consumers, including analysts, scientists, and business stakeholders.

Your role at our client As a Senior Data Engineer, you may play one or more roles according to our clients' needs. The role is very hands-on and you'll support as a senior contributor for a project, focusing on both delivering engineering work as well as upskilling members of the client team. At other points, you might play more of a technical architect role and work with the larger MadeTech team to identify growth opportunities within the account.

You'll need to have a drive to deliver outcomes for users. You'll make sure that the wider context of a delivery is considered and maintain alignment between the operational and analytical aspects of the engineering solution.

Skills, Knowledge & Expertise We are looking for candidates with a range of skills and experience, please apply even if you don't meet all the criteria.

Enthusiasm for learning and self-development

Proficiency in Git (inc. Github Actions) and able to explain the benefits of different branch strategies

Gathering and meeting the requirements of both clients and users on a data project

Strong experience in IaC and able to guide how one could deploy infrastructure into different environments

Owning the cloud infrastructure underpinning data systems through a DevOps approach

Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop

Good understanding of the possible architectures involved in modern data system design (e.g. Data Warehouse, Data Lakes and Data Meshes) and the different use cases for them

Ability to create data pipelines on a cloud environment and integrate error handling within these pipelines. With an understanding how to create reusable libraries to encourage uniformity of approach across multiple data pipelines.

Able to document and present an end-to-end diagram to explain a data processing system on a cloud environment, with some knowledg...