Data Engineer

Position Title: Advanced Data Engineer

Contract Type: Contract (6-month initial duration, with potential for extension)

Location: Cardiff, UK

Work Arrangement: Onsite

Hours: 37.5 hours per weekJob Description

Core Technical Skills

Strong proficiency in programming languages such as Python, SQL, and R (experience with R is desirable).

Proven experience working with relational database management systems and data warehousing architectures.

Hands-on expertise using Git-based version control tools (e.g., GitHub, GitLab, Azure DevOps) within collaborative software development environments.

Demonstrated experience operating effectively within Agile delivery frameworks.

Background in designing and deploying cloud-native data architectures, with a preference for Google Cloud Platform (GCP).

Proven track record of migrating legacy on-premises data platforms, pipelines, and services to cloud infrastructure.

Solid grasp of data modeling concepts across conceptual, logical, and physical levels.

Proven capability to design, construct, test, and maintain robust, scalable data pipelines and data products.

Experience establishing CI/CD automation and streamlined deployment workflows.

Experience leveraging container technologies such as Docker.

Skilled in architecting and implementing robust ETL and ELT data pipelines.

Familiarity with modern workflow orchestration frameworks (e.g., Airflow, Cloud Composer, Prefect, Dagster).

Experience processing large-scale datasets using distributed computing engines (e.g., BigQuery, Spark, Dataproc).

Proficiency in monitoring, troubleshooting, and optimizing platform performance and data pipelines.

Solid understanding of DataOps and DevOps methodologies.

Experience integrating systems via APIs and standard data integration techniques.Data Architecture and Engineering Practices

Understanding of how enterprise data platforms interoperate with broader organizational systems.

Practical experience utilizing metadata management solutions, data lineage tools, and data catalogs.

Experience implementing comprehensive data quality profiling, assessment, and validation frameworks.

Demonstrated capability to design, develop, test, document, and support production-ready code and end-to-end data pipelines.

Adept at leading code reviews and applying software engineering best practices, including unit testing, refactoring, and comprehensive technical documentation.Governance, Compliance ...