Senior Data Engineer

We're making energy fair, clean, and simple for all using technology.

Key Responsibilities

• Acting as the flexibility team’s first full-time data engineer, establish the team’s data capability, lead the team’s data engineering technical direction and represent the team to the cross-team data community
• Upskill data-literate backend engineers, act as the team's data expert and as a multiplier on other engineers' impact
• Establish greenfield data engineering systems in a newly established AWS environment
• Provide a foundational layer of high quality data processing and warehousing that will enable future projects for both our team and others
• Maintain and develop critical data applications - scoping and implementing architectural improvements for long-term global scalability, robustness and operational efficiency
• Interface with the established data warehouse, built on Databricks
• Take ownership of data platform improvements that enhance the capabilities for all teams and drive trust in the stability of the system

Requirements

• Strong aptitude with SQL, Python and AWS data stack essential;
• Experience building efficient, scalable databases and APIs (e.g. Django, FastAPI) a huge plus;
• Experience in Kubernetes, Docker, Spark and related monitoring tools (e.g. DataDog, Grafana, Prometheus) for DataOps a huge plus;
• Experience with Airflow a huge plus;
• Experience with dbt for pipeline modelling also beneficial;
• Skilled at shaping needs into a solid set of requirements and designing scalable solutions to meet them;
• Able to quickly understand new domain areas and visualise data effectively;
• Team player excited at the idea of ownership across lots of different projects and tools;
• Drives knowledge sharing and documentation for a more effective platform;