Tech Lead (Azure Databricks Data Platform)

Aubay UK is seeking an experienced Tech Lead (Azure Databricks Data Platform) to lead the design, development, and delivery of enterprise-scale data platform solutions leveraging the Azure Data Platform and Databricks ecosystem.The successful candidate will be responsible for providing end-to-end technical leadership across data engineering initiatives, driving solution architecture, platform strategy, engineering standards, and delivery excellence. Working closely with business stakeholders, architects, product teams, and engineers, the Tech Lead will translate complex business requirements into scalable, high-performance data solutions that support critical business operations and analytics capabilities.This role requires deep hands-on expertise in Azure Databricks, Structured Streaming, Delta Live Tables (DLT), PySpark, Python, and SQL, alongside strong stakeholder management and leadership skills. The ideal candidate will have a proven track record of delivering large-scale cloud data platforms, optimising platform performance and costs, and mentoring engineering teams to achieve successful business outcomes.Required Skills and Experience:Strong hands-on experience with Azure Databricks, PySpark, Python, and SQLExperience within financial trading, commodities trading, or energy trading environmentsUnderstanding of trading data models and trading business processesExperience building and supporting enterprise-scale data platforms on Microsoft AzureExpertise in Structured Streaming and Delta Live Tables (DLT)Experience designing scalable, high-performance data solutions and architecturesStrong understanding of data governance, data quality, and engineering best practicesExperience leading technical delivery from design through to production supportStrong stakeholder management and communication skillsExperience leading and mentoring engineering teamsDesired Skills and Experience:Experience optimising Databricks performance and managing cloud platform costsKnowledge of Lakehouse architecture and modern data platform design patternsExperience working within Agile delivery environmentsRelevant Azure and/or Databricks certificationsRoles and Responsibilities:Lead the design, development, and implementation of scalable data platform solutions using Azure and DatabricksProvide technical leadership across the full solution lifecycle, from architecture through delivery and production supportDefine and promote best practices for data engineering, governance, security, quality, and platform standardsDesign and optimise real-time data processing solutions using Structured Streaming and Delta Live Tables (DLT)Collaborate with business stakeholders to translate business requirements into technical solutionsDrive technical decision-making and ensure alignment with platform strategy and business objectivesMonitor and optimise platform performance, reliability, scalability, and cloud costsSupport the development of data architecture, integration patterns, and engineering roadmapsMentor Data Engineers and provide technical guidance across delivery teamsEnsure successful delivery of business outcomes through strong ownership, accountability, and leadershipWork closely with architects, product teams, and stakeholders to manage technical risks, dependencies, and prioritiesPromote continuous improvement and adoption of modern data engineering practices across the platform teamAbout Aubay UK – Ahead of Innovation!Aubay UK is a recognised InSourcing Partner for client-side deployment delivered across London. Our team, based in Canary Wharf, specialises in hiring IT professionals within London’s Energy and FinTech sectors, helping our clients to expand their operations with top-tier talent who are experts in their fields. We work exclusively with clients who are globally recognized as Energy Super Majors/Financial Services and innovative FinTech players.Aubay UK is the most recently started branch of Aubay Group www.aubay.com.Aubay Group is an international Digital Services Company, listed on a Euronext Stock Exchange, who have been operating for 25 years in the European market and working alongside some of the biggest names in the Banking, Finance, Insurance, Energy, IT/Digital, Manufacturing, Transport and Telecoms sectors. With over 7,800 employees across 7 countries and 16 offices in England, France, Belgium, Luxembourg, Italy, Spain, and Portugal, Aubay Group generated revenues of €534 million in 2023.Our ClientOur client is one of the Super Major global energy companies with around 84,000 employees across 70+ countries who are working to power progress through cleaner energy solutions.Specialities: Upstream/Downstream, Biofuels, Integrated Gas, New Energies, Chemicals, Energy and Trading.You will have the opportunity to work in a challenging but rewarding environment that is fast-paced and changing fundamentally and work towards transforming the business of a Super Major energy company to meet the ambition to be a net-zero emissions energy business by 2050 whilst delivering a world-class business case that has a robust societal license to operate. In your role, you will be expected to enact change and deliver value globally across business lines and geographies.

#J-18808-Ljbffr