Start Your Search Here

push notification bell

Would you like to receive notifications about IT & Technology jobs in High Street?

push notification bell

You have blocked notifications

Oops! You have blocked notifications. Click here for more info

You have blocked notifications, please check your browser settings.

push notification bell

You're currently subscribed to job notifications

Want to change your notifications for job alerts?

push notification bell

Subscribe to notifications

You will no longer receive notifications

Job Search

ELLIOTT MOSS CONSULTING PTE. LTD.

High Street / Global

Lead Data Engineer

  • £90000

Job Description

Job Description

We are seeking an experienced Databricks Architect / Lead Data Engineer to lead the design, development, and delivery of enterprise-scale data platforms on Databricks and AWS.

The role combines architecture leadership with hands-on engineering responsibilities, including building scalable Lakehouse solutions, modern data pipelines, and AI/ML workloads.

Key Responsibilities

  • Design and own Databricks Lakehouse architecture on AWS. Lead multiple Databricks initiatives and mentor data engineers.
  • Build and optimize data pipelines using Python, PySpark, SQL, and Delta Lake. Implement Medallion Architecture (Bronze, Silver, Gold layers).
  • Develop batch, streaming, and near real-time data solutions. Enable AI/ML and Generative AI use cases on Databricks.
  • Establish governance, security, Unity Catalog, RBAC, and operational controls. Integrate Databricks with AWS services including S3, IAM, networking, and data platforms.
  • Drive CI/CD, automated testing, deployment automation, and observability.
  • Troubleshoot and optimize Spark workloads for performance, reliability, and cost. Required Experience In Data Engineering experience.
  • And hands-on Databricks experience. as a Databricks Architect or equivalent architecture role.
  • delivering AI/ML or Generative AI solutions on Databricks. and AWS experience.
#J-18808-Ljbffr
Apply Now

Similar Opportunities

View all jobs