Data Engineering Lead

Ref: ZR_6518_JOB  ·  Posted on 12 October 2021

Introduction

As the Data Engineering Lead, you will lead a team of Data Engineers within the client’s development team of Data Scientists, Data Architects, Machine Learning Engineers and Developers. You will be responsible for leading and growing the team, nurturing an industry leading capability for the business.

The Data Engineering Lead will be responsible for making sense of large volumes of historical and real-time data, leading a team to apply cutting edge tools and techniques to develop and deploy robust data pipelines to enable advanced analytics.  As part of the technical leadership team, you will help define analytics use cases to support a wide variety of exciting business challenges.

Key responsibilities
  • Design and build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of large and complex data sources, that meet functional and non-functional business requirements
  • Set and maintain software engineering and DevOps best practices across the team including automated testing, version control, and CI/CD in a cloud-based environment
  • Create and maintain optimal data architectures and pipelines. Work with Data Scientists to build analytics tools that utilize the data pipeline to provide actionable insights into asset management, operational efficiency, and other key business performance metrics
  • Work with Cloud Architects to ensure that data is stored in secure locations and that access is granted in accordance with data governance principles. Partner with the Security and DevOps teams in defining security and operations practices for data applications
  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
  • Work with business stakeholders to understand data sources and to develop semantics and ontologies so that data can be analysed within the right business context
  • Work with stakeholders including the Product, Data and Application teams to assist with data-related technical issues and support their data infrastructure needs
  • Coach, mentor and support a team of Data Engineers and create a collaborative and open environment.
Skills & experience
  • Extensive experience working with Big Data tools and building data solutions for advanced analytics. Spark experience strongly preferred
  • Practical knowledge across data extraction and transformation tools with significant hands-on experience with cloud platform solutions (e.g. Azure Data Factory, Azure Synapse and Databricks)
  • Experience designing data models and architectures for data warehouses, APIs, analytics/ML applications (Python, R, Spark), and BI/dashboard tools
  • Knowledge in data architecture, cloud infrastructure, security best practices and data operations (monitoring, prod ops, and the overall data lifecycle)
  • Experience operating under an Agile delivery model, and/or a software development environment would be a benefit but not essential
  • Individual Skills
  • A professional attitude and an ability to think outside of the box to solve complex challenges
  • Excellent communicator, with strong written and verbal communication skills along with strong desire to work in cross-functional team (technical and non-technical).
  • Able to clearly articulate technical concepts quickly to those without a technical background and resolve conflicts.

Our client is committed to providing a diverse and inclusive workplace and welcomes applications from all backgrounds.

Part-time opportunities/flexible working is available to suit individual needs.

Apply for role
Data Engineering Contract Python Analytics Big Data DEVOPS

Refer your friend: We want to give you a Netflix or Amazon Prime subscription and £250 Love2Shop vouchers!

Refer your friend
Refer your friend:
Contact us

Let's connect and work together

Start the conversation

Submit CV  ·  Submit Vacancy