Senior Data Engineer K (Remote)

Exciting opportunity for an experienced cloud data engineer to join an expanding data team who are using data in an exciting and advanced way. They will support your learning and development from a technical and leadership perspective as you help them design, build and optimise their data platform.

Salary & Benefits

• 22 days holiday, plus bank holidays
• Staff discounts & Friends and Family discounts
• Cycle to work scheme and Tech Scheme
• Breakfast and drinks provided (if in office)
• Yearly charity day supported
• Summer and Christmas Parties
• Perkbox membership
• Maternity & Paternity leave
• Employee referral scheme

Role & Responsibilities

• Design, develop, and maintain scalable and efficient data pipelines
• Optimise ETL processes to ensure efficient data ingestion, processing, and integration across various systems
• Lead development and maintenance of real-time data streaming platform using Apache Kafka, Databricks, etc.
• Ensure the integration of streaming data with batch processing systems for comprehensive data management
• Utilize AWS data engineering services to build and manage data infrastructure
• Continuously optimize the platform for performance, scalability, and cost-effectiveness
• Collaborate with cross-functional teams, including data scientists and BI developers, to understand data needs and deliver quality, deliverable solutions
• Leverage the project management team to coordinate project, requirements, timelines and deliverables, allowing you to concentrate on technical excellence
• Establish ML Ops practices within the data engineering framework, focusing on automation, monitoring, and optimization of machine learning pipelines
• Implement and maintain data quality frameworks, ensuring accuracy, consistency, and reliability of data across the platform
• Drive data governance initiatives, including data cataloguing, lineage tracking, and adherence to security and compliance standards

What do I need to apply for the role

• 3+ years of experience in data engineering, with a proven track record in building and maintaining data platforms, preferably on AWS (cloud experience essential)
• Proficiency in Python, experience in SQL and PostgreSQL. PySpark, Scala or Java
• Familiarity with Databricks and Delta Lakehouse
• Experience mentoring or leading junior engineers
• Deep understanding of cloud-based data architectures and best practices
• Proficiency in designing, implementing, and optimizing ETL/ELT workflows
• Strong database and data lake management skills
• Familiarity with ML Ops practices and tools, with a desire to expand skills in this area
• Excellent problem-solving abilities and a collaborative mindset

My client have very limited interview slots and they are looking to fill this vacancy by Christmas, so this process will have a rapid turnaround. I have limited slots for 1st stage interviews this week so if you’re interest, get in touch ASAP with a copy of your most recent and up to date CV and email me at or you can call me on .

Please Note: This is a permanent role for UK residents only. This role does not offer Sponsorship. You must have the right to work in the UK with no restrictions. Some of our roles may be subject to successful background checks including a DBS and Credit Check.

Nigel Frank are the go-to recruiter for Power BI and Azure Data Platform roles in the UK, offering more opportunities across the country than any other. We’re the proud sponsor and supporter of SQLBits, Power Platform World Tour, the London Power BI User Group, Newcastle Power BI User Group and Newcastle Data Platform and Cloud User Group. To find out more and speak confidentially about your job search or hiring needs, please contact me directly at