Data Engineer (Remote)

This is a full-time remote role for a Data Engineer in Sri Lanka. The Data Engineer will design, build, maintain, and troubleshoot our data infrastructure. The Data Engineer will also be responsible for creating and managing data pipelines, ensuring data quality, and working with our Data Science team to provide the best possible insights to our clients.

Main Duties and Responsibilities

• Responsible for end-to-end development on the data platform, including test development, to ensure continued service delivery versus expectations.
• You will understand all engineering stages for a data platform by providing end-to-end, full-stack data expertise from design to build.
• Collaborate closely with data analysts, data scientists, and other stakeholders to understand their unique data requirements and translate them into robust technical solutions.
• Champion the implementation of best practices for data quality, security, and privacy to ensure compliance with industry regulations and standards.
• Monitor and proactively troubleshoot data pipelines, data warehouses, and data lakes, ensuring optimal performance and high availability for critical data operations.
• Provide technical guidance and mentorship to the team of data engineers, fostering their professional growth and enhancing their technical capabilities.
• Stay abreast of emerging technologies and tools in the property sector, continuously evaluating and incorporating them to enhance and improve our data infrastructure.
• Work with the Data Science team to build, implement, and maintain ML data processes (churn prediction, for example).
• End-to-end experience of the project lifecycle with experience of testing the developed solution to ensure it meets requirements and expectations.
• Skills in data cleansing and data quality management for multiple areas of the business, from the various products themselves to the linkage with the customer support software.
• Ability to interpret and analyse data from various source systems to support data integration and data reporting needs.
• Strong analytical skills with the ability to collect, organise, analyse, and disseminate significant amounts of information with attention to detail and accuracy.
• Experience and ability to produce business intelligence reports is essential.
• High understanding of data schemas is essential.
• Experience integrating third-party software such as CRM/Email Marketing etc. and working with GIS/spatial data is desirable.

Skills and Experience

• Programming experience and coding skills in Python.
• 5+ years of experience with custom, platform agnostic, ETL design, implementation and maintenance.
• A minimum of 5 years of hands-on experience in data engineering or a closely related field, showcasing your deep understanding of data engineering fundamentals and practices.
• Demonstrable understanding of MS SQL Server & Postgres modelling and indexing strategies.
• Proficiency in Azure cloud infrastructure enabling you to effectively leverage cloud-based solutions for data management and processing.
• Experience in architectural best practices for building data lakes.
• Expert knowledge in data streaming and messaging frameworks (Kafka).
• Experience with Using parallel computing to process large datasets and to optimize computationally intensive tasks (Apache Spark)
• Experience integrating third-party software such as CRM/Email Marketing etc. and working with GIS/spatial data is desirable.
• High understanding of data schemas is essential.

Role

• Full Time/Contractor
• Remote – Sri Lanka (For a UK-based company)

Salary

Competitive Salary: £12,000 – £21,600, based on experience and qualifications.