Senior Data Engineer (Remote)

Luupli is a social media app that has equity, diversity, and equality at its heart. We believe that social media can be a force for good, and we are committed to creating a platform that maximizes the value that creators and businesses can gain from it, while making a positive impact on society and the planet. Luupli started internal testing since June 2024 and getting ready for a commercial BETA testing from December 2024, with the hope of launching fully summer of 2025. Our team is made up of passionate and dedicated individuals who are committed to making Luupli a success.

Job Title: Senior Data Engineer

About Luupli:

Luupli is a social media app that has equity, diversity, and equality at its heart. We believe that social media can be a force for good, and we are committed to creating a platform that maximise the value that creators and businesses can gain from it, while making a positive impact on society and the planet. Luupli started internal testing since June 2024 and getting ready for a commercial BETA testing from December 2024, with the hope of launching fully summer of 2025.Our team is made up of passionate and dedicated individuals who are committed to making Luupli a success.

About The Role:

We’re seeking a skilled Senior Data Engineer with experience in recommendation systems to join our team. This role is pivotal in enhancing our backend systems, managing databases, and optimizing recommendation algorithms. You’ll play a critical role in the architecture and performance of our recommendation infrastructure, ensuring accurate, relevant, and efficient recommendations for our users.

Responsibilities:

Data and Database Management:

• Design, optimize, and maintain tables and data structures to support recommendation and trending content data.
• Work with structured data storage solutions, including PostgreSQL and JSONB, to manage recommendation and interaction data.

Recommendation Systems:

• Implement and refine recommendation algorithms (e.g., collaborative filtering, content-based, and hybrid approaches) to enhance relevancy.
• Use similarity search libraries like Annoy or Faiss to optimize recommendation speed and accuracy.
• Continuously evaluate recommendation logic to better serve user preferences, ensuring real-time delivery.

Data Aggregation And Analysis:

• Aggregate, analyze, and process user interaction data to support recommendations and trending content.
• Design efficient queries and implement aggregation methods to capture relevant data and insights for recommendations.

Debugging And Optimisation:

• Identify, troubleshoot, and resolve data handling issues to ensure accurate recommendation delivery.
• Optimise queries, processing workflows, and containerised services for high performance and scalability within AWS ECS.

Requirements:

• Proven experience as a Data Engineer or Backend Engineer, with a focus on recommendation systems.
• Proficient in SQL and database management, especially with PostgreSQL and JSONB for structured data handling.
• Solid understanding of recommendation algorithms (collaborative filtering, content-based, hybrid approaches).
• Experience with similarity search libraries such as Annoy or Faiss.
• Strong programming skills in Python, with experience in building backend logic for data-intensive applications in a containerised environment.
• Familiarity with AWS ECS for container management, including task scheduling and scaling.
• Experience using AWS EventBridge to trigger workflows or automate tasks in response to application events.
• Analytical skills for data aggregation, querying, and insights generation.
• Strong debugging and optimisation skills for handling large-scale data processing in cloud-based environments.

Preferred Qualifications:

• Knowledge of data aggregation pipelines, ETL processes, and data handling at scale.
• Familiarity with additional AWS services (e.g., S3, Lambda) for data storage and event-driven architectures.
• Experience with machine learning libraries or tools used in recommendation systems.

Compensation:

This is an equity-only position, offering a unique opportunity to gain a stake in a rapidly growing company and contribute directly to its success.