Senior Data Engineer (Remote)

Overview

At Cybernetic Controls Limited (CCL), we are committed to being a global leader in providing innovative digital solutions that empower businesses to reach their full potential. As a remote-first company, we believe in empowering our employees to work in a way that best suits their individual needs, fostering a culture of flexibility and trust. Since our founding in 2020, we have successfully delivered high-quality resources to our clients in the FinTech sector across various business areas.

Our Client:

We are a multi-award winning RegTech company on a mission to transform the quality of regulatory reporting in the financial services industry. We’ve combined regulatory expertise with advanced technology to develop our market-leading quality assurance services. Unique in being able to fully assess data quality, our services are used by some of the world’s largest investment banks, asset managers, hedge funds and brokers, helping them to reduce costs, improve quality and increase confidence in their regulatory reporting.

Job summary

Our client is seeking a Senior Data Engineer to join our fast-growing team. The successful candidate will join the testing team to work on ETL and development tasks. This is an exciting and challenging opportunity to build out new pipelines combining and processing large amounts of structured data from a variety of sources with the power of PySpark at your fingertips.

Key Responsibilities

• Architect and build pipelines using AWS cloud computing solutions that make data available with robustness, maintainability, efficiency, scalability, availability and security.
• Develop Python and/or Spark (Preferably PySpark, but Spark-Scala is also good to have) code that implements complex data transformations.
• Design and maintain databases and APIs for storage and transmission of data between applications.
• Monitor pipelines in production (and develop tools to facilitate this).
• Work collaboratively with other team members (brainstorming, troubleshooting, and code review).
• Liaise with other development teams to ensure the integrity of data pipelines.

Skills, Knowledge & Expertise

Skills:

• Excellent Python and PySpark programming (including Pandas/PySpark dataframes, web and database connections)
• Excellent understanding of ETL processes within Amazon Web Services (AWS)
• Apache Spark, AWS Glue, Athena, S3, Step Functions, Lake Formation
• Software development lifecycle best practices
• Test-driven development
• Serverless computing (AWS Lambda, API Gateway, SQS, SNS, EventBridge, S3, etc.)
• SQL and NoSQL database design and management (DynamoDB, MySQL)
• Strong SQL coding skills (Spark-SQL, Presto SQL, MySQL, etc.)
• Infrastructure as code (CloudFormation)
• Experience in Shell Scripting (preferably Linux)
• Version control with Git/Github
• Agile principles, processes and tools
• Excellent written and verbal communication skills.

Experience:

• Designing, deploying and managing complex production data pipelines that interact with a range of data sources (file systems, web, database, users)
• Strong experience with Amazon Web Services (AWS)
• 5 years’ work in data engineering field
• At least 2 years’ experience with PySpark and AWS data tools (particularly Glue)

Knowledge:

• Data modeling, data pipeline architecture, Big Data implementation.
• Software development lifecycle best practices.
• Financial knowledge would be an asset. Qualifications/Training:
• Bachelor’s degree or equivalent in Computer Science or a related subject.

What you’ll get in return

• Competitive salary package
• Private healthcare contribution
• Annual pay review
• Regular team socials
• Working within a culture of innovation and collaboration
• Opportunity to play a key role in a pioneering growth company
• Company laptop

About Cybernetic Controls Limited

Founded in 2020 and based in the UK, Cybernetic Controls Limited was created to meet the demand of the FinTech/RegTech sector for highly-skilled technical resources.

We recruit top-end professionals to join our clients’ teams in the finance industry and other sectors. Our goal is to provide opportunities for our team members to learn and grow in a fast-paced international environment where they can enhance their skills and gain knowledge in cutting-edge technologies. Our belief is that by being part of a collaborative team where you have ownership of your work and can contribute ideas and opinions, our resources are in a position to further their careers, expand their expertise and ultimately achieve their professional goals.

Read more on our Cybernetic Controls website www.cybernetic-controls.com