Senior Software Engineer In Test Ml Ai (Remote)

Senior Software Engineer in Test 2 – ML/AI

Location – Remote

About Us

We empower the people who power modern, digital business. We enable customers to deliver reliable and secure cloud-native applications through its SaaS Analytics Log Platform, which helps practitioners and developers ensure application reliability, secure and protect against modern security threats, and gain insights into their cloud infrastructures. Customers worldwide rely on us to get powerful real-time analytics and insights across observability and security solutions for their cloud-native applications.

Responsibilities

• Design comprehensive test plans that rigorously assess the accuracy, reliability, and robustness of machine learning algorithms.
• Develop, expand, and maintain an automated testing framework tailored for evaluating the performance and accuracy of machine learning models
• Collaborate with ML Engineers and Product Managers to deeply understand the user experience and product context of ML product features under test.
• Work closely with ML Engineers to understand the structure, characteristics, and intended use of the datasets created for machine learning tasks.
• Analyze test results meticulously, identifying discrepancies, diagnosing potential causes, and recommending corrective measures.
• Establish benchmarks and metrics for model performance and ensure that machine learning outputs meet pre-defined quality standards.
• Integrate quality assurance methodologies into the continuous integration and deployment pipeline (CI/CD), ensuring consistent delivery of high-quality machine learning applications.
• Continuously monitor model performance post-deployment to detect and resolve issues through systematic re-testing and persistent optimization.
• Collaborate with ML Engineers to develop in-product (“live”) quality monitoring via mechanisms such as explicit feedback, user behavior tracking, or post-hoc annotation.

Requirements

• Having an experience of 6+ years in testing around ML models
• Bachelor’s or master’s degree in Data Science, Computer Science, Statistics, Mathematics, or a related field.
• Good understanding of machine learning concepts and model training processes.
• Experience with data visualization tools and techniques.
• Demonstrable skill in SQL and working knowledge of database technologies.
• Solid understanding of statistical methods and their application in validating and testing ML models.
• Familiarity with version control systems like Git, and CI/CD tools such as Jenkins or Travis CI.
• Strong analytical and critical thinking skills with a problem-solving mindset

Preferred Skills

• Previous experience collaborating with data science teams and leveraging their datasets in a testing environment.
• Experience with Docker, Kubernetes, or similar container and orchestration platforms is advantageous.
• Exceptional attention to detail and a passion for achieving data accuracy through rigorous testing.
• Experience with modern DevOps monitoring and deployment/rollout technologies.

If interested, share your resume at anshul@hirojet.com