Orion Innovation is a premier, award-winning, global business and technology services firm. Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and engineering, with a unique combination of agility, scale, and maturity. We work with a wide range of clients across many industries including financial services, professional services, telecommunications and media, consumer products, automotive, industrial automation, professional sports and entertainment, life sciences, ecommerce, and education.

Required Knowledge and Skills

• 5+ years of experience designing, building, deploying, testing, maintaining, monitoring, and owning scalable, resilient, and distributed data pipelines.
• High Proficiency in at least two of Scala, Python, Spark applied to large scale data sets
• Expertise with big data technologies (Spark, Data Lake, Delta Lake, Hive)
• Knowledge of batch and streaming data processing techniques
• Understanding of the Data Lifecycle Management process to collect, access, use, store, transfer, delete data.
• Expert level knowledge of using SQL to write complex, highly optimized queries across large volumes of data.
• Hands-on object-oriented programming experience using Scala, Python, R, or Java
• Skilled with Azure DevOps, creating PBIs and work items
• Strong knowledge of Agile
• 1+ years managing a team of developers
• At Effectual, Data engineers are responsible for designing, building, and maintaining datasets that can be leveraged in data projects.
• Obsession for service observability, instrumentation, monitoring, and alerting
• Knowledge or experience in architectural best practices in building data lakes

Responsibilities

• You will be responsible for managing a team of developers who will be responsible for building optimized data pipelines using emerging technologies, in a cloud environment, for the purpose of driving analytic insights.
• Create Product Backlog Items (PBIs) and work items in Azure DevOps
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of sources like API, Blob Storage, and no SQL Database
• Assist the Product Manager and Business Manager in designing the technical solution

Good to have

• Passionate about testing strategy, problem solving, learning new skills, sharing expertise and knowledge.
• Always Be Learning
• Product / Engineering Mindset