Data Engineer (Remote)

Skills:
SQL, Python, ETL, Data Modeling, Big Data Technologies, Data Warehousing, Cloud Platforms, Data Pipeline Development,

The Position

We are seeking an experienced Data Engineer to join our team of passionate professionals working on cutting-edge technology. In this role, you will be responsible for the ELT of Canibuild data using Python, Airflow, and SQL within an AWS environment. Additionally, you will create and maintain data visualizations and dashboards using PowerBI, connecting to our SQL Server and PostgreSQL Aurora database through a gateway. This position offers a unique opportunity to influence architectural decisions and contribute to the development of a data warehouse in a cloud-native environment.

Responsibilities

Design, develop, and maintain ELT pipelines using Python, Airflow, and SQL in an AWS environment.

Create and manage a data lake and data warehouse solutions on AWS.

Develop and maintain data-driven dashboards and reporting solutions in PowerBI.

Connect PowerBI to SQL Server and PostgreSQL Aurora databases using a gateway.

Perform data profiling and source system analysis to ensure data quality and integrity.

Collaborate with business stakeholders to capture and understand data requirements.

Implement industry best practices for data engineering and visualization.

Participate in architectural decisions and contribute to the continuous improvement of data solutions.

Follow agile practices and a Lean approach in project development.

Optimize SQL queries for performance and ensure efficient database operations.

Perform database tuning and optimization as needed.

Required Skills And Experience

Expert-level proficiency in SQL (TSQL, MS SQL) with a strong focus on optimizing SQL queries for performance.

At Least 7 Years Of Experience Will Be Preferred.

Extensive experience with Python and Airflow for ELT processes.

Proven experience in designing and developing data warehousing solutions on the AWS cloud platform.

Strong expertise in PowerBI for data visualization and dashboard creation.

Familiarity with connecting PowerBI to SQL Server and PostgreSQL Aurora databases.

Experience with REST APIs and JSON.

Agile development experience with a focus on continuous delivery and improvement.

Excellent problem-solving skills and a proactive can-do attitude.

Strong communication skills and the ability to work collaboratively in a team environment.

Curiosity and eagerness to learn and adapt to new technologies and methodologies.

Ability to perform database tuning and optimization to ensure efficient data operations.Desired Skills:

Exposure to AWS Cloud Data Services such as RedShift, Athena, Lambda, Glue, etc.

Experience With Other Reporting Tools Like Tableau.

Knowledge of data lake architectures and best practices.Benefits:

Flexible work arrangements.

Opportunity to grow with a fast-scaling company.

Recognition and reward for your contributions.Join Canibuild and be a part of a transformative journey in the construction industry. Apply today to become a key player in our innovative and dynamic team!

Must Have Skills:Python, Airflow, SQL and powerBI in a commercial way

Budget: 22-25LPA

Job Type

Payroll

Categories

Data Engineer (Software and Web Development)

Data Scientist (Software and Web Development)

Must have Skills

• Python – 5 Years
• Intermediate
• SQL – 5 Years
• Intermediate
• Power BI – 3 Years
• Intermediate
• Apache Airflow – 3 Years
• Intermediate
• Data Engineer – 6 Years
• Intermediate