Principal Data Engineer

We are undertaking a Data Transformation and as data becomes fundamental to what weare doing, the ability to have the right data in the right place at the right time, that is accurate andconsistent is essential. The purpose of this job is to have a Principal Data Engineer resource that can participate in day-to-day QA activities like, testing sprint tickets, providing the updates on the assigned tickets, to improve yourknowledge on the relevant domain. You need to be passionate enough to think & perform out of thebox, to add innovation to improve your daily task and work FAST (Frequently discussed, Ambitious,Specific & Transparent).

Responsibilities:

Designing and implementing data pipelines to collect, clean, and transform data from varioussources

Building and maintaining data storage and processing systems, such as databases, datawarehouses, and data lakes

Ensuring data is properly secured and protected

Developing and implementing data governance policies and procedures

Collaborating with business analysts, data analysts, data scientists, and other stakeholdersto understand their data needs and ensure they have access to the data they need.

Sharing knowledge with the wider business, working with other BA's and technology teamsto make sure processes and ways of working is documented.

Collaborate with Big Data Solution Architects to design, prototype, implement, and optimizedata ingestion pipelines so that data is shared effectively across various business systems.

Ensure the design, code and procedural aspects of solution are production ready, in termsof operational, security and compliance standards.

Participate in day-to-day project and agile meetings and provide technical support for fasterresolution of issues.

Clearly and concisely communicating to the business, on status of items and blockers.

Have an end-to-end knowledge of the data landscape

Requirements:

7-9 years of design & development experience with big data technologies like Azure, AWSor GCP

Preferred is Azure & Databricks, with experience in Azure DevOps

Experience in data visualising technology in DL, like PowerBI

Proficient in Python, PySpark and SQL

Proficient in querying and manipulating data from various DB (relational and big data).

Experience of writing effective and maintainable unit and integration tests for ingestionpipelines.

Experience of using static analysis and code quality tools and building CI/CD pipelines.

Excellent communication, problem-solving, and leadership skills, and be able to work well in a fast-paced, dynamic environment.

Experience of working on high-traffic and large-scale Software Products

We have an amazing team of 700+ individuals working on highly innovative enterpriseprojects & products. Our customer base includes Fortune 5retail and CPGcompanies, leadingstore chains, fast-growth fintech, and multipleSilicon Valley startups.

What makes Confiz stand out is our focus on processes and culture. Confiz is ISO 9001:2015 certified. We have a vibrant culture of learning via collaboration and making the workplace fun.

People who work with us work with cutting-edge technologies while contributing success to the company as well as to themselves.

To know more about Confiz Limited, visit:https://web.facebook.com/lifeatconfiz

#J-18808-Ljbffr IT & Technology