Senior Data Engineer

We are seeking a seasoned Data Engineer with 5-10 years of overall experience and a strong background in Python for data processing and scripting. The ideal candidate will have a robust understanding of SQL, data modeling, and data warehousing practices. In addition, experience with cloud platforms such as AWS, Azure, and Google Cloud is essential. This role specifically seeks individuals with experience in the renewable energy industry to help us drive forward our data initiatives. Education Qualification

• Bachelors degree in Computer Science, Information Systems, or related field

• Design, develop, and maintain large-scale data pipelines using GCP tools such as Big Query, Data Flow, and Pub/Sub.
• Collaborate with cross-functional teams to gather requirements and design ETL processes for complex datasets.
• Develop scalable and efficient SQL queries on Azure Data Lake Storage (ADLS) to extract insights from massive datasets.
• Implement data quality checks, validation rules, and transformations to ensure high-quality data processing.
• Troubleshoot issues related to pipeline failures or errors in real-time.
• Provide business analytics support to the Management team.
• Analyze business results and manage studies to collect relevant data.
• Design, build, and maintain data pipelines and ETL processes using Python as part of larger data platform projects.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and ensure data quality. Optimize database performance, including indexing, partitioning, and query optimization
• Monitor data pipelines, troubleshoot issues, and perform data validation.
• Develop and maintain documentation for data processes and workflow

• Proficiency in Python for data processing and scripting
• Strong SQL knowledge and experience with relational databases (e.g., MySQL, PostgreSQL, SQL Server)
• Understanding of data modeling, data warehousing, and data architecture
• Experience with cloud platforms (e.g., AWS, Azure, Google Cloud) and their data services
• Proficiency in working with GCP (especially Big Query and GCS)
• Version control skills using Git
• Familiarity with GCP and Python with a minimum of 2 years of experience.