Data Engineer (Chennai/Pune)

Company Description We suggest you enter details here. Role Description We are looking for highly skilled Data Engineers to join our team for a long-term offshore position. The ideal candidates will have 5+ years of experience in Data Engineering, with a strong focus on Python and SQL programming. The role requires proficiency in leveraging AWS services to build efficient, cost-effective datasets that support Business Reporting and Al/ML Exploration. Candidates must demonstrate the ability to functionally understand the Client Requirements and deliver Optimized Datasets for multiple Downstream Applications. The selected individuals will work under the guidance of a Lead from Onsite and closely with Client Stakeholders to meet business objectives. Key Responsibilities Cloud Infrastructure

• Design and implement scalable, cost-effective data pipelines on the AWS platform using services like S3, Athena, Glue, RDS, etc.
• Manage and optimize data storage strategies for efficient retrieval and integration with other applications.
• Support the ingestion and transformation of large datasets for reporting and analytics.

• Develop and maintain automation scripts using Python to streamline data processing workflows.
• Integrate tools and frameworks like PySpark to optimize performance and resource utilization.
• Implement monitoring and error-handling mechanisms to ensure reliability and scalability.

• Work closely with the onsite lead and client teams to gather and understand functional requirements.
• Collaborate with business stakeholders and the Data Science team to provide datasets suitable for reporting and A/ML exploration.
• Document processes, provide regular updates, and ensure transparency in deliverables.

• Optimize AWS service utilization to maintain cost efficiency while meeting performance requirements.
• Provide insights on data usage trends and support developing reporting dashboards for cloud costs.

• Ensure secure handling of sensitive data with encryption (e.g. AES-256,TLS) and role-based access control using AWS IAM.
• Maintain compliance with organizational and industry regulations.

• 5+ years of experience in Data Engineering
• 4+ years experience in AWS platforms
• Experience with programming languages like Python, SQL, or Scala
• Knowledge of big data technologies like Hadoop, Spark, or Kafka
• Experience working with cloud platforms like AWS or Azure
• Bachelor's degree in Computer Science, Information Technology, or related field