Data Engineer, Sensor & Machine Learning Lab

As a Machine Learning Data Engineer at CAG, you will be responsible for designing, implementing, and maintaining the data pipelines and infrastructure that support our machine learning projects. You will work closely with data scientists, machine learning engineers, cloud engineer and other cross-functional teams to ensure the availability, reliability, and performance of our data systems. Your role will be critical in enabling the development and deployment of advanced machine learning models that drive key business insights and innovations. Key Responsibilities:

• Work on large-scale structured and unstructured data sets to solve a wide array of challenging problems, using analytical, statistical, machine learning or deep learning approaches.
• Design and develop robust, scalable and efficient data pipelines for collecting, processing and storing large volumes of structured and unstructured data.
• Collaborate with data scientists and machine learning engineers to understand data requirements, feature engineering needs and model deployment strategies.
• Implement data pre-processing, transformation and enrichment techniques to ensure high-quality and reliable input data for machine learning models.
• Build and maintain data infrastructure, including data warehouses, data lakes and other relevant storage systems, to facilitate easy access to data for analysis and model training.
• Optimise data pipelines for performance, scalability, and cost-effectiveness, considering factors such as data volume, processing speed, and resource utilisation.
• Work closely with DevOps and IT teams to ensure proper deployment, monitoring and maintenance of data pipelines in production environments.
• Collaborate with cross-functional teams to ensure compliance with data governance, security and privacy regulations.
• Stay up-to-date with the latest developments in data engineering and machine learning technologies and contribute to the adoption of best practices within the team.

• Bachelor's or higher degree in Computer Science, Engineering or a related field.
• Proven experience as a Data Engineer, Software Engineer or related role, with a focus on building data pipelines and infrastructure for machine learning projects.
• Strong proficiency in programming languages, such as Python, Java or Scala; as well as SQL for data manipulation and analysis.
• Hands-on experience with data processing tools or frameworks, such as Pandas, SKlearn, Apache Spark, Hadoop or similar technologies.
• Familiarity with cloud platforms, such as AWS, GCP or Azure; and experience with cloud-based data services (e.g., S3, BigQuery, Data Lakes).
• Solid understanding of database systems, data modeling and ETL principles.
• Experience with containerisation and orchestration tools (e.g., Docker, Kubernetes) is a plus.
• Strong problem-solving skills and the ability to work in a fast-paced, collaborative environment.
• Excellent communication skills and the ability to effectively communicate complex technical concepts to non-technical stakeholders.