Data Engineer

Primary Objective:

1. Integrating data from diverse sources into the enterprise data lake.
2. Implementing solutions for handling and analyzing unstructured data.
3. Enforcing data governance policies, ensuring compliance with data privacy regulations, and implementing security measures to protect sensitive data.
4. Collaborating with cross-functional teams to understand business requirements and align data solutions with organizational goals.
5. Providing technical leadership and mentorship to junior data engineers.
6. Planning and implementing scalable data infrastructure to accommodate growing data volumes.
7. Evaluating and adopting new technologies to enhance data engineering capabilities.
8. Implementing monitoring solutions to track the performance of data systems.
9. Proactively identifying areas for optimization and implementing improvements.
10. Creating and maintaining comprehensive documentation for data architectures and processes.
11. Identifying opportunities for process improvements and implementing best practices.
12. Staying informed about emerging trends and technologies in the data engineering field.

1. Designing and implementing complex ELT/ETL processes for large-scale data integration, optimizing ELT/ETL workflows for performance, scalability, and resource efficiency.
2. Working with distributed computing frameworks and big data technologies (e.g., Hadoop, Spark) for processing large datasets.
3. Designing, implementing, and managing data warehouses for efficient data storage and retrieval.
4. Creating and maintaining data models to support business requirements.
5. Implementing and optimizing database schemas for performance and scalability.
6. Conducting performance analysis and implementing optimizations for database queries and data processing.
7. Identifying and resolving performance bottlenecks in data pipelines.
8. Facilitating knowledge sharing within the team and across departments.

1. Degree in Computer Science, Data Engineering, or equivalent.
2. A minimum of 3-5 years of experience (banking & insurance experience is a plus).
3. Proficiency in SAS, Python, SQL.
4. Hands-on experience with Hadoop, Spark, or similar technologies for large-scale data processing.
5. Understanding of data quality checks, monitoring, and testing procedures.
6. Minimum 3 years of experience in analytics-related roles (banking experience is a plus).

1. Agile development and DevOps operating practices.
2. Listening and communication/storytelling.
3. Critical and evaluative thinking.
4. Proactive approach with the ability to work independently.

1. Understanding of what data is needed to be collected and analyzed across business solutions.

1. Technical leadership in data and platform design.