Data Engineer

Location: Dubai Who Can Apply: Candidates who are currently in Dubai Job Type: Contract Experience: Minimum 8+ years Job Summary: We are looking for an experienced Data Engineer to design, develop, and optimize data pipelines, ETL processes, and data integration solutions. The ideal candidate should have expertise in AWS cloud services, data engineering best practices, open-source tools, and data schema design. The role requires hands-on experience with large-scale data processing, real-time data streaming, and cloud-based data architectures.Key Responsibilities: Develop and Maintain Data Pipelines to process structured and unstructured data efficiently.Implement ETL/ELT Workflows for batch and real-time data processing.Optimize Data Processing Workflows using distributed computing frameworks.Ensure Data Integrity and Quality through data validation, cleaning, and transformation techniques.Work with AWS Cloud Services, including S3, Redshift, Glue, Lambda, Dynamo DB, and Kinesis.Leverage Open-Source Tools like Apache Spark, Airflow, Kafka, and Flink for data processing.Manage and Optimize Database Performance for both SQL and No SQL environments.Collaborate with Data Scientists and Analysts to enable AI/ML model deployment and data accessibility.Support Data Migration Initiatives from on-premise to cloud-based data platforms.Ensure Compliance and Security Standards in handling sensitive and regulated data.Develop Data Models and Schemas for efficient storage and retrieval.Required Skills & Qualifications: 8+ years of experience in data engineering, data architecture, and cloud computing.Strong knowledge of AWS Services such as Glue, Redshift, Athena, Lambda, and S3.Expertise in ETL Tools, including Talend, Apache Ni Fi, Informatica, dbt, and AWS Glue.Proficiency in Open-Source Tools such as Apache Spark, Hadoop, Airflow, Kafka, and Flink.Strong Programming Skills in Python, SQL, and Scala.Experience in Data Schema Design, normalization, and performance optimization.Knowledge of Real-time Data Streaming using Kafka, Kinesis, or Apache Flink.Experience in Data Warehouse and Data Lake Solutions.Hands-on experience with Dev Ops and CI/CD Pipelines for data engineering workflows.Understanding of AI and Machine Learning Data Pipelines.Strong analytical and problem-solving skills.Preferred Qualifications: AWS Certified Data Analytics – Specialty or AWS Solutions Architect certification.Experience with Kubernetes, Docker, and serverless data processing.Exposure to MLOps and data engineering practices for AI/ML solutions.Experience with distributed computing and big data frameworks. #J-18808-Ljbffr