Data Engineer

Quantum Gate is a cybersecurity venture launched by Venture One, supported by the Advanced Technology Research Council (ATRC) and its research arm, the Technology Innovation Institute (TII). We specialize in developing and commercializing advanced post-quantum cryptographic solutions, that safeguard enterprise digital environments. Our innovative solutions, including password-free authentication and identity management, are shaping the future of secure corporate ecosystems and tackling the challenges of the post-quantum era.We are seeking a talented and motivated Data Engineer to join our fast-growing ML-team. Theideal candidate has strong experience with distributed data processing, No SQL databases, andmodern data lake technologies. You will play a key role in designing and maintaining scalabledata pipelines that power business intelligence, analytics, and machine learning initiatives.Key Responsibilities Design, develop, and maintain scalable ETL/ELT pipelines using Apache Spark and/or similar solutions.Manage and optimize large datasets stored in Apache Iceberg tables.Design, build, and optimize data infrastructure.Integrate data from various sources including Mongo DB, APIs, and flat files.Optimize data processing workflows for batch and real-time analytics.Implement and enforce data governance, data quality, and security best practices.Troubleshoot and resolve data-related issues in dev/production environments.Work closely with data scientists, analysts, and product teams to understand data needs and deliver efficient solutions.Monitor pipeline performance and troubleshoot data quality issues.Requirements Bachelor's degree in Computer Science, Engineering, or related field.3+ years of experience in data engineering or related roles.Strong hands-on experience with: Apache Spark (Py Spark, Scala, or Java); Mongo DB (data modeling, aggregation framework, replication); Apache Iceberg or other data lake table formats (Delta Lake, Hudi)Proficient in SQL and at least one programming language (Python, Scala, or Java).Experience with data pipeline orchestration tools (e.g., Airflow).Understanding of data warehousing concepts (star schema, partitioning).Familiarity with cloud platforms such as AWS or Azure.Nice to have Experience with stream processing (e.g., Kafka, Flink).Experience with Kubernetes for containerized data workloads.Familiarity with CI/CD practices and infrastructure as a code (e.g., Terraform).Join Quantum Gate as we pioneer innovative solutions to secure the future of digital technology and protect tomorrow’s digital society.Seniority level Seniority level Mid-Senior level Employment type Employment type Full-time Job function Industries Technology, Information and Internet Referrals increase your chances of interviewing at Quantum Gate by 2x Get notified about new Data Engineer jobs in Abu Dhabi Emirate, United Arab Emirates.Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 6 hours ago Abu Dhabi Emirate, United Arab Emirates 2 weeks ago Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 1 month ago Abu Dhabi Emirate, United Arab Emirates 1 week ago Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 9 months ago Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 1 month ago Abu Dhabi Emirate, United Arab Emirates 6 days ago Abu Dhabi Emirate, United Arab Emirates 1 week ago Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 1 month ago Abu Dhabi Emirate, United Arab Emirates 2 weeks ago Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 6 months ago Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 1 day ago Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 1 month ago Senior System Engineer- IT Infrastructure-Abu Dhabi Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 5 months ago Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates AED3,000.00-AED7,000.001 day ago Abu Dhabi Emirate, United Arab Emirates 3 weeks ago Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 7 months ago Abu Dhabi Emirate, United Arab Emirates 2 days ago Abu Dhabi Emirate, United Arab Emirates 5 days ago Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 4 months ago Data Analyst - graduated in IT (UAE National)Abu Dhabi Emirate, United Arab Emirates 2 weeks ago Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 1 month ago System Engineer - Ground Control Systems Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 2 days ago Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 6 months ago Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 10 months ago We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI. #J-18808-Ljbffr