Data Engineer

Join to apply for the Data Engineer role at Dicetek LLC Join to apply for the Data Engineer role at Dicetek LLC Get AI-powered advice on this job and more exclusive features. Overview: A highly skilled Data Engineer with 4 years of experience specializing in cloud data architecture and integration. Expertise in leveraging AWS cloud services to build scalable data pipelines, ensuring seamless integration with cloud-based ERP systems like Microsoft Dynamics, Salesforce, and Oracle Fusion. Adept at handling large-scale data processing, transformation, and storage solutions. Strong communicator and problem solver with a focus on delivering efficient, reliable, and high-performance data workflows that support business decision-making processes across departments.Key Responsibilities Data Engineering & Cloud Architecture: Designed and optimized scalable data pipelines in AWS, integrating ERP systems like Dynamics 365, Salesforce, and Oracle Fusion. Python & Py Spark Development: Developed custom Python and Py Spark scripts for automating data transformations and processing large datasets efficiently. ETL Process Development: Managed ETL processes to integrate data from ERP systems into AWS data lakes/warehouses, ensuring timely and accurate data flow. AWS Cloud Services: Utilized AWS services (S3, Redshift, Lambda, Glue) to store, process, and analyze data, optimizing for scalability and cost-efficiency. Data Validation & Cleansing: Built validation routines using Python/Py Spark to ensure data accuracy, consistency, and integrity, addressing discrepancies proactively. Data Modelling & Transformation: Transformed raw data into structured formats using Python, Py Spark, and AWS, optimizing models for fast querying and insights. SQL & No SQL Optimization: Wrote and optimized complex SQL and No SQL queries for efficient data extraction and transformation at scale. Automation & Workflow Management: Automated data workflows using Python and AWS Lambda, reducing manual intervention and minimizing errors. Performance Monitoring & Troubleshooting: Monitored and optimized data pipelines and cloud infrastructure to identify and resolve bottlenecks and improve performance. Documentation & Reporting: Maintained detailed documentation for data systems, ETL processes, and pipeline performance, providing updates to stakeholders. Qualifications Bachelor’s degree in computer science, Information Technology, or a related field. 4+ years of experience in data engineering, focusing on cloud-based data architectures, programming in Python and Py Spark, and ERP system integrations. AWS Cloud Expertise: Advanced knowledge of AWS services, including S3, Redshift, Lambda, Glue, RDS, and Athena for building and optimizing data pipelines. Python & Py Spark: Extensive experience with Python and Py Spark for building efficient data processing solutions, automating workflows, and processing large datasets. ERP System Integration: Demonstrated expertise in integrating cloud-based ERP systems such as Microsoft Dynamics 365, Salesforce, and Oracle Fusion with AWS data pipelines. ETL Process Development & Management: Proven ability to develop, maintain, and optimize robust ETL processes that handle large volumes of data efficiently. SQL & No SQL Optimization: Proficient in writing optimized SQL queries for relational databases and working with No SQL systems (such as Mongo DB or Cassandra) for data processing and transformation. Data Quality Assurance & Validation: Strong experience in implementing data quality checks, validation, and cleansing routines to ensure high integrity in the data pipeline. Problem Solving & Optimization: Strong troubleshooting skills with the ability to identify and resolve data-related performance issues and optimize workflows for better scalability and speed. Project Management & Leadership: Ability to manage multiple data engineering projects, ensuring that all tasks are completed on time, within scope, and according to requirements. Skills AWS Cloud Services (S3, Redshift, Lambda, Glue, RDS, Athena) Python & Py Spark (Data Transformation, Big Data Processing, Automation) ETL Process Development & Optimization ERP System Integration (Microsoft Dynamics 365, Salesforce, Oracle Fusion) Data Pipeline Development & Workflow Automation SQL & No SQL Query Optimization Data Modelling & Data Transformation Data Validation, Cleansing & Quality Assurance Performance Monitoring & Troubleshooting Project Management & Documentation Seniority level Seniority level Not Applicable Employment type Employment type Contract Job function Job function Information Technology Industries IT Services and IT Consulting Referrals increase your chances of interviewing at Dicetek LLC by 2x Get notified about new Data Engineer jobs in Dubai, Dubai, United Arab Emirates . Full Stack Developer Trainees(Internship) We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI. #J-18808-Ljbffr