Cloud Data Engineer

Job Title: Cloud Data Engineer Skills: AWS, Snowflake, Python / Pyspark, SQL, ETL, Data Warehousing, Data Modelling & Dimensional Modelling Experience: 4 - 14 years Location: Greater Noida & Bengaluru We at Coforge are hiring Cloud Data Engineers with the following skillset: Qualifications:

• Proficiency in Data Warehouse Design/ETL Architecture : Extensive experience in designing and architecting Data Warehouse ETL processes, including dimensional and relational data modeling.
• ETL Development Experience : Experience with at least one ETL development project, including writing and analyzing complex stored procedures.
• Python/PySpark : Entry-level to intermediate experience with Python/PySpark, including working knowledge of Spark/Pandas DataFrames, Spark multi-threading, exception handling, and familiarity with various boto3 libraries. Experience with data transformation and ingestion methods and the ability to write User Defined Functions (UDFs) is essential.
• Snowflake : Familiarity with Snowflake features such as stages, external tables, commands (e.g., COPY, UNLOAD), and working knowledge of variant data types. Ability to flatten nested structures using SQL, familiarity with marketplace integrations, role-based masking, pipes, data cloning, logs, and user/role management is a plus.
• Coalesce/DBT : Familiarity with Coalesce or DBT (Data Build Tool) is advantageous.
• Collibra Integration : Experience with Collibra for Data Quality and Governance in ETL pipeline development is a plus.
• AWS : Hands-on experience with AWS services such as S3, Glue (jobs, triggers, workflows, catalog, connectors, crawlers), CloudWatch, RDS, and Secrets Manager. Experience with AWS VPC, IAM, Lambda, SNS, SQS, and MWAA is a bonus.
• Version Control & CI/CD : Hands-on experience with version control tools like GitHub and knowledge of setting up CI/CD pipelines using YAML and pip files.
• Streaming Services : Familiarity with streaming services such as Confluent Kafka, Spark Streaming, or Kinesis (or equivalent) is a plus.
• Data Vault 2.0 : Experience with Data Vault 2.0 (hubs, satellites, links) is a plus.

• Proficient in tools such as Publisher, PowerPoint, SharePoint, Visio, Confluence, and Azure DevOps.
• Strong understanding of best practices in value-driven development, including requirements management, prototyping, hypothesis-driven development, and usability testing.
• Excellent communicator with a problem-solving mindset and focus on continuous process improvement.
• Clear and concise written and verbal communication skills.
• Strong interpersonal skills and the ability to interact with senior management.
• Highly self-motivated with a strong sense of initiative.
• Excellent multitasking, task management, and time management skills.
• Ability to work well in a team environment, meet deadlines, and manage multiple tasks in a fast-paced project setting.

• Partner with business stakeholders to gather requirements and translate them into technical specifications and process documentation for IT teams (both onshore and offshore).
• Expertise in the architecture and development of an event-driven data warehouse, including streaming and batch processing, data modeling, and storage.
• Advanced knowledge of databases, including creating and optimizing SQL queries, stored procedures, functions, partitioning data, indexing, and analyzing execution plans.
• Skilled in writing and troubleshooting Python/PySpark scripts to generate data extracts, cleanse, conform, and deliver data for consumption.
• Expert in designing and implementing ETL architecture, including data profiling, process flow, metric logging, and error handling.
• Support continuous improvement by investigating and presenting alternative processes and technologies to an architectural review board.
• Develop and ensure adherence to established system architecture decisions and development standards.
• Manage multiple ongoing projects and daily duties of varying priorities.
• Collaborate with global technical teams to communicate business requirements and develop data solutions.