Data Architect (PySpark, Python, Spark, ETL)

NorthBay Solutions , with a team of 400+ professionals, is a US-based premier AWS partner and the only premier-level partner in the region. We have been serving our clients globally and providing AWS Big Data, Data Lake & Data Warehousing solutions for over a decade now.We are currently recruiting for a Tech Lead/Architect with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of overall12+ years and 5+ years specifically in data engineering .Technology Stack Used & Required Knowledge:Must have 12+ years experience of developing and implementing pipelines onAWS cloudthat extract, transform, and load data into an information product that helps the organization reach its strategic goals.

Must have experience onETL Data Engineer, Python, Bigdata (PySpark, Spark SQL, Hadoop, Hive) and/or Java .

Experience in creating and driving large-scale ETL pipelines in AWS-based environment.

Experience with integration of data from multiple data sources.

Overall 5+ years of relevant work experience in Big data engineering, ETL, Data Modeling, and Data Architecture.

Strong software development and programming skills with a focus on data using Python/ PySpark and/or Scala for data engineering.

Experience and understanding of various core AWS services such as IAM, Cloud Formation, EC2, S3, EMR/Spark, Glue, Lambda, Athena, and Redshift will be a plus.

Experience with the AWS data management tools such as Data lake and Databricks or AWS Snowflake is a plus.

Key Responsibilities:Design and develop using Big Data technologies.

Design and develop cutting edge Big Data, and Cloud technologies to build critical, highly complex distributed systems from scratch.

Design, develop and manage complex ETL jobs and pipelines.

Act as a critical thought leader, consulting and mentoring other teams as your new systems transform the way this entire firm leverages data.

Note : NorthBay Solutions follows a hybrid-work-model.#J-18808-Ljbffr