Databricks Developer :: C2H :: Bengaluru

Role & Responsibilities:

• Over 8 years of IT experience in the big data domain, specializing in data lakes, and data engineering using Python and Spark programming languages.
• At least 3+ years of experience with Databricks workspace, including working with Databricks Notebooks , Job Clusters , Delta Lake , Databricks Lakehouse , and Unity Catalog.
• A minimum of 5 years of experience in PySpark and Python development.
• Over 6 years of experience in designing and implementing data pipelines.
• Proficient in Spark Streaming and its integration into data processing workflows.
• Experienced in designing and developing data pipelines, as well as ETL/ELT jobs for data ingestion and processing in data lakes.
• Strong expertise in SQL Server, NoSQL, Spark SQL , data modeling, identity & access management, query optimization, and parallel processing.
• Proven problem-solving abilities along with excellent verbal and written communication skills.
• Familiar with processing streaming data using Kafka and Pub/Sub technologies.
• Experienced in Agile development, SCRUM methodologies, and Application Lifecycle Management (ALM).
• In-depth knowledge of data warehouse concepts.