Senior Data Engineer (Azure Databricks)

Job Title: Senior Data Engineer (Azure Databricks) Location: Hyderabad / Ahmedabad (Hybrid) Experience: 5+ Years Only candidates available for immediate joining are preferred. Job Overview: We are looking for a Senior Data Engineer with expertise in Azure Databricks to design, develop, and optimize large-scale data solutions. The ideal candidate will have strong experience in ETL processes, big data processing, and distributed computing using Azure Databricks and Apache Spark . This role will focus on building scalable data pipelines, transforming raw data into valuable business insights, and ensuring high-performance data processing . Key Responsibilities: Databricks & Big Data Engineering:

• Design and develop efficient, scalable, and optimized data pipelines using Azure Databricks .
• Implement ETL/ELT workflows for structured and unstructured data using Apache Spark (PySpark, Spark SQL) .
• Work with Databricks notebooks to perform data transformation, aggregation, and complex data operations .
• Optimize data workflows and pipeline performance to ensure cost efficiency and scalability .
• Collaborate with cross-functional teams to enable data-driven decision-making .

• Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
• 5+ years of experience in Data Engineering, with at least 2+ years specializing in Azure Databricks and Apache Spark .
• Strong hands-on experience with PySpark, Spark SQL, and Databricks notebooks .
• Expertise in ETL/ELT development, data transformation, and pipeline optimization .
• Proficiency in Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), and other Azure data services .
• Solid understanding of distributed computing, performance tuning, and cloud data architecture .
• Knowledge of data governance, security policies, and compliance in cloud environments .
• Excellent problem-solving skills with a focus on big data workflows and optimizations .
• Strong communication and collaboration skills.