تفاصيل الوظيفة
We are looking for Candidates who are having experience in Big Data. Experience Required: 3+ years. Location: PAN India Responsibilities:
- Design, develop and deploy solutions using different tools, design principles and conventions.
- Assemble large, complex data sets that meet functional / non-functional business requirements.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources and ‘big data’ technologies.
- Optimize data ingestion and processing performance by tuning configuration parameters, optimizing SQL queries, and implementing parallel processing techniques.
- Monitor pipeline performance and resource utilization to identify bottlenecks and optimize system resources allocation.
- Set up monitoring and alerting systems to track pipeline health, detect anomalies, and notify stakeholders of potential issues or failures.
- Configure logging and metrics collection to facilitate troubleshooting and performance analysis.
- Understand existing processes and facilitate change requirements as part of a structured change control process.
- Maintain proper documentation for the solutions, test procedures and scenarios during UAT and Production phase.
- Understand and implement Pipeline Development, Data Quality and Governance, Monitoring and Alerting, Data Virtualization and Query Optimization, Data Source Integration, Data Catalog Management, Security and Access Control, Integration with Applications and Ecosystems, Data Ingestion and Management
- Coordinate with process owners and business to understand the as-is process and design the automation process flow.
- Produce high-quality code resulting from knowledge of the latest frameworks, code peer review, and automated unit test scripts.
- Collaborate with other developers and team members.
Apply safely
To stay safe in your job search, information on common scams and to get free expert advice, we recommend that you visit SAFERjobs, a non-profit, joint industry and law enforcement organization working to combat job scams.
