Big Data Developer
Job details
ITTConnect is seeking a Big Data Engineer to work remotely for a client in the US. This is a long term, ongoing position with a client that is a global leader in consulting, digital transformation, technology and engineering services present in nearly 50 countries. As a Senior Big Data Engineer, you will work on one of the world's largest social media platforms, which deals with a few petabytes of data coming to the system daily. You will contribute as part of R&D self-organized team working in a challenging, innovative environment for our client. Investigate, create, and implement the solutions for many technical challenges using cutting edge technologies, including building/enhancing Big Data processing platform enabling work of software used by hundreds of millions of users. Job location: Remote, work from home anywhere in Brazil, following US Eastern time.
- All interviews will be in English only (must be professionally fluent).
- You must also submit a resume in English.
- Obtains tasks from the project lead or Team Lead (TL), prepares functional and design specifications, approves them with all stakeholders.
- Ensures that assigned area/areas are delivered within set deadlines and required quality objectives.
- Provides estimations, agrees task duration with the manager and contributes to project plan of assigned area.
- Evaluating existing data systems.
- Developing best practices for data coding to ensure consistency within the system.
- Updating and optimizing local and metadata models.
- Evaluating implemented data systems for variances, discrepancies, and efficiency.
- Troubleshooting and optimizing data systems.
- Understand the business drivers and analytical use-cases and translate these to data products.
- Design, implement and maintain pipelines that produce business critical data reliably and efficiently using cloud technology.
- Addresses area-level risks, provides and implements mitigation plan.
- Reports about area readiness/quality, and raise red flags in crisis situations which are beyond his/her AOR.
- Responsible for resolving crisis situations within his/her AOR.
- Initiates and conducts code reviews, creates code standards, conventions, and guidelines.
- Suggests technical and functional improvements to add value to the product.
- Constantly improves his/her professional level.
- Collaborates with other teams.
- University degree in Computer Related Sciences or similar
- 5+ years of experience as a Bigdata Engineer.
- Solid Python and Java coding skill with 5+Years of experience.
- Established big-data systems experience in Hadoop, Spark/SparkSQL, Hive SQL.
- Experience with large-scale graph data storage.
- Strong OOP skills.
- Effective communication (oral & written), collaboration, and interpersonal skills.
- Result oriented approach.
- Experience with Airflow, Apache Atlas.
- Experience with AWS.
- Experience with Kafka.
- Experience with JavaScript and Go.
Apply safely
To stay safe in your job search, information on common scams and to get free expert advice, we recommend that you visit SAFERjobs, a non-profit, joint industry and law enforcement organization working to combat job scams.
