Big Data Developer with Spark [Immediate joiner needed]

Project Description: The Cirrus project is developing cloud-based compliance state-of-the-art archive products to archive and retain real-time communications data in line with internal and external regulatory requirements. The product is developed in-house in an agile development setup, based on business requirements from multiple stakeholder parties. By employing continuous development and deployment principles, the team is aiming to transition from project to product management to support the bank with a robust compliance archive solutions for the next decade. For a large global investment bank, we are looking for GCP-qualified cloud engineers to help with the FIC (fixed income and currencies) cloud migrations under project Cirrus. Knowledge of Financial Services/FIC would be great - but the primary skills we need are in building, migrating, and deploying applications to GCP, terraform module coding, google infrastructure, cloud-native services such as GCE, GKE, CloudSQL/Postgres, logging, and monitoring, etc. & good written and spoken English, as we would like these engineers to help with knowledge transfer to our existing development & support teams. We would like to place people alongside the engineers they'll be working with in the bank. Responsibilities: 1. Creation of a robust solution design that supports all functional and non-functional requirements 2. Assist in the creation of automated testing tooling that will reduce the time spent on manual testing 3. Assist in the creation of DevOps tooling to establish strong automation for the software build and deployment processes 4. Follow bank processes and use industry-standard tools for implementation and productization of software 5. Involvement at all stages of SDLC: analysis, design, development, unit and integration testing, UAT facilitation, packaging, and deployment Cooperation with other development teams, functional analysts, business analysts, and numerous stakeholders of the project, as appropriate for the area of work Mandatory Skills: • 7+ years of experience in Big Data technologies • 3+ years of Extensive experience with Big Data technologies/engineering (Hadoop, HDFS, MapReduce, and other big data tools)• Handling data and persistence of data using databases, key-value stores (SQL, NoSQL)• Data streaming pipelines employing Kafka, Spark • Design/implementation of reporting solutions using Big Data technologies • Build tools like Maven, Gradle • Testing tools like Postman, curl, JUNIT, Mockito • Strong coding methodologies, design patterns, and best practices • Strong ability to work independently, propose architectural solutions, create prototypes, and deliver necessary technical documentation • Ability to work on collaboration with local and external teams • Relational and object databases like MongoDB, Sybase • Experience with CI (Continuous Integration) tools (Jenkins) and code quality (e.g., SonarQube), Code Coverage, Unit tests, etc. • Linux administration and maintenance, including bash scripts • Solid understanding of networking, DNS management, and firewall policies • Queue systems such as RabbitMQ, Apache ActiveMQ, Apache Kafka • Alerting and monitoring for cloud platform with tools such as Kibana, Grafana In addition, we seek someone with experience in developing reporting solutions on a data lake such as HDFS : • Scala developer with Hadoop/HDFS knowledge is a must • Experiencing of hosting data on HDFS • Experience in writing reporting services on data in HDFS • Familiarity or experience with the following tools: Dremio/SPARC/Notebook/Tableau You should also be or have: • Ability to provide technical expertise with a focus on efficiency, reliability, scalability, and security; including planning, evaluating, recommending, designing, operationalizing, and supporting solutions in compliance with enterprise and industry standards. • Ability to plan, evaluate, recommend, design, operationalize, and support solutions in compliance with enterprise and industry standards. • Ability to lead and help team members on technical implementations. • Ability to work on multiple initiatives at the same time • Familiar with Agile, and SDLC processes Nice-to-Have Skills: Advanced full-stack Java programming skills Domain knowledge in Investment Banking & Trading (ideally FX Options) will be advantageous Strong analytical and technical background with good problem-solving skills Good team-working and communication skills Willing to go the extra mile to achieve team objectives Languages: English: C2 Proficient