Home Online Data Engineer(Scala & Spark)

Home Online Data Engineer(Scala & Spark)

Data Engineer(Scala & Spark)

Full time at Capgemini in Online
Posted on April 1, 2024

Job details

  • Job Title: Data Engineer(Scala and Spark)
  • Location: London
  • Travel/Relocation/Remote: Travel to client’s location may be required depending on project requirement (Mode – Hybrid)
  • As a Data Engineer within the exciting, new Finance Risk and Data Analytics capability, you will be building big data solutions to solve some of the organization's toughest problems and delivering significant business value. This is an exciting time to join as you will be helping to shape the Reference Data Mastering and Distribution architecture and technology stack within our new cloud-based data lake-house. Qualifications
    • Meaningful experience in following technologies: Scala , SQL
    • Experience and interest in Cloud platforms such as AZURE Data Factory.
    • Experience in Distributed Processing using Apache Spark
    • Ability to debug using tools like Ganglia UI, expertise in Optimizing Spark Jobs
    • The ability to work across structured, semi-structured, and unstructured data, extracting information and identifying linkages across disparate data sets.
    • Expert in creating data structures optimized for storage and various query patterns for e.g., Parquet and
    • Delta Lake
    • Meaningful experience in at least one database technology such as:
    • Traditional RDBMS (MS SQL Server, Oracle)
    • NoSQL (MongoDB, Cassandra, Neo4J, Cosmos DB, Gremlin)
    • Understanding of Information Security principles to ensure compliant handling and management of data.
    • Experience in traditional data warehousing / ETL tools (Informatica, Azure Data factory)
    • Ability to clearly communicate complex solutions.
    • Proficient at working with large and complex code bases (GitHub, Gitflow, Fork/Pull Model)
    • Working experience in Agile methodologies (SCRUM, XP, Kanban)
    What You Will Be Doing
    • Shape the portfolio of business problems to solve by building detailed knowledge of internal data sources.
    • Model data landscape, obtain data extracts and define secure data exchange approaches.
    • Acquire, ingest, and process data from multiple sources and systems into Cloud Data Lake
    • Operate in the fast-paced, iterative environment while remaining compliant with bank’s Information Sec
    • policies/standards
    • Collaborate with others to map data fields to hypotheses and curate, wrangle, and prepare data for use in
    • their advanced analytical models
    • Help architect the strategic advanced analytics technology landscape.
    • Build reusable code and data assets.
    What We Need
    • Experience in software development, including a clear understanding of data structures, algorithms,
    • software design and core programming concepts
    • Comfortable multi-tasking, managing multiple stakeholders and working as part of agile team.
    • Excellent communication skills including experience speaking to technical and business audiences and
    • working globally
    • Expertise in Spark & Distributed Datasets design patterns
    • Strong problem solving and analytical skills.
    • Keen to learn and share new concepts.
    About Capgemini Capgemini is a global leader in partnering with companies to transform and manage their business by harnessing the power of technology. The Group is guided everyday by its purpose of unleashing human energy through technology for an inclusive and sustainable future. It is a responsible and diverse organization of over 360,000 team members in more than 50 countries. With its strong 55-year heritage and deep industry expertise, Capgemini is trusted by its clients to address the entire breadth of their business needs, from strategy and design to operations, fueled by the fast evolving and innovative world of cloud, data, AI, connectivity, software, digital engineering, and platforms. The Group reported in 2022 global revenues of €22 billion. Get The Future You Want | What we'll offer you Professional development. Accelerated career progression. An environment that encourages entrepreneurial spirit. It’s all on offer at Capgemini. And although collaboration is at the core of the way we work, we also recognize everyone’s needs with flexible benefits package you can tailor to suit you. Why we're different. At Capgemini, we help organizations across the world become more agile, more considerate, and more successful. Smart, tailored, often-ground-breaking technical solutions to complex problems are the norm. But so, too, is a culture that’s as collaborative as it is forward thinking. Working closely with each other, and with our clients, we get under the skin of businesses and to the heart of their goals. You will too Inclusion Capgemini positively encourages applications from suitably qualified and eligible candidates regardless of sex, race, disability, age, sexual orientation, gender reassignment, religion or belief, marital status, or pregnancy and maternity. We are committed to hiring, developing, and retaining the best people to deliver innovative, world-class solutions for our clients. We foster an inclusive culture that enables everyone to achieve their full potential and enjoy a fulfilling career with us. Our comprehensive flexible benefits package and lifestyle policies enable our employees to balance their family and work-life needs.

Apply safely

To stay safe in your job search, information on common scams and to get free expert advice, we recommend that you visit SAFERjobs, a non-profit, joint industry and law enforcement organization working to combat job scams.

Share this job
See All Data Jobs
Feedback Feedback