Kafka SME Developer

Job Title: Kafka SME Developer Experience Level: 8+ years Location: Remote Key Responsibilities:

• Lead and contribute to the design, development, and maintenance of Kafka-based systems, ensuring scalability, reliability, and high performance.
• Collaborate with cross-functional teams to define data streaming and messaging requirements.
• Architect and implement solutions using Apache Kafka, Kafka Streams, and Kafka Connect.
• Provide technical leadership and mentorship to junior developers and engineers.
• Troubleshoot and resolve complex Kafka-related issues related to system performance, scaling, and fault tolerance.
• Ensure optimal configuration of Kafka brokers, producers, and consumers for efficient data processing and streaming.
• Develop custom Kafka Connectors and integrate with various systems (e.g., databases, third-party services).
• Implement Kafka security best practices, including encryption, access controls, and monitoring.
• Participate in code reviews, providing feedback on design, implementation, and testing.
• Write and maintain technical documentation for Kafka architecture and solutions.
• Stay up-to-date with the latest Kafka features and contribute to the continuous improvement of our data streaming systems.

• 8+ years of hands-on experience working with Apache Kafka in large-scale, high-throughput environments.
• Strong expertise in Kafka architecture, including brokers, partitions, topics, and consumer groups.
• Proficient in Kafka Streams and Kafka Connect, as well as experience with other messaging and streaming platforms (e.g., Pulsar, RabbitMQ, etc.) is a plus.
• Deep knowledge of message queuing and event-driven architecture.
• Experience with Kafka administration, including broker management, monitoring, and performance tuning.
• Strong programming skills in languages such as Java, Scala, Python, or similar.
• Expertise in integrating Kafka with other big data technologies (e.g., Hadoop, Spark, Flink).
• Solid understanding of containerization and orchestration technologies (Docker, Kubernetes).
• Familiarity with cloud platforms (AWS, Azure, Google Cloud) and deploying Kafka on the cloud.
• Strong understanding of distributed systems, fault tolerance, and high availability principles.
• Experience in designing and implementing secure Kafka solutions, including access control and data encryption.
• Excellent problem-solving skills, with the ability to debug and optimize complex systems.
• Strong communication and collaboration skills, with experience working in a remote, distributed team environment.

• Kafka certification or other relevant certifications.
• Familiarity with DevOps and CI/CD practices.
• Experience with monitoring tools (e.g., Prometheus, Grafana, Confluent Control Center).
• Knowledge of containerized microservices architectures and API design.
• Experience working with Agile/Scrum methodologies.