AI / ML Engineer

About PieLabs PieLabs is at the forefront of revolutionizing app testing with our autonomous system, Pie. Leveraging GenAI, Pie tests applications end-to-end by interacting with them just like real users would. Pie replicates the capabilities of an experienced testing team but with the speed and efficiency of automated e2e testing, eliminating the need for manual test plan maintenance. As our clients’ applications evolve, Pie adapts, ensuring continuous and comprehensive coverage. We’re seeking an AI/ML Engineer with expertise in Large Language Models (LLMs) to enhance our AI-driven system. You’ll be working in a collaborative environment that values innovation and enables you to explore new ways to leverage AI in transforming app quality assurance. Role Overview As an AI/ML Engineer, you will focus on developing and optimizing LLM-driven solutions within our autonomous testing platform. You’ll be responsible for benchmarking, fine-tuning, and deploying LLM models, ensuring they adapt effectively to user interactions and complex workflows. Collaborating closely with our product and engineering teams, you’ll play a pivotal role in ensuring Pie remains at the cutting edge of AI-based testing. Key Responsibilities

• Design, build, and deploy LLM-based models to enhance Pie’s autonomous testing capabilities.
• Benchmark and fine-tune models to improve performance, reduce latency, and ensure accuracy.
• Develop methods to continuously train and adapt models as client applications evolve.
• Conduct performance analysis on AI models, refining them based on user interactions and testing scenarios.
• Collaborate with the engineering team to integrate AI/ML solutions seamlessly into our tech stack.
• Stay updated with the latest advancements in AI/ML, particularly LLMs, and apply relevant innovations to our platform.
• Implement tools and processes for model monitoring, performance tracking, and anomaly detection.

• Proven experience (3+ years) as an AI/ML Engineer, with a focus on LLMs and natural language processing.
• Proficiency in model benchmarking, fine-tuning, and optimization of large-scale AI models.
• Experience working with ML frameworks such as PyTorch, TensorFlow, and Hugging Face Transformers.
• Strong understanding of NLP and GenAI methodologies, with a record of applying them to practical applications.
• Familiarity with model deployment in cloud environments (e.g., AWS, GCP, or Azure).
• Excellent coding skills in Python or Go, with a focus on writing efficient, clean, and scalable code.
• Experience in data management, including preprocessing, feature engineering, and dataset curation.
• Familiarity with evaluation metrics and methods for LLMs, including techniques for model improvement.
• Strong analytical and problem-solving skills, with the ability to work collaboratively in a remote setting.

• Experience with model interpretability and explainability.
• Familiarity with Reinforcement Learning from Human Feedback (RLHF).
• Knowledge of MLOps tools and practices for model lifecycle management.
• Prior experience in automated testing, quality assurance, or similar domains.

• Competitive salary and equity package.
• Flexible working hours and remote work setup.
• Opportunities for professional growth in a collaborative environment focused on cutting-edge technology.
• Access to learning resources, conferences, and workshops.