🛠 Tech Stack

💼 About This Role

You'll own the quality roadmap for Joist AI's AI-powered RevOps platform, focusing on complex document processing and LLM architectures. You will architect testing frameworks for AI accuracy and compliance, ensuring best-in-class performance for enterprise customers. This role offers AI/LLM validation challenges and automation architecture leadership.

🎯 What You'll Do

• Define and implement end-to-end testing strategy for Proposal Processing Pipeline and Knowledge Graph.
• Develop novel methods to test LLM narratives for accuracy, hallucination reduction, and compliance.
• Own quality of extraction services, object/image detection, and captioning services.
• Lead automated test suite development using Playwright and API automation integrated into CI/CD.

📋 Requirements

• 6+ years of professional QA experience
• SQL proficiency for validating RDS and OpenSearch data structures
• Advanced automation skills in Python or Java
• Expertise in Postman/Swagger for API testing in microservices architecture

✨ Nice to Have

• AEC industry knowledge (SMPS proposal process or RFP workflows)
• Experience with Langchain or evaluating LLM performance metrics
• Interest in compliance or improving Developer Experience

🎁 Benefits & Perks

• 🏖️ Flexible remote work
• 💰 Competitive compensation (not explicitly listed, inferred as likely standard)
• 📈 Growth opportunities in a high-growth startup

📨 Hiring Process

30-min Zoom with hiring manager, 30-min Zoom with team member, take-home project, 60-min project review call.