3Pillar is an AI transformation partner on a mission to help enterprises build the AI-native products and intelligent agents that will define the next era of business. With teams across North America, Europe, Latin America, and Asia, we work with the most ambitious companies in financial services, healthcare, media, and technology — helping them move faster, modernize boldly, and compete on their own terms. Our HelixAI platform and Helix Pods delivery model put our engineers at the center of real agentic transformation — doing work that is open, portable, and built to last. We are building the future of enterprise AI.
Senior QA Automation Engineer
You'll own test architecture and semantic validation for a mission-critical AI-powered system where consistency guarantees are non-negotiable. This isn't traditional QA; you're designing test harnesses that prove AI outputs are correct, not just crash-free.
Your core responsibility: build a golden-set validation harness that measures whether the system's LLM stage produces the right decisions at the right confidence levels. You'll design automated tests across unit, integration, E2E, and access-control layers. You'll also establish CI/CD quality gates that block bad builds—schema validation failures, cross-tenant isolation defects, missing floor controls, semantic inconsistencies—before they reach production.
This is front-loaded work (weeks 1–5), then you support the team through delivery. You'll mentor a junior test engineer and set the standard for test coverage in a compressed timeline.
•
Leadership skills
•
Mentoring and coaching abilities
•
Strategic thinking and decision-making skills
•
Project management skills
•
Stakeholder management and collaboration skills
•
Ability to motivate and inspire team members
•
Continuous improvement mindset
•
Familiarity with DevOps processes and tools.
•
Proficiency in programming languages such as Java, Python, or Ruby.
•
Design comprehensive test architecture – plan unit, integration, E2E, and semantic validation test suites that cover the full pipeline from profile intake through roadmap generation; define coverage targets and acceptance criteria
•
Build semantic validation harnesses for AI outputs – design golden-set test cases (5–8 CISO-approved reference profiles) that measure AI stage precision/recall, rigor-tier correctness, floor-control invariants, traceability completeness, and run-to-run stability; a semantic regression blocks merge
•
Implement access-control testing – design automated probes at API and database layers that verify cross-tenant isolation; simulate compromised sessions and verify RLS policies enforce tenant boundaries
•
Design and maintain CI/CD quality gates – build automated test runners that execute on every commit: unit tests, integration tests, schema validation, migration guards, image scanning, access-control probes; red builds block merge
•
Lead test automation practices – expand automation frameworks and practices; identify opportunities for improving test processes; recommend tools and technologies for testing efficiency
•
Collaborate across the team – serve as the QA voice in technical design reviews; flag testability issues early; work with the AI/LLM Lead to define what "correct" means for each pipeline stage
•
Mentor junior test engineers – guide the team on test design, semantic validation, and CI/CD best practices; share knowledge on testing AI systems where reproducibility is critical
•
Own test documentation and reporting – maintain clear test plans, results, and traceability matrices; report test coverage and quality metrics to leadership; communicate blockers and risks transparently
•
5+ years of hands-on software QA with a track record of:
•
Test strategy, planning, execution, and maintenance
•
Comprehensive test documentation and traceability
•
Advanced test automation (custom frameworks, advanced scripting)
•
3+ years with automated testing tools – Selenium, Pytest, or equivalent; ability to write custom testing frameworks
•
Advanced proficiency in a programming language relevant to test automation:
•
Python (primary for this engagement; Pytest, async testing)
•
Or Java, C#, Go
•
Deep knowledge of databases and SQL – execute complex data validation, verify multi-tenant isolation at the DB layer, test RLS policies
•
CI/CD pipeline expertise – understand how tests integrate into automated deployment workflows; experience optimizing test execution for speed and reliability
•
Understanding of Agile/Scrum methodologies – ability to drive QA strategies in rapid, iterative development cycles
•
Proficiency in defect tracking (Jira or equivalent)
•
Proven leadership – mentor junior engineers, advocate for quality standards, drive process improvements
•
Exceptional communication skills – articulate complex technical issues to engineers, product, and stakeholders
•
Self-motivated and detail-oriented – work independently under time pressure, manage multiple priorities
•
Exceptional English proficiency – work effectively with distributed, global teams
•
ISTQB Advanced Level, CSTE, or equivalent QA certification
•
Performance and load testing (JMeter, LoadRunner) – design and execute tests under various conditions
•
Security testing practices – knowledge of OWASP vulnerabilities, experience with security testing tools (OWASP ZAP, Fortify)
•
Test-driven development (TDD) and behavior-driven development (BDD) – familiarity with Gherkin, Cucumber, or equivalent
•
Experience testing multi-tenant systems – understanding of isolation, RLS, and cross-tenant security validation
•
Experience with AI/ML systems – familiarity with testing LLM outputs, semantic validation, and consistency guarantees
•
API testing – REST/GraphQL endpoint validation, contract testing
•
Experience in regulated environments – audit logging, reproducibility, compliance testing
•
Demonstrated track record of process improvement – initiated and led projects that enhanced quality, reduced costs, or improved efficiency
•
Medical Insurance benefits as per company policy.
•
Life Insurance as per company policy
•
12 days of paid vacation, sick leave and paid holidays as per local law
•
Paternity and maternity leave as per as per local law
•
Marriage, bereavement and graduation leaves as per company policy
•
Sick leave and paid holidays as per local law
•
Christmas bonus as per local law
•
Saving Fund “Asociacion Solidarista”
•
Beach house (Managed by the Asociacion Solidarista)
•
Discounts in local shops
•
Direct deposit of payroll.
•
Paid professional certifications
At 3Pillar, we offer a world of opportunity:
•
Imagine a flexible work environment – whether it's the office, your home, or a blend of both. From interviews to onboarding, we embody a remote-first approach.
•
You will be part of a global team , learning from top talent around the world and across cultures, speaking English everyday. Our global workforce enables our team to leverage global resources to accomplish our work in efficient and effective teams.
•
We’re big on your well-being – as a company, we spend a whole trimester in our annual cycle focused on wellbeing. Whether it is taking advantage of fitness offerings, mental health plans (country-dependent), or simply leveraging generous time off, we want all of our team members operating at their best.
•
Our professional services model enables us to accelerate career growth and development opportunities - across projects, offerings, and industries.
•
We are an equal opportunity employer. It goes without saying that we live by values like Intrinsic Dignity and Open Collaboration to create cutting-edge technology AND reinforce our commitment to diversity - globally and locally.
Join us and be a part of a global tech community! 🌍💼 Check out our Linkedin site and Careers page to learn more about what it’s like to be part of our #oneteam!