Principal Quality Engineer - AI/ML

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.  

The team builds and operates an AI-powered chat platform that supports pharmacy members across digital channels. It helps deliver reliable, contextual support for common pharmacy and benefits-related needs, and is built on scalable cloud infrastructure.

Work Details

• Willing to work in Alabang or Quezon City
• Shift Schedule: 5PM - 2AM PHT

Primary Responsibilities:

This Principal Quality Engineering role requires a hybrid of manual testing, strong scripting-based automated testing, and familiarity with AI-specific evaluations to ensure correctness, safety, and regulatory readiness across a multi-layer agent system.

• Set quality vision and strategy for the AI-powered conversational platform, leading the design of comprehensive testing approaches and governance frameworks
• Mentor quality engineering team members, driving adoption of testing best practices and building a culture of quality excellence
• Execute both manual and automated testing for AI agent workflows across chat experiences
• Build and maintain Python scripting-based test automation for API validation, tool invocation correctness, data verification, and end-to-end conversational scenario testing
• Design and run test suites across functional, integration, system, regression, and exploratory testing layers
• Validate AI agent behavior for intent handling, response quality, tool-selection correctness, and conversation context and memory continuity across multi-turn interactions
• Design and build scalable test automation frameworks and infrastructure that support high-velocity AI/ML deployments and continuous quality monitoring
• Perform AI bias, fairness, safety, and adversarial testing, including sensitive data handling, prompt injection scenarios, escalation correctness, and out-of-scope topic blocking
• Define and track quality metrics and KPIs for AI systems, including accuracy thresholds, bias detection signals, and compliance controls; establish go/no-go decision criteria for releases
• Support AI Review Board (Governance) readiness through evidence-based test reporting, risk documentation, and traceable quality controls aligned to compliance and governance standards
• Execute scale and performance tests (concurrency, latency, throughput) and identify bottlenecks across backend service and API chains
• Drive cross-functional alignment on quality standards and risk acceptance across product, engineering, compliance, and leadership teams
• Establish production monitoring and continuous evaluation strategies for AI models, detecting quality drift and triggering remediation
• Work closely with engineering and product teams to triage defects, drive root-cause analysis, and verify fixes through closure
• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Required Qualifications:

• 10+ years of engineering experience, including 6+ years leading quality engineering initiatives with both hands-on testing and team leadership
• Experience defining quality strategies, roadmaps, and standards for complex, customer-facing digital platforms
• Experience testing APIs and microservices, including payload validation, response schemas, error handling, and end-to-end flows across distributed systems
• Experience in functional, integration, and system testing in modern software delivery environments
• Experience with CI/CD-integrated test execution and quality gate configuration
• Experience with performance and load testing, including tools such as JMeter, Locust, or Python async clients, and familiarity with latency, throughput, and concurrency analysis
• Experience building and scaling test automation frameworks and reusable testing infrastructure
• Experience establishing quality gates, release criteria, and metrics-driven decision frameworks
• Experience with log analysis and production issue investigation
• Experience with Python testing tooling such as cloud SDK scripting, API testing tools such as Postman/Insomnia, and cloud observability dashboards
• Solid Python scripting capability for QA automation frameworks, test data generation, and results reporting
• Working knowledge of JavaScript for test scripting and front-end integration validation
• Proven solid ability to work collaboratively across diverse groups of business and technical stakeholders

Preferred Qualifications:

• Experience with AI/LLM evaluation frameworks like scoring patterns, retrieval evaluation, or managed model evaluation services for metrics such as faithfulness, relevancy, completeness, and instruction following
• Experience with bias, toxicity, hallucination, and factuality testing in AI/LLM systems
• Experience with security and guardrail testing like prompt injection, adversarial inputs, sensitive data masking verification, and safe response generation validation
• Experience with performance and load testing tools for conversational AI systems
• Experience with post-production AI evaluation observability platforms
• Experience with production AI/ML observability and continuous evaluation platforms
• Experience driving quality culture transformation and building high-performing QA organizations
• Experience with AI governance frameworks, including risk assessment and compliance controls for regulated AI systems
• Healthcare or pharmacy domain experience and compliance-sensitive quality assurance practices, including awareness of protected health and personally identifiable information handling requirements
• Track record of leading cross-functional quality initiatives and influencing product roadmap decisions based on quality insights
• Familiarity with cloud-native architectures for AI systems, including serverless compute, managed API services, cloud monitoring, and distributed data stores

At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission.

Optum is a drug-free workplace. © 2026 Optum Global Solutions (Philippines) Inc. All rights reserved.