Automation Test Analyst - Senior Manager

About Vialto Labs (VLabs)

Vialto Labs (VLabs)is responsible forredesigning how work is delivered in the tax and immigration service lines, as well as driving operational efficiency acrossVialto’sfunctional areas using AI. The team builds and deploys novel AI-enabled solutions that directly improve productivity and increase delivery quality for our clients.VLabsis accountable for rapidly turning innovative experiments into production-ready deliverables at scale and embedding them into day-to-day operations. This team focuses on the highest-impact workflows, creating standardized, repeatable capabilities that can be deployed globally. Operating with a mandate for speed and measurable outcomes,VLabsworks alongside serviceline,product, and platform leaders.

About the Role

The Automation Test Analyst is a hands-on role responsible for building and scaling automation capabilities within theVLabsdelivery lifecycle. This roleoperateswithin sprint cycles, embedding automation into development rather than executing post-build validation.The role focuses on ensuring the reliability, stability, andproductionreadiness of both traditional and AI-enabled platform components. Given the presence of AI-generated outputs, this role requires designing resilient validation approaches based on structure, confidence, and range rather than static assertions.

Working closely with AI Test Engineering, product, and engineering teams, this role contributes to continuous testing, release readiness, and scalable quality assurance aligned toVLabs’ speed and delivery expectations.

Key Responsibilities

Test Automation Development

• Build andmaintainPlaywright-based UI regression suites covering core platform workflows including case creation, document processing, workflow interaction, and deliverable generation

  
  
  

• Develop API contract tests using Postman/Newman for document pipelines, classification outputs, extraction results, and system integrations

  
  
  

• Create Python and SQL-based data validation scripts tovalidatefield mapping, data integrity, and transformation logic

  
  
  

AI-Aware Validation

• Design validation approaches for AI-generated outputs using structural checks, field presence, value ranges, and confidence indicators

  
  
  

• Partner with AI Test Engineering toidentifyand automate applicable AI output validations

  
  
  

• Ensure test design accounts for variability in AI outputs whilemaintainingreliability and meaningful validation

  
  
  

Sprint-Embedded Automation

• Build automation incrementally within sprint cycles, prioritizing API validation, followed by data validation and UI regression

  
  
  

• Ensure regression suites are continuously runnable andprovideactionable insights for each release cycle

  
  
  

• Support release readiness by contributing to automated validation within CI/CD pipelines where applicable

  
  
  

Data & Test Environment Management

• Maintain test data pipelines including parameterized datasets, database seeding, and volume test preparation

  
  
  

• Ensure test environments are stable, repeatable, and aligned with automation requirements

  
  
  

Integration & Quality Assurance

• Partner with engineering teams to ensure testability, observability, and reliability of platform components

  
  
  

• Validate end-to-end workflows including AI-integrated components and multi-step processes

  
  
  

• Support performance testing efforts through JMeter contributions and result interpretation

  
  
  

Standardization & Scaling

• Contribute to reusable automation frameworks and standardized testing patterns acrossVLabs

  
  
  

• Design automation with sufficient abstraction to localize failures and reduce maintenance overhead

  
  
  

• Support continuous improvement of automation coverage and efficiency

  
  
  

Governance & Reporting

• Integrate automated test results into ADO Test Plans with traceability to test cases and release milestones

  
  
  

• Provide outputs that support milestone sign-off and production readiness decisions

  
  
  

• Ensure adherence to quality, security, and governance standards in testing practices

  
  
  

Qualifications & Experience

Professional Experience

• 4+ years in test automation as a primary discipline

  
  
  

• Demonstrated experience working with AI-enabled systems, including GenAI, LLMs, or agentic applications

  
  
  

• Proven ability to build automation frameworks in active, sprint-based development environments

  
  
  

• Experience designing validation strategies for variable or non-deterministic system outputs

  
  
  

• Strong practical judgment on where to prioritize automation investment for maximum impact

  
  
  

Technical Expertise

Playwright:

• Development of scalable UI automation frameworks using page object models

  
  
  

• Handling dynamic, asynchronous, and AI-populated interfaces

  
  
  

• Headless execution for CI/CD and debugging workflows

  
  
  

Postman/ Newman:

• API contract testing with parameterized, data-driven scenarios

  
  
  

• Multi-step workflow validation and CI/CD integration

  
  
  

• Python:

  
  
  

• Data validation scripting, JSON/XML parsing, and automation utilities

  
  
  

• Development of reusable validation and testing components

  
  
  

SQL:

• Data validation, persistence checks, and mapping verification

  
  
  

JMeter:

• Basic performance testing support and result interpretation

  
  
  

ADO:

• Test case management, automation integration, and defect tracking

  
  
  

Operating Capabilities

• Ability tooperateindependently in fast-paced, iterative environments

  
  
  

• Strong problem-solving and analytical skills with attention to detail

  
  
  

• Ability to balance speed and rigor in automation development

  
  
  

• Proactive mindset inidentifyingautomation opportunities and improving test coverage

  
  
  

• Strong collaboration with engineering and AI teams

  
  
  

Education

• Bachelor’s degree required

  
  
  

• Advanced degree in Computer Science, Data Science, or related field preferred

  
  
  

Additional Information:

• This job is based in our Bangalore office with the possibility of hybrid work mode

  
  
  

• We are an equal opportunity employer that does not discriminate on the basis of any legally protected status

  
  
  

• Please note, AI is used as part of the application process