Barclays – QA (AI)

Barclays is hiring a highly skilled QA – AI professional to join its Customer Digital and Data technology division in Pune. This role is focused on ensuring quality, reliability, and governance across cutting-edge Artificial Intelligence and Generative AI applications. As financial institutions rapidly adopt Large Language Models (LLMs), Agentic AI systems, and machine learning platforms, the need for advanced AI testing and validation has never been greater. In this position, you will design and implement comprehensive testing strategies that validate functionality, performance, scalability, and ethical compliance of AI-driven solutions operating at enterprise scale.

This role goes beyond traditional software testing. You will work extensively with LLM frameworks such as LangChain and LangGraph, validate RAG (Retrieval-Augmented Generation) architectures using vector databases, and assess model outputs through evaluation frameworks like RAGAS and DeepEval. From Natural Language Processing (NLP) quality assurance to bias detection and responsible AI validation, your work will ensure that AI systems deployed within Barclays meet regulatory standards, customer expectations, and enterprise-grade reliability benchmarks. This position sits at the intersection of AI innovation, DevOps excellence, and cloud engineering.

As part of Barclays’ technology ecosystem, you will collaborate with cross-functional teams including development, product, operations, and data engineering. Leveraging AWS services such as Bedrock, Lambda, and CloudWatch, along with CI/CD pipelines powered by Jenkins or GitLab CI, you will embed quality gates across the software development lifecycle. This role is ideal for professionals who thrive in Agile environments, possess strong Python automation skills, and are passionate about shaping the future of AI quality engineering in the financial technology industry.

Roles & Responsibilities:

1. Design and implement comprehensive testing strategies for Generative AI, LLM-based systems, and Agentic AI applications to ensure functional accuracy and performance stability.

2. Develop scalable Python-based automation frameworks to validate AI models, APIs, and enterprise cloud deployments across distributed environments.

3. Perform machine learning model testing, including validation of training data quality, prediction accuracy, bias detection, and ethical AI compliance checks.

4. Execute NLP and NLU testing strategies to validate language understanding, contextual relevance, and output consistency in production AI systems.

5. Conduct RAG testing using vector databases and evaluation frameworks such as RAGAS or DeepEval to assess model response quality and retrieval effectiveness.

6. Architect and optimize CI/CD pipelines using tools like Jenkins and GitLab CI to integrate automated AI testing within DevOps workflows.

7. Perform performance, scalability, and load testing for AI-powered applications hosted on AWS services including Bedrock, Lambda, and CloudWatch.

8. Collaborate with cross-functional stakeholders to define acceptance criteria, participate in design discussions, and establish enterprise-level quality gates.

9. Conduct root cause analysis for defects in AI outputs or system behavior, ensuring clear documentation and alignment with development teams.

10. Promote a culture of testing excellence, knowledge sharing, and continuous improvement across teams, aligning with Agile/Scrum methodologies and SDLC best practices.

11. Establish governance frameworks for Gen AI applications, ensuring risk mitigation, compliance adherence, and responsible AI standards.

12. Support UI, mobile, and API testing initiatives where AI systems integrate with broader digital banking platforms.

Requirements & Eligibility:

1. Strong understanding of Large Language Models (LLMs), LangChain, LangGraph, and modern Generative AI architectures used in enterprise environments.

2. Advanced Python programming skills with demonstrated experience building scalable test automation frameworks for AI and cloud-native applications.

3. Hands-on experience with Generative AI and Agentic AI implementations in production, including testing and validation of real-time AI systems.

4. Deep knowledge of Machine Learning testing methodologies, covering model validation, bias detection, explainability, and responsible AI practices.

5. Practical experience in NLP and NLU quality assurance strategies to validate semantic accuracy and contextual intelligence in AI outputs.

6. Familiarity with RAG architectures, vector databases, and evaluation frameworks such as RAGAS, DeepEval, or similar LLM evaluation tools.

7. Comprehensive knowledge of AWS cloud services, particularly Bedrock, Lambda, and CloudWatch, along with enterprise-scale cloud testing strategies.

8. Proven expertise in CI/CD implementation using Git, Stash, Jenkins, GitLab CI, and branching strategies aligned with DevOps culture.

9. Experience with BDD frameworks, test management tools like Jira, Xray, ALM, or TestRail, and governance frameworks for enterprise AI systems.

10. Strong leadership capabilities or advisory expertise, with the ability to influence stakeholders, guide teams, and embed risk-aware testing practices.

Preferred qualifications include exposure to Docker and Kubernetes for test environments, data engineering or data quality testing backgrounds, and contributions to AI testing communities or open-source initiatives.

Expected Salary:

For an AI-focused QA role at Barclays in Pune, the expected salary typically ranges between ₹18 LPA to ₹30 LPA, depending on experience with Generative AI, cloud architecture, and enterprise testing leadership. Professionals with deep expertise in LLM validation, AWS-based AI deployments, and CI/CD automation may command compensation at the higher end of this range. Barclays also offers performance bonuses, healthcare benefits, retirement plans, and flexible working arrangements aligned with industry-leading financial institutions.

🚨 Before You Apply: Your Resume Needs to Shine!

Did you know? 75% of applications get rejected before reaching a human recruiter – all because of poorly formatted resumes that fail ATS scans!

🔥 Get Interview-Ready in Minutes with Our Professionally Designed Resume Templates!

✅ 5+ ATS-Friendly Designs – Beat the bots and get noticed
✅ Recruiter-Approved Layouts – Highlight your skills the right way
✅ Easy-to-Edit (Word & Google Docs) – No design skills needed
✅ Free Bonus: Cover Letter Template + Resume Writing Guide

🎁 Limited-Time Offer: Get yours for just ₹249 (originally ₹999)
📥 Instant Download – Apply to Google with confidence today!

👉 Grab Your Resume Template Now: Tap Here to get your resume Templates