Sign up

Overview

The PatSnap PatentBench is a benchmark for patent tasks in real-world scenarios. It evaluates the performance of AI tools against curated test samples each consisting of a "test question" and a "standard answer" that closely represent the ideal references used in actual patent work.

Each benchmark is based on hundreds of carefully selected real-world cases, covering different patent tasks, jurisdictions, and industries. The PatSnap PatentBench currently covers three patent workflows:

Novelty Search - a key patent task that involves systematically identifying prior art worldwide to determine whether a technical solution is new and inventive under patent law.

Design FTO Search - a process of searching globally registered design patents to determine whether a product's appearance infringes on existing design patent rights.

FTO Search - an analysis to determine whether a product or process potentially infringes valid claims of existing utility patents in target markets.

The PatSnap PatentBench Methodology

The PatSnap PatentBench Novelty Search benchmark encompasses four key dimensions:

1) Test samples to establish the “benchmark value”

Each test sample is a basic unit in the benchmark, consisting of a “test question” and a “standard answer.” Since ideal answers rarely exist in patent tasks—like a complete list of “X documents” to determine novelty—domain experts must carefully define realistic questions and construct answers that closely approximate ideal outcomes.

2) Datasets to create reliable and unbiased test results

To ensure statistical validity, each benchmark dataset includes hundreds to thousands of carefully selected samples that reflect real-world diversity across regions, patent offices, and industries. Samples are drawn from recent timeframes to avoid overlap with data used in LLM pretraining, ensuring unbiased evaluation.

3) Evaluation metrics - the core of benchmarking

Evaluation metrics are used to measure and compare performance. They can be single or combined indicators, carefully designed by experts to reflect the practical needs of patent professionals.

Novelty search aims to identify relevant prior art to determine whether a patent claim is truly new. It follows the general principles of traditional search logic, but with a specialized focus on patent validity.

The PatSnap PatentBench uses the following indicators to measure the quality of search results:

X Hit Rate

Proportion of samples where a correct answer appears among the top 1, 3, or 5 results

X Hit Rate: Proportion of samples where a correct answer appears among the top 1, 3, or 5 results.

X Recall Rate

Percentage of correct answers found within the top 100 results

X Recall Rate: Percentage of correct answers found within the top 100 results.

4) Comparison AI tools with industry experts

AI tools are designed to support and enhance the work of professionals, so direct comparisons with professional performance are essential. Benchmarks also compare specialized AI agents with general-purpose models like LLMs to establish baselines and better understand their capabilities.

PatSnap Novelty Search AI Agent
Use Cases & Its Impact

PatSnap’s Novelty Search AI Agent stands out in benchmark testing thanks to its domain-specific fine-tuning and advanced Retrieval-Augmented Generation (RAG) technology.

Built on an open-source base model, the agent has been systematically refined with specialized patent knowledge, allowing it to understand the nuances of patent language and search logic. By integrating RAG, it combines real-time data retrieval with generative capabilities, enabling high-quality, low-hallucination search results. This empowers the agent to accurately identify key technical features, apply precise search strategies, and outperform general-purpose models in professional novelty search tasks.

For IP professionals in corporations and patent firms, this translates into a powerful productivity boost. Tasks like searching, filtering, and ranking—once taking hours—can now be completed in minutes. This shift allows experts to spend less time on repetitive work and more on strategic analysis and decision-making, transforming workflows from “three days of searching” to “three hours of insight.”

R&D teams also benefit significantly during early-stage project evaluations. The agent enables fast and effective novelty searches from the outset, helping teams avoid investing in non-novel ideas and reducing wasted resources. This leads to a more efficient and impactful innovation process.