AI patent infringement detection: how to evaluate screening

How do you tell whether artificial intelligence (AI) patent infringement detection will produce useful leads for your team? A ranked list leaves patent counsel and intellectual property (IP) leaders with questions about source coverage, evidence quality, and potential misses. A practical evaluation makes those questions concrete: define the screening task, review returned candidates, and look for known products that should have appeared.

What a screening evaluation should answer

Scope: This workflow evaluates AI screening of candidate products and supporting evidence against selected U.S. patent claims.

Separate 2 questions before comparing platforms:

• Candidate screening: Did the workflow return products that meet your agreed threshold for further investigation against the selected claims?
• Evidence mapping: Does each cited passage support the limitation, product version, and technical proposition assigned to it?

A claim limitation is an individual requirement within a patent claim. Count each selected claim–product combination once, however many citations support it.

The infringement-search methods comparison explains where conventional research and AI assistance fit.

1. Define source coverage before running the screen

Start with a bounded research task. Record the patent, selected claims and their exact text, the working claim interpretation, target companies and products, jurisdiction, product versions, and relevant period. Hold these inputs constant when comparing results.

Create a short coverage record for each source category:

• Source identity: Manufacturer manuals, datasheets, technical support pages, regulatory records, or other named repositories.
• Boundary: Included companies, product categories, languages, territories, versions, and dates.
• Access: Indexed material, live retrieval, supplied documents, or material requiring a separate collection step.
• Availability: Included, inaccessible, excluded by design, or unknown.
• Preservation: Collection date, source identifiers, and retained copies where permitted.

Record the reason for each access gap or exclusion, including login restrictions, missing historical versions, and excluded languages.

When comparing platforms on supplied documents, give each the same source collection. For live discovery, record the run date and available sources so changes in public documentation remain visible.

2. Agree on what makes a candidate useful

Define “relevant” in terms of the screening decision. For example, a candidate might qualify for follow-up when identified product documentation supports specified technical requirements and leaves a concrete investigable gap. Apply the same rule to returned candidates and known products used to look for misses.

Classify the candidates you can assess as relevant or irrelevant. Record the evidence state for each:

• Direct support: The source expressly describes the technical proposition being assessed.
• Partial support: The source supports part of that proposition.
• Inference required: The reviewer must infer the proposition from disclosed facts.
• Irrelevant: The material fails the agreed relevance criterion.
• Unverifiable: Product identity, source access, version, or content prevents a reliable judgment.

Assign technically qualified reviewers to apply the agreed criteria. Have reviewers assess some of the same candidates independently, then resolve disagreements and retain the reasons. Where practical, hide the originating platform and AI rating during review.

The National Institute of Standards and Technology (NIST) reports in its relevance-assessment research that large language model judgments can be biased and behave differently across tasks and data. Use human-assigned labels as the reference for the evaluation, with model suggestions kept identifiable.

Fictional screening example: A September 2026 semiconductor-package specification describes conductive pillars between 2 dies but omits the claimed bonding sequence. The reviewer records direct support for the pillar arrangement and an unresolved sequence question, then applies the agreed follow-up threshold.

Keep the reason for each unverifiable label in the review record.

3. Review a sample and look for known misses

Choose a manageable sample from the full returned list. Include candidates from different positions in the ranking, product families, and source types, rather than reviewing only the strongest-looking matches. Record which candidates you reviewed and how you chose them.

For each candidate, review the underlying source against the agreed screening criteria. Record whether the product merits follow-up and whether the cited passage supports its assigned limitation and product version. Keep unsupported inferences and unresolved limitations visible alongside the candidate label.

Record precision on the reviewed sample as the number of relevant candidates divided by the total classified as relevant or irrelevant. Show those counts alongside the percentage. List unverifiable candidates separately and state how many reviewed items were excluded from the calculation. Label the percentage with its reviewed sample and run date. This applies the Text Retrieval Conference's precision definition to the reviewed candidates.

Then deliberately look for a few known products that the screen should have found. Select them before the run using existing product knowledge and documentation that meets the agreed criteria. Record the product versions, the expected supporting sources, and whether each appeared in the results.

Log each known-product miss separately, with its source-access, language, version, or evidence-discovery gap.

True recall, the share of all relevant products found, requires a defined product universe and independent judgments about which products in it are relevant.

4. Decide where to deploy the screen

Combine the scope record, reviewed-candidate decisions, precision, known-product results, and evidence-mapping errors into 1 evaluation record. Use recurring errors to decide whether to adopt screening for the defined task, narrow its use, or pursue deeper coverage. For example, useful leads from current manuals can support present-product investigations while missing historical versions point to a separate collection need.

Keep screening decisions separate from U.S. direct-infringement analysis under the 2024 edition of 35 U.S.C. § 271(a), including the relevant act, whether it occurred without authority, the U.S. territorial connection, and the patent term. The infringement evidence guide covers the downstream investigation and claim analysis.

Retain a new version of the evaluation record when claims, source access, products, or screening settings change materially.

Where Patlytics fits

Patlytics gives you candidate product mappings, cited disclosures, and editable charts to evaluate which leads merit follow-up investigation.

• Define the screen: The Infringement module generates Detection Reports from selected claims, target companies, and jurisdiction. It searches public sources and indexed product documentation and returns product mappings, cited supporting disclosures, and read-strength ratings.
• Apply the working interpretation: Claim Construction proposes constructions with pinpoint citations to the specification. The constructions are editable, and the chart can be rerun with the revised interpretation.
• Deepen priority-claim coverage: Exhaustive mode provides deeper coverage on selected priority claims.
• Retain the evidence work product: Claim Charting produces editable limitation-by-limitation charts from surfaced or supplied evidence, with Word and Excel export.

Attorneys retain review and decision authority over the analysis and resulting work product.

Get candidate product mappings and cited, limitation-by-limitation evidence. Book a demo.