What is Benchmarking?
By definition, benchmarking is to evaluate or check (something) by comparison with a standard. What is meant by this? In terms of document imaging solutions, we compare sample images against the same images that have already been processed to see how well the solution is working. There are three areas that we look at to determine the accuracy of a project: extraction, classification, separation.
Benchmarking extraction is the process of checking the accuracy of the data extracted from the test images compared to what should be extracted (the standard). In this process, the system would pull the data from the image automatically, and then check that data against what it should be. At this point, you are able to see how well your solution is working with respect to the extraction process.
Benchmarking classification is the process of determining what type of document the test image is compared to the same image after it was already classified. In this process, the system classifies each document in the batch and compares it to what it should be classified as. From this point you can determine if the documents being classified are what they are supposed to be.
Benchmarking separation is the process of testing the system against an entire batch of pages to determine if the system separated these pages into the correct number of documents and the correct number of pages. This gives you the view of how well the system detects the first, middle, and last pages of a document to determine where the document breaks are.
Benefit of Benchmarking
The first benefit of performing benchmarking against a system is to determine quality. For example, how well the system is processing documents, how accurately it knows the documents, and how accurately it can pull the correct data from the document.
As you test and fine tune your solution, benchmarking allows you to see how well your changes are doing. It answers the question: Did that change increase or decrease the accuracy of your solution?
To learn more about Benchmarking, read our related articles: Processing & Analyzing Benchmarking Results, Kofax Benchmarking Tools.