G06V 10/776

Definition

(Für diese Definition ist die deutsche Übersetzung noch nicht abgeschlossen)

Techniques for validation and performance evaluation of algorithms for image or video recognition or understanding.

Notes – technical background

These notes provide more information about the technical subject matter that is classified in this place:

Validation and performance evaluation of algorithms for image or video recognition or understanding normally involve:

• a training data set which is a set of examples used to fit the parameters of a pattern recognition or machine learning model;
• a validation data set which provides an unbiased evaluation of the model fit on the training data set (while optionally tuning the model's parameters); and
• a test data set used to provide an unbiased evaluation of a final model. If the data in the test data set has never been used in training (for example in cross-validation), the test data set is also called a holdout data set.

Common classification metrics to evaluate the models are true positive rate (TPR) or sensitivity, false positives rate (FPR) or fall-out, true negatives rate (TNR) or specificity, false negative rate (FNR) or miss rate, receiver operating characteristic (ROC) curves (TP rate divided by the FP rate), z-score, accuracy, precision (or positive predictive value), recall, negative predictive value, intersection over union (IoU), the Jaccard index (also referred to as Tanimoto index), etc. Other metrics are also possible, for instance regression metrics, explained variance, validation curves, detection error trade-off etc. In case of decision-tree learning, the compactness of a cluster, the purity of a cluster in terms of class labels, the minimum distance of samples from the class boundary, a calculated likelihood score, etc.

In order to get more stable results and use all valuable data for training, a data set can be repeatedly split into several training and a validation datasets. This strategy is known as cross-validation.

The performance can be measured automatically, e.g. by a stochastic process such as when using bootstrapping, or by a human operator in the case of relevance feedback.

Examples

Example of an iterative “loss function” calculation for 4 different recognition models trained with different subsets of images, which is indicative of the performance of the classification of each model.

Querverweise

Informative Querverweise

Glossar