Critique the design and quality of this TrainingData class used in my image recognition software

Question

I'm working on a project for an image recognition software. The software takes an image and returns a set of labels (a label is just a string of text) which describe the contents of that image. Each returned label is also associated with a confidence value which quantifies how certain the algorithm of its label assignment.

A key component of the project is a library of training data. Training data is built by taking labeled images (images with descriptive strings of text already associated with them), passing the images through a bunch of tests called heuristics, and saving the set of values returned from those heuristics along side the associated label. The heuristic values generated from unlabeled images are compared to the library of heuristic return values and labels stored in the training data library to determine what labels should be associated with the unlabeled image.

Please critique the design and quality of the following code:

/// <summary>Training data is a set of heuristics return values associated with their corresponding input label.
/// Traning data is used to take an unlabeled set of heuristic return values and to compare those return values
/// to labeled sets of heuristic return value and use that comparison to determine the most appropriate label to 
/// associate.</summary>
public class TrainingData {
    /// <summary>Key is the label associated with the value which is a set of heuristic return values</summary>
    private Dictionary<string, List<HeuristicReturnValues>> library 
                                                        = new Dictionary<string, List<HeuristicReturnValues>>();
    public void AddHeuristics(HeuristicReturnValues returnValuesToAdd) {
        if (returnValuesToAdd.Label == null) {
            throw new NullReferenceException(
                "Trying to add an unlabeled set of heuristic return values to the traning data library");
        }
        if(library.ContainsKey(returnValuesToAdd.Label)){
            //Add the heuristic return values to the list associated with the corresponding label in the library
            List<HeuristicReturnValues> listOfHeuristics = library[returnValuesToAdd.Label];
            listOfHeuristics.Add(returnValuesToAdd);
            library[returnValuesToAdd.Label] = listOfHeuristics;
        } else {
            //Create a new label entry in the library
            library.Add(returnValuesToAdd.Label, new List<HeuristicReturnValues>(){returnValuesToAdd});
        }
    }
    /// <summary>Take an unlabeled HeursiticReturnVaules object and compare it to each key value pair in the 
    /// library and return the best match as a LookupResult</summary>
    public List<LookupResult> PerformLookUp(HeuristicReturnValues unlabeledReturnValues) {
        if (unlabeledReturnValues.Label != null)
            throw new Exception("This guy is supposed to be unlabeled!");
        List<LookupResult> comparisonValues = new List<LookupResult>();
        foreach (var labeledReturnValues in library) {
            comparisonValues.Add(labeledReturnValues.Value.Compare(unlabeledReturnValues));
        }
        return comparisonValues.OrderBy(i => i.ConfidenceValue).ToList();
    }
}
/// <summary>Contains the label to be associated with the unlabeled HeuristicReturnValues and a confidence value
/// which reflects the algorithm's confidence in making that assignment.</summary>
public class LookupResult {
    public LookupResult(string lbl, double confidence) {
        this.Label = lbl;
        this.ConfidenceValue = confidence;
    }
    public string Label { get; set; }
    public double ConfidenceValue { get; set; }
}

public static class LibraryExtensionMethods {
    public static LookupResult Compare(this List<HeuristicReturnValues> labeledSet, 
                                                                    HeuristicReturnValues unlabledHeuristic) {
        //Implement a comparison between labeled and unlabeled heuristic return values
        throw new NotImplementedException();
    }
}

Answer 1

for starters, if you are going to be linqy, you can be totally linqy

public List<LookupResult> PerformLookUp(HeuristicReturnValues unlabeledReturnValues)
{
if (unlabeledReturnValues.Label != null) 
   throw new Exception("This guy is supposed to be unlabeled!");

return library.Select(labeled =>   labeled.Value.Compare(unlabeledReturnValues))
              .OrderBy(i => i.ConfidenceValue).ToList();
            }
        }

use 'var' a bit more to make things less wordy.

In fact, if you have resharper, it will help you clean up a lot of this code

rename 'PerformLookup' to be what your comment says it is.. FindBestMatch or something similar