Reusing strings read from I/O

Question

I am working on an application that reads lots of data from the network and puts it in a grid. I noticed that I could save some memory by reusing existing strings instead of always using the new strings that came off the wire. So here is the class that I wrote to accomplish that. It is simple and it works. But I was wondering if this had a name and, of course, if the code could be improved. One thing that I don't like is that I'm storing two references to each string.

public class StringCollection
{
  private Dictionary<string, string> lookup_ = new Dictionary<string, string>();

  public string Reuse(string s)
  {
    if (s == null)
    {
      return null;
    }
    string existing;
    if (lookup_.TryGetValue(s, out existing))
    {
      return existing;
    }
    else
    {
      lookup_.Add(s, s);
      return s;
    }
  }

  public void Clear()
  {
    lookup_.Clear();
  }

  public int Count { get { return lookup_.Count; } }

}

Answer 1

You might want to look into string interning:

The common language runtime conserves string storage by maintaining a table, called the intern pool, that contains a single reference to each unique literal string declared or created programmatically in your program. Consequently, an instance of a literal string with a particular value only exists once in the system.

While it has some memory-related side-effects, you could avoid having to build up a dictionary like this and throw away allocated strings to keep it updated.

Answer 2

You can use a HashSet<string> instead of a dictionary to half the number of references that you keep, but getting the reference out of the hash set is a bit more difficult.

I used this method to solve it:

public class LocalIntern {

  private HashSet<string> _lookup = new HashSet<string>();

  public string Reuse(string s) {
    if (s != null) {
      if (_lookup.Contains(s)) {
        s = _lookup.Where(i => i == s).First();
      } else {
        _lookup.Add(s);
      }
    }
    return s;
  }

  public void Clear() {
    _lookup.Clear();
  }

  public int Count { get { return _lookup.Count; } }

}

Basic function verification:

LocalIntern intern = new LocalIntern();

// store a string    
string data = "asdf";
intern.Reuse(data);

// create another string instance with the same value    
string data2 = String.Concat("as", "df");
// verify that they are in fact separate instances
Debug.Assert(!Object.ReferenceEquals(data, data2));

// look for the string
string d = intern.Reuse(data2);
// verify that the string was replaced
Debug.Assert(Object.ReferenceEquals(data, d));

Answer 3

The ideal thing here would be a WeakDictionary<string,string> (or better yet, a variation of a WeakHashSet<T> which allowed one to retrieve a reference to the stored item which matched a supplied key). Such a data structure would ensure that strings were shared whenever possible (a behavior which would not only reduce memory requirements, but also expedite comparisons among them, since comparing two distinct million-character strings that happen to be equal will take much longer than comparing two references to the same string). If one uses a Dictionary<string,string>, which is the best of the pre-existing classes for the purpose, it may be tricky to ensure that strings get kept as long as they may be useful, without having the dictionary keep them around even after they've become useless (if a string value has been used once before but all references outside the dictionary have been abandoned, having the string remain in the dictionary past the next GC cycle will generally serve no purpose; even if the same sequence of characters gets read again, it would generally be faster to have the old string evaporate and store a new string in the dictionary, than to have to compare all the characters in the new string against the ones in the old string and abandon the new one).