Threads and logfiles

Question

The c# newbie is back :) I have another problem with my threads. Here is what i am trying to achieve:

I am starting 5 threads which are performing the same task but on different URLs. So i am keeping a "masterLogFile.txt" to keep track of what URLs have already been visited. Each thread compares its own "thread1LogFile.txt" to the "masterLogFile.txt" before deciding whether to execute the task.

My question is, is there any more efficient way to handle this? Currently each thread runs this piece of code before deciding if the URL is ok or not:

using (FileStream fs = File.Open("masterLogFile.txt", FileMode.Open, FileAccess.Read, FileShare.Read)) 
{
    byte[] bff = new byte[1024];
    UTF8Encoding temp = new UTF8Encoding(true);

    while (fs.Read(bff,0,bff.Length) > 0) 
    {
        if (temp.GetString(bff).Contains(variableWithUrlFrom_thread1LogFile.txt))
        {
            found = true;
        }

    }
    fs.Close();
}

Answer 1

I'd use a dictionary (or a HashSet if you don't need the ID information) for coordination.

class UrlLog {
    private Dictionary<string, int> visited = new Dictionary<string, int>();
    public bool HasUrlBeenVisited(string Url) {
        lock(visited) return visited.ContainsKey(Url);
    }
    public void SetUrlVisited(string Url, int threadId) { /* id not strictly necessary */
        lock(visited) visited[Url] = threadId;
    }
}

Then pass one instance of this UrlLog to your threads, and they can use that.

Also, I'm not sure you even need the locks for the checks, but it's better to be safe than sorry. :)