1
\$\begingroup\$

The c# newbie is back :) I have another problem with my threads. Here is what i am trying to achieve:

I am starting 5 threads which are performing the same task but on different URLs. So i am keeping a "masterLogFile.txt" to keep track of what URLs have already been visited. Each thread compares its own "thread1LogFile.txt" to the "masterLogFile.txt" before deciding whether to execute the task.

My question is, is there any more efficient way to handle this? Currently each thread runs this piece of code before deciding if the URL is ok or not:

using (FileStream fs = File.Open("masterLogFile.txt", FileMode.Open, FileAccess.Read, FileShare.Read)) 
{
    byte[] bff = new byte[1024];
    UTF8Encoding temp = new UTF8Encoding(true);

    while (fs.Read(bff,0,bff.Length) > 0) 
    {
        if (temp.GetString(bff).Contains(variableWithUrlFrom_thread1LogFile.txt))
        {
            found = true;
        }

    }
    fs.Close();
}
\$\endgroup\$
  • \$\begingroup\$ Ah no, sorry. I just wanted to ask if my logic for comparing data in files from different threads can be improved \$\endgroup\$ – user1213488 May 10 '12 at 13:25
  • \$\begingroup\$ Why do to want to use files to track the work? Aren't all the threads in the same process? \$\endgroup\$ – M Afifi May 10 '12 at 13:27
  • \$\begingroup\$ Are you reading in the master log file each time the thread is called - and does this change? \$\endgroup\$ – Neil May 10 '12 at 13:28
5
\$\begingroup\$

I'd use a dictionary (or a HashSet if you don't need the ID information) for coordination.

class UrlLog {
    private Dictionary<string, int> visited = new Dictionary<string, int>();
    public bool HasUrlBeenVisited(string Url) {
        lock(visited) return visited.ContainsKey(Url);
    }
    public void SetUrlVisited(string Url, int threadId) { /* id not strictly necessary */
        lock(visited) visited[Url] = threadId;
    }
}

Then pass one instance of this UrlLog to your threads, and they can use that.

Also, I'm not sure you even need the locks for the checks, but it's better to be safe than sorry. :)

| improve this answer | |
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy