5
\$\begingroup\$

I written this code to search 'blob' items (text files) on the basis of their content. For example, if I search for "Good", then the names of files that contain "Good" or "good" should appear in the search result.

My code is working but I want to optimize it:

class BlobSearch
{
    public static int num = 1;
    static void Main(string[] args)
    {
        string accountName = "accountName";
        string accessKey = "accesskey";
        string azureConString = "DefaultEndpointsProtocol=https;AccountName=" + accountName + ";AccountKey=" + accessKey;
        string blob = "MyBlobContainer";
        string searchText = string.Empty;

        Console.WriteLine("Type and enter to search : ");
        searchText = Console.ReadLine();

        CloudStorageAccount account = CloudStorageAccount.Parse(azureConString);
        CloudBlobClient blobClient = account.CreateCloudBlobClient();
        CloudBlobContainer blobContainer = blobClient.GetContainerReference(blob);

        blobContainer.FetchAttributes();
        var blobItemList = blobContainer.ListBlobs();
        GetBlobList(searchText, blobContainer, blobItemList);

        Console.ReadLine();
    }

    private static async void GetBlobList(string searchText, CloudBlobContainer blobContainer, IEnumerable<IListBlobItem> blobItemList)
    {
        foreach (var item in blobItemList)
        {
            string line = string.Empty;
            CloudBlockBlob blockBlob = blobContainer.GetBlockBlobReference(item.Uri.ToString());
            if (blockBlob.Name.Contains(".txt"))
            {
                await Search(searchText, blockBlob);
            }
        }
    }


    private async static Task Search(string searchText, CloudBlockBlob blockBlob)
    {
        string text = await blockBlob.DownloadTextAsync();
        if (text.ToLower().IndexOf(searchText.ToLower()) != -1)
        {
            Console.WriteLine("Result : " + num + " => " + blockBlob.Name.Substring(blockBlob.Name.LastIndexOf('/') + 1));
            num++;
        }
    }
}

I think blobContainer.ListBlobs(); is blocking code because search will not work until all the blob item is loaded. Is there anyway to optimize that, or anything else in my code?

\$\endgroup\$
6
  • \$\begingroup\$ Optimize against what? What specific problem are you having? Please edit your question to explain what issues you are experiencing and why you think the code should be improved. \$\endgroup\$
    – user26710
    Commented Jun 10, 2014 at 11:02
  • \$\begingroup\$ A lot of the time it isn't worth optimizing heavy IO based activity, because 99% of your time is disk reads. Profile it, and see what the results tell you. \$\endgroup\$
    – Phoshi
    Commented Jun 10, 2014 at 11:10
  • \$\begingroup\$ @GlenH7 I written in my question blobContainer.ListBlobs(); is much time to download all blob item & it will increase as blob items increases. \$\endgroup\$
    – Vicky
    Commented Jun 10, 2014 at 12:15
  • \$\begingroup\$ @Phosi How i can do this. Please give some description. \$\endgroup\$
    – Vicky
    Commented Jun 10, 2014 at 12:21
  • \$\begingroup\$ Please grab a book on Information Retrieval and start reading it. A short description is that you will index your data as each piece of data comes in. The index is about things that you would later want to search on, such as words, terms and specific numbers. The more specific you know what you will search on, the more efficient indexing will be. Later, your query is made against the index, instead of having to go over the entire data set again. \$\endgroup\$
    – rwong
    Commented Jun 11, 2014 at 5:18

1 Answer 1

2
\$\begingroup\$

If I understand correctly:

  1. blobContainer.ListBlobs() is what is taking a long time
  2. ListBlobs is not code that you control or can improve
  3. There isn't a way to perform the filtering you want without calling ListBlobs

It follows that there isn't any way to appreciably improve your program without fundamentally altering your approach. As the comments suggest, indexing is one common approach, either a simple solution you implement yourself or an integration with an existing software package (depending on your needs and requirements).

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.