1
\$\begingroup\$

I was tasked with conducting a large number of text searches in the code files of multiple projects. Initially, I just used the Find in Files feature of Visual Studio in a couple of the running instances, but then I realized I needed to search other solutions as well. So I quickly created an F# Script (below) to conduct the searches across all code files async.

For all matching results, the script prints:

C:\path-to-file
linenumber: line of code

How can this be improved? The script runs extremely fast and did its job well. Now, out of curiosity, I would like to know what improvements could be made to optimize performance?

open System
open System.IO

let directory = @"C:\path-to-top-level-directory" 

// this just keeps async operations from writing to the console concurrently (garbled result)
let agent = MailboxProcessor.Start(fun inbox -> 
    // the message processing function
    let rec messageLoop () = async {
        let! msg = inbox.Receive()
        printfn "%s" msg
        return! messageLoop()
    }
    // start the loop
    messageLoop ()
)

let checkIfContains value file = async {
    use reader = new StreamReader(File.OpenRead(file))
    let lineNum = ref -1
    let postResult (i, s) = sprintf "%s\r\n%i: %s\r\n" file i s |> agent.Post

    seq { while not reader.EndOfStream do 
            incr lineNum 
            yield (!lineNum, reader.ReadLine()) } 
        |> Seq.filter (fun (_, s) -> s.Contains(value))
        |> Seq.iter postResult
}

let files = 
    Directory.EnumerateFiles(directory, "*.vb", SearchOption.AllDirectories)
        |> Seq.append (Directory.EnumerateFiles(directory, "*.cs", SearchOption.AllDirectories))

while true do
    agent.Post "Search string: "
    let value = Console.ReadLine()
    agent.Post "Searching...\r\n"

    files
        |> Seq.map (checkIfContains value)
        |> Async.Parallel
        |> Async.RunSynchronously
        |> ignore

    agent.Post "\r\nDone.\r\n\r\n"
\$\endgroup\$

migrated from stackoverflow.com Sep 22 '14 at 11:01

This question came from our site for professional and enthusiast programmers.

  • \$\begingroup\$ seems fine - I think maybe the mailboxprocessor is not really needed (a simple lock on the Console.out should do) - but hey: that's nitpicking \$\endgroup\$ – Carsten Sep 12 '14 at 19:14
  • \$\begingroup\$ You appear to have reinvented grep. \$\endgroup\$ – John Palmer Sep 13 '14 at 23:36
1
\$\begingroup\$

For a base line, I modified your script so that it is not dominated by printing to the console:

open System
open System.IO

let directory = @"C:\Dev" 

// this just keeps async operations from writing to the console concurrently (garbled result)
let agent = MailboxProcessor.Start(fun inbox -> 
    // the message processing function
    let count = ref 0
    let rec messageLoop () = async {
        let! msg = inbox.Receive()
        count :=  !count + 1
        return! messageLoop()
    }
    // start the loop
    messageLoop ()
)

let checkIfContains value file = async {
    use reader = new StreamReader(File.OpenRead(file))
    let lineNum = ref -1
    let postResult (i, s) = sprintf "%s\r\n%i: %s\r\n" file i s |> agent.Post

    seq { while not reader.EndOfStream do 
            incr lineNum 
            yield (!lineNum, reader.ReadLine()) } 
        |> Seq.filter (fun (_, s) -> s.Contains(value))
        |> Seq.iter postResult
}

let files = 
   Directory.EnumerateFiles(directory, "*.cs", SearchOption.AllDirectories)

let value = "papa"

files
    |> Seq.map (checkIfContains value)
    |> Async.Parallel
    |> Async.RunSynchronously
    |> ignore


agent.Post "\r\nDone.\r\n\r\n"

This takes ~10 seconds on my machine. Like Carsten said, there's no need for a MailboxProcessor. Removing it takes me down to ~7.5 seconds. Next, there's no need for Async. There is no ! in that async comp expression. We could use Seq.AsParallel instead (in System.Linq namespace). Down to ~6.3 secs.

Now this task is dominated by IO, and AsParallel shouldn't help here. In fact, removing it yields the same result, ~6.3 secs.

Now, that seq inside of checkIfContains looks wasteful. Getting rid of that yields ~6.1 secs, so no big win.

I wouldn't consider this idiomatic code. I just took out some unused stuff, and wound up with this:

open System
open System.IO
open System.Linq
let directory = @"C:\Dev"
let count = ref 0 
let print s = lock count (fun () -> count := !count + 1)

let checkIfContains value file = 
    use reader = new StreamReader(File.OpenRead(file))
    let lineNum = ref -1
    let postResult i s = sprintf "%s\r\n%i: %s\r\n" file i s |> print
    while not reader.EndOfStream do 
      incr lineNum 
      let line = reader.ReadLine()
      if line.Contains(value) then postResult !lineNum line

let files = Directory.EnumerateFiles(directory, "*.cs", SearchOption.AllDirectories)
let value = "papa"
let f = files
              |> Seq.map (checkIfContains value)
              |> Seq.iter id
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.