# Reading lines from a file and processing them concurrently

I'd like to read lines from a file and process them concurrently. I came up with the following code to do this:

var wg sync.WaitGroup

func main(){
file := "data/input.txt"
for scanner.Scan() {
go processLine(scanner.Text())
}
wg.Wait()
}

func processLine(line string) {
time.Sleep(time.Duration(rand.Intn(5)) * time.Second)
fmt.Println("line:", line)
wg.Done()
}


I added the random sleep time in there to simulate potential differences in processing times.

Is there any potential drawbacks that I should be aware of with this method of concurrent processing? Are there any better ways that I should consider concurrently processing lines in a file?

• Did you try it? Does it work as intended? – Mast Aug 29 '18 at 6:28
• This is artificial code. This is not "actual code from a project rather than pseudo-code or hypothetical code." codereview.stackexchange.com/help/dont-ask – peterSO Aug 29 '18 at 12:41
• @PeterSO - Why do you say that? The thing I am asking about improving is exactly as it will appear in the program I'm working on. I left the processLine function empty because it's not relevant to the question i'm asking – Abe Miessler Aug 29 '18 at 14:05
• @Mast - it works exactly as intended. There is no question about that - only whether I am not following best practices some way or if there is some problem that I might be missing in non-trivial examples. – Abe Miessler Aug 29 '18 at 14:06
• @peterSO - to address the link you posted, it says: Questions should not contain purely generic, hypothetical code, such as This is clearly not "...purly generic, hupothetical code". The concurrency code (which is what I'm asking about) is what will be used in my production application – Abe Miessler Aug 29 '18 at 14:13

2. You're also using a global variable to share the WaitGroup, which I think you should avoid. Either pass it in as a function parameter or use an anonymous function that calls processLine() and then wg.Done() (instead of calling wg.Done() in processLine())