4
\$\begingroup\$

I need to run over all the IP addresses on the CIDR 10.96.0.0/12 network.
There are 1,048,574 IP addresses.

I created a multi threaded program in Go to do it, but even when I use 100 threads it takes lots of time (didn't check exactly but it is more than 10 minutes).

Is there are way I can do it more efficient?
By the way, on Windows it takes couple of minutes but I did also test on container in Linux and it took 40 seconds to scan 500K IP addresses. But let me know if you have faster way to check it.

My program:

package main

import (
    "fmt"
    "log"
    "net"
)

func getIPAddresses(cidr string) ([]string, error) {
    ip, ipnet, err := net.ParseCIDR(cidr)
    if err != nil {
        return nil, err
    }

    var ips []string
    for ip := ip.Mask(ipnet.Mask); ipnet.Contains(ip); inc(ip) {
        ips = append(ips, ip.String())
    }
    // remove network address and broadcast address
    lenIPs := len(ips)
    var ipAddresses []string

    switch {
    case lenIPs < 2:
        ipAddresses = ips

    default:
        // Shouldn't be panic here because we are checking the lenIPs before
        ipAddresses = ips[1 : len(ips)-1]
    }

    return ipAddresses, nil
}

//  http://play.golang.org/p/m8TNTtygK0
func inc(ip net.IP) {
    for j := len(ip) - 1; j >= 0; j-- {
        ip[j]++
        if ip[j] > 0 {
            break
        }
    }
}

func main() {
    ipAddresses, err := getIPAddresses("10.96.0.0/12")
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("[*] Scanning %d IP addresses\n", len(ipAddresses))

    //var hostsMap map[string][]string
    //l := sync.Mutex{}
    threads := 100
    sem := make(chan bool, threads)

    count := 0
    for _, ip := range(ipAddresses) {

        sem <- true

        go func(ip string) {
            count += 1
            if count % 1000 == 0 {
                fmt.Printf("Scan: %d addresses\n", count)
            }
            _, err := net.LookupAddr(ip)
            if err != nil {
                //log.Fatal(err)
                //fmt.Println("Error")
            }

            <- sem
        }(ip)
    }

    for i := 0; i < cap(sem); i++ {
        sem <- true
    }

    fmt.Println("DONE")
}
\$\endgroup\$
6
  • \$\begingroup\$ Why did you decide to multithread it ? Were you sure that the CPU is the bottleneck, not the IO? \$\endgroup\$ – aki Aug 2 '20 at 8:36
  • 2
    \$\begingroup\$ Because the function net.LookupAddr() need to send DNS request which means it is a network I/O and it takes time (not too much). When I comment the net.LookupAddr() function it finish to pass all the IPs in 2 seconds. \$\endgroup\$ – E235 Aug 2 '20 at 8:39
  • \$\begingroup\$ Every run of net.LookupAddr() takes me 1 millisecond on average. \$\endgroup\$ – E235 Aug 2 '20 at 8:45
  • 1
    \$\begingroup\$ Is this a network that you own/control ? I would guess that the answer is yes. Then you should be able to get a copy of the zone file and use it as a lookup source. The information that you want has to be available in some form, thus the mass lookup looks wasteful to me. PS: not familiar with the GO language but implementation on Windows differs than Linux - see the Name Resolution section here \$\endgroup\$ – Anonymous Aug 2 '20 at 15:44
  • \$\begingroup\$ Every run of net.LookupAddr() takes me 1 millisecond on average How do you assert that ? \$\endgroup\$ – mh-cbon Aug 4 '20 at 11:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.