Increment struct value using reflection

Question

The main idea is, I have a struct and I will get a struct field as a string from another function, which I shall increment its value by 1.
Below is the code I used and I think it is memory consuming.
How can I improve it? Is there a better way than the reflection from the start?
Thanks

package main

import (
    "fmt"
    "reflect"
)

func main() {

    type chrinfo struct {
    
    Size0_50    int
    Size50_100  int
    Size100_1000    int
    Size1000_10000  int
    Size10000   int
    }
    
    mychr := chrinfo{}
    n := "Size1000_10000"
    
    y := reflect.ValueOf(&mychr).Elem().FieldByName(n)
    reflect.ValueOf(&mychr).Elem().FieldByName(n).SetInt(int64(y.Interface().(int))+1)
    



    fmt.Println(mychr, y)
}

Answer 1

OK, after reading you're using a struct because maps use more memory, and that you posted this question here because you're new to go, let's dive in to this a bit more.

Do maps use more memory

The short answer is yes. However, things are a bit more complex than that. In essence, a map is a hash table, so of course, a map of type map[string]int will internally store all the strings, the buckets, hashes, data tables, etc... Whereas a struct is just a block of memory where fields point to an offset within that memory, it's pretty much the most efficient way to group values.

That said, while a map with 5 keys does use more memory than a struct with 5 fields of type int, the difference on modern systems is quite negligible. There are a number of other factors to consider when it comes to choosing the type.

Runtime memory management

Once again, this may seem like a point where maps look like a terrible option, but it's just some background so you understand how maps, WRT memory management, work. Consider this:

foo := map[string]int{
    "one":   1,
    "two":   2,
    "three": 3,
}

What happens here is that the runtime will allocate memory to hold the three strings, and the three values. The strings are passed through a hashing function, all memory required to store this map, the hashes, and its three values is allocated and assigned. Easy. Now compare this to:

bar := map[string]int{}
// some code
bar["one"] = 1
// more core
bar["two"] = 2
// etc...
bar["three"] = 3

In this scenario, the runtime creates an empty map and assigns it to bar. Some memory is allocated for the internal structures the map needs, but each time we assign a new key, the memory required to store the data for this map grows, and the runtime potentially has to reallocate memory, move stuff around, etc... this definitely is less efficient. We can, however, initialise a map with a certain size. I often do this if I know the map I'm populating will eventually hold at least 10, and at most 15 values. I'll just use make and specify the max size:

foobar := make(map[string]int, 3) // guarantees no realloc while adding the first 3 values

Another area where maps put some added strain on the runtime is when it comes to garbage collection. Remember: we need to allocate memory for the value, hash/buckets (map internals), and key, so doing something like:

delete(foo, "two")

affects 3 objects the runtime is having to manage. That's the reality of maps. The go implementation is fairly optimised, but if you run a heavy application that were to rely solely on maps, your pprof data after a while might show that the runtime is spending a lot of time on its GC cycles as a result.

The latter is not really something you should be taken into consideration when comparing maps to structs, however. Structs don't support deleting fields to begin with. I just mentioned this for completeness sake. While it looks like a downside, you could equally argue that having the ability to delete keys from a map is a feature you might want to have. Indeed, sometimes you need to be able to do that.

Restrictions of a Struct

So far, we've established structs are more memory efficient, generally easier on the GC cycles (discounting nested structs, interfaces, embedding, etc...). They do have their limitations, however. The thing discussed earlier about memory management (adding more keys to a map causing reallocation) aren't an issue here. The memory is allocated all at once, and is nicely padded. On the flip side: the memory is allocated all at once. Even if I only were to need one field, a struct is an all or nothing type.

Dynamic access

Maps can use strings, ints, and really any comparable type. A struct has fields. This means the data contained stored in a map is a lot easier to access dynamically:

// given foo from earlier
func get(k string) (int, error) {
    v, ok := foo[k]
    if !ok {
        return 0, fmt.Errorf("key '%s' not found", k)
    }
    return v, nil
}

Whereas using a string to fetch the field of a struct requires a lot more code. The easiest way would be a receiver function on the struct type (method)

// assuming the chrinfo type from your code
func (c chrinfo) get(k string) (int, error) {
    switch k {
    case "Size0_50", "size0_50": // accounting for capitalisation
        return c.Size0_50, nil
    case "Size50_100", "size50_100":
        return c.Size50_100, nil
    case "Size100_1000", "size100_1000":
        return c.Size100_1000, nil
    case "Size1000_10000", "size1000_10000":
        return c.Size1000_10000, nil
    case "Size10000", "size10000":
        return c.Size10000, nil
    }
    return 0, fmt.Errorf("field '%s' doesn't exist", k)
}

This function will need to be updated for each field you add, and although capitalisation is something you may want to take into account, it's cumbersome. Typo's can lead to vague bug reports that cost hours to replicate, only to conclude there was no bug in the code you were looking at, but the person filing the issue didn't bother to check whether or not their input was correct...

Reflection

So obviously, one may conclude that using reflection at least takes away the burden of having to maintain this type of getter function. However, let's circle back to your original reason for not using a map: memory consumption.

Compared to the runtime cost reflection incurs, the memory overhead of a map is laughable. It's akin to hopping on a plane to the arctic to cool a can of coke, instead of using an ice-cube, because using a freezer to freeze ice is wasting electricity, all the while burning literally tons of kerosine to fly your beverage around the globe instead.

Reflection is crazy expensive. It uses a hell of a lot of CPU cycles, and if you look into the source code every time you access something through reflection, you'll allocate the reflect types, like the reflect.Value type, which contains a reflect.rtype. These are just 2 structs you're allocating just to get access to a single field:

type Value struct {
    // typ holds the type of the value represented by a Value.
    typ *rtype

    // Pointer-valued data or, if flagIndir is set, pointer to data.
    // Valid when either flagIndir is set or typ.pointers() is true.
    ptr unsafe.Pointer

    // flag holds metadata about the value.
    // The lowest bits are flag bits:
    //  - flagStickyRO: obtained via unexported not embedded field, so read-only
    //  - flagEmbedRO: obtained via unexported embedded field, so read-only
    //  - flagIndir: val holds a pointer to the data
    //  - flagAddr: v.CanAddr is true (implies flagIndir)
    //  - flagMethod: v is a method value.
    // The next five bits give the Kind of the value.
    // This repeats typ.Kind() except for method values.
    // The remaining 23+ bits give a method number for method values.
    // If flag.kind() != Func, code can assume that flagMethod is unset.
    // If ifaceIndir(typ), code can assume that flagIndir is set.
    flag

    // A method value represents a curried method invocation
    // like r.Read for some receiver r. The typ+val+flag bits describe
    // the receiver r, but the flag's Kind bits say Func (methods are
    // functions), and the top bits of the flag give the method number
    // in r's type's method table.
}

type rtype struct {
    size       uintptr
    ptrdata    uintptr // number of bytes in the type that can contain pointers
    hash       uint32  // hash of type; avoids computation in hash tables
    tflag      tflag   // extra type information flags
    align      uint8   // alignment of variable with this type
    fieldAlign uint8   // alignment of struct field with this type
    kind       uint8   // enumeration for C
    // function for comparing objects of this type
    // (ptr to object A, ptr to object B) -> ==?
    equal     func(unsafe.Pointer, unsafe.Pointer) bool
    gcdata    *byte   // garbage collection data
    str       nameOff // string form
    ptrToThis typeOff // type for pointer to this type, may be zero
}

I'm not going to walk through the entirety of the reflect package, although you may want to take some time to do that yourself, just to see what using reflection actually implies. For now, let me assure you this:

Using reflection uses tons of CPU cycles and memory

---

So the question you had was: is there a better way? I'd say the answer to that is a resounding yes. A map will do just fine, in fact. The slightly higher memory consumption is nothing compared to the overhead you're creating using reflection.

I touched on tags in my comment, which is something you can look into, but I wouldn't recommend it for this particular use-case (seeing as tags require reflection, too). You could use the combination of a map/struct as a field lookup table, like the one I mentioned. Assuming these calls don't happen concurrently, you can create a map statically (as in: as a field of a type, or a global variable if there is no other option), and do something like this:

var (
    mychr  crinfo          // your struct
    lookup map[string]*int // lookup map
)

func init() {
    lookup = map[string]*int{
        "Size0_50":       &mych.Size0_50,
        "Size50_100":     &mych.Size50_100,
        "Size100_1000":   &mych.Size100_1000,
        "Size1000_10000": &mych.Size1000_10000,
        "Size10000":      &mych.Size10000,
    }
}

Then, to increment the corresponding field, it's quite simple:

func inc(k string) {
    if ptr, ok := lookup[k]; ok {
        *ptr += 1
    }
}

The map contains pointers to each corresponding field, if the field exists, we increment its value. Job done.