Traversing a JSON-like data structure in Go

Question

In my code I need a function to traverse an arbitrary JSON-like data structure with a path definition. E.g., given the structure

{"root": {"inner": [{"name": "obj1"}, {"name": "obj2"}]}}

and the path ("root", "inner", "0", "name"} I would obtain "obj1". I just started learning Go and to be honest I found it quite hard to write it given the constraints imposed by Go's type system. Now it compiles and it seems to be working. To use it in my code, I needed additional helper functions with specific return types. All in all it seems to be a ugly hack with a lot of repetition. I'd like to see if this is the best one can do in Go.

Note: especially terrible is the distinction between map[string]string and map[string]interface{}. I though that interface{} would represent every type, but apparently not in this case. If I remove that part from the switch statement, the program does not work because map[string]string is not handled by the map[string]interface{} branch.

package main

import (
    "fmt"
    "strconv"
)

func main() {
    test := map[string]interface{}{
        "root": map[string]interface{}{
            "inner": []interface{}{
                map[string]string{"name": "obj1"},
                map[string]string{"name": "obj2"},
            },
        },
    }
    fmt.Println(TraverseToStr(test, []string{"root", "inner", "0", "name"}))
}

func TraverseToUint64(data interface{}, path []string) uint64 {
    val, err := Traverse(data, path)
    if err != nil {
        return 0
    } else {
        return val.(uint64)
    }
}

func TraverseToStr(data interface{}, path []string) string {
    val, err := Traverse(data, path)
    if err != nil {
        return ""
    } else {
        return val.(string)
    }
}

func TraverseToMap(data interface{}, path []string) map[string]interface{} {
    val, err := Traverse(data, path)
    if err != nil {
        return make(map[string]interface{})
    } else {
        return val.(map[string]interface{})
    }
}

func TraverseToArray(data interface{}, path []string) []interface{} {
    val, err := Traverse(data, path)
    if err != nil {
        return make([]interface{}, 0)
    } else {
        return val.([]interface{})
    }
}

func Traverse(data interface{}, path []string) (interface{}, error) {
    var ok bool
    current := data
    for _, p := range path {
        switch current.(type) {
        case map[string]string:
            if current, ok = current.(map[string]string)[p]; !ok {
                return nil, fmt.Errorf("key not found in map: %s", p)
            }
        case map[string]interface{}:
            if current, ok = current.(map[string]interface{})[p]; !ok {
                return nil, fmt.Errorf("key not found in map: %s", p)
            }
        case []interface{}:
            if i, err := strconv.ParseInt(p, 10, 64); err == nil {
                array := current.([]interface{})
                if int(i) < len(array) {
                    current = array[i]
                } else {
                    return nil, fmt.Errorf("index %d out of bounds for %v", i, array)
                }
            } else {
                return nil, fmt.Errorf("integer required, got: %s", p)
            }
        default:
            return nil, fmt.Errorf("cannot traverse %T\n", current)
        }
    }
    return current, nil
}
```

Answer 1

Readability is essential for correct and maintainable programs.

---

For example,

traverse.go:

package main

import (
    "fmt"
    "strconv"
)

func main() {
    test := map[string]interface{}{
        "root": map[string]interface{}{
            "inner": []interface{}{
                map[string]string{"name": "obj1"},
                map[string]string{"name": "obj2"},
            },
        },
    }
    fmt.Println(
        TraverseToStr(test, []string{"root", "inner", "0", "name"}),
    )
}

func TraverseToUint64(data interface{}, path []string) (uint64, error) {
    val, err := Traverse(data, path)
    if err != nil {
        return 0, err
    }
    return val.(uint64), nil
}

func TraverseToStr(data interface{}, path []string) (string, error) {
    val, err := Traverse(data, path)
    if err != nil {
        return "", err
    }
    return val.(string), nil
}

func TraverseToMap(data interface{}, path []string) (map[string]interface{}, error) {
    val, err := Traverse(data, path)
    if err != nil {
        return nil, err
    }
    return val.(map[string]interface{}), nil
}

func TraverseToArray(data interface{}, path []string) ([]interface{}, error) {
    val, err := Traverse(data, path)
    if err != nil {
        return nil, err
    }
    return val.([]interface{}), nil
}

func Traverse(data interface{}, path []string) (interface{}, error) {
    var ok bool
    current := data
    for _, p := range path {
        switch current.(type) {
        case map[string]string:
            if current, ok = current.(map[string]string)[p]; !ok {
                return nil, fmt.Errorf("key not found in map: %s", p)
            }
        case map[string]interface{}:
            if current, ok = current.(map[string]interface{})[p]; !ok {
                return nil, fmt.Errorf("key not found in map: %s", p)
            }
        case []interface{}:
            i, err := strconv.ParseInt(p, 10, 64)
            if err != nil {
                return nil, fmt.Errorf("integer required, got: %s", p)
            }
            array := current.([]interface{})
            if i < 0 || i >= int64(len(array)) {
                return nil, fmt.Errorf("index %d out of bounds for %v", i, array)
            }
            current = array[i]
        default:
            return nil, fmt.Errorf("cannot traverse %T\n", current)
        }
    }
    return current, nil
}

Playground: https://play.golang.org/p/fMcDZr1AZy_X

---

Commentary:

Readability is essential for correct and maintainable programs.

---

fmt.Println(
    TraverseToStr(test, []string{"root", "inner", "0", "name"}),
)

The primary line of code should be obvious. The first read through, we focus on the primary line, returning later to read error handling and other secondary details.

TraverseToStr is the primary code, fmt.Println is a secondary detail. We highlight the primary code on a separate line.

---

func TraverseToUint64(data interface{}, path []string) (uint64, error) {
    val, err := Traverse(data, path)
    if err != nil {
        return 0, err
    }
    return val.(uint64), nil
}

In Go, don't ignore errors.

Unnecessary conditional indentation is hard to read. An idiosyncratic coding style is extrememly hard to read. Return immediately on error. CodeReviewComments: Indent Error Flow

---

func TraverseToMap(data interface{}, path []string) (map[string]interface{}, error) {
    val, err := Traverse(data, path)
    if err != nil {
        return map[string]interface{}{}, err
    }
    return val.(map[string]interface{}), nil
}

Replace make(map[string]interface{}), a built-in function call, with map[string]interface{}{}, a composite literal value.

func TraverseToMap(data interface{}, path []string) (map[string]interface{}, error) {
    val, err := Traverse(data, path)
    if err != nil {
        return nil, err
    }
    return val.(map[string]interface{}), nil
}

However, on error, return the zero value, nil for a map.

---

i, err := strconv.ParseInt(p, 10, 64)
if err != nil {
    return nil, fmt.Errorf("integer required, got: %s", p)
}
array := current.([]interface{})
if i < 0 || i >= int64(len(array)) {
    return nil, fmt.Errorf("index %d out of bounds for %v", i, array)
}
current = array[i]

Replace a stream of code with the basic Go error pattern.

Don't overuse the "if expression; simple-statement statement" form.

---

if i < 0 || i >= int64(len(array)) {
    return nil, fmt.Errorf("index %d out of bounds for %v", i, array)
}

When you see this error message,

invalid operation: i < len(array) (mismatched types int64 and int)

and this fix,

if int(i) < len(array) {
}

stop and think. i may be negative. i is int64, int(i) will discard high-order bits when int is implemented as 32-bits.

---

The Go Programming Language Specification

The Go Programming Language

"I though that" is not sufficient justification. Use authoritative sources.

Go is strongly typed. Types interface{}, map[string]string, and map[string]interface{} are distinct and have different memory layouts.

---

When the code is readable, it's possible to read much of the code and prove it correct.

Sometimes it can be hard to establish invariants for maps. "The iteration order over maps is not specified and is not guaranteed to be the same from one iteration to the next." The Go Programming Language Specification. Also, since the Go 1.12 release, "fmt: Maps are now printed in key-sorted order to ease testing." Which leads some to believe, incorrectly, that maps are sorted.

For this code, it's important to note that maps are not read iteratively in a for loop in random order. All map accesses are deterministic.