Working With Opaque Data in Go

Opaque data, is data where you don’t know what information it contains. For this article we are going to assume it has some structure: the data is in key, value form. The keys are strings. The values can be maps, arrays or single values.

In Go this can be modeled as map[string]interface{}. Most encoding packages will encode and decode into values using this type. To be more precise the two types used are map[string]interface{} for maps and []interface{} for arrays. Using these limited types to model opaque data is a good idea, even if you are not encoding/decoding, rather than using struct.

The meaning of the keys and values are not defined. But we still want to be able to work with data like this.

Merge Example:

This function will merge any data in src into dst preserving the structure, but will not overwrite plain values. All that is required is to use a type switch to deal with maps and array structure. Having this limited set of map and array types to deal with makes this easy.

 1// doesn't overwrite
 2func mergeMapStringInterface(dst, src map[string]interface{}) {
 3  for k, v := range src {
 4    if _, ok := dst[k]; ok {
 5      switch v2 := v.(type) {
 6      case map[string]interface{}:
 7        if v3, ok := dst[k].(map[string]interface{}); ok {
 8          mergeMapStringInterface(v3, v2)
 9        }
10      case []interface{}:
11        if v3, ok := dst[k].([]interface{}); ok {
12          dst[k] = append(v3, v2...)
13        }
14      default:
15        // don't overwrite plain values
16      }
17    } else {
18      dst[k] = src[k]
19    }
20  }
21}

Zero Values

It’s worth remembering that in Go indexing a map with a key which does not exist evalutes to the zero value for the type defined in the map. In other languages doing this can raise an exception.

Also when using a type assertions that assign to two variables (the special form, with boolean check), if the type assertions fails it also results in the assignment to be the zero value of the asserted type.

Combining these two features can make working with data easier

Checking a Value

In the following code if the value for “expires” doesn’t exist or if it’s not of type time.Time then we don’t run the expiry check. When using this method, what the zero value represents needs to be ruled out as being signficant, in many cases a zero value time.Time or empty string is invalid. (Note the brackets around the if clause, needed because of literal here, the parser needs this, a bit of 👀❓with Go, oldschool)

1if expires, _ := fields["expires"].(time.Time); (expires != time.Time{}) {
2  if time.Now().Before(expires) {
3    loggedIn = true
4  }
5}

The Meaning of Things

Here we creating meaning out of “public_metrics.like_count” and “likeCount”. So we don’t know the data, it could be from some API, it’s opaque. But we’ve created meaning here, they are likes 👍📈. The way of thinking about this is treating data as opaque even though you may know roughly what the data is. Probably the best use of this, is giving the user of your app the power to give meaning to data they are interested in.

 1var likeSemantics = map[string][]string{"likes" : {"public_metrics.like_count", "likeCount"}}
 2
 3func getLikes(data map[string]interface{}) (out map[string]interface{}) {
 4  for k, v := range data {
 5    switch v2 := v.(type) {
 6    case map[string]interface{}:
 7        getLikes(v2)
 8    case []interface{}:
 9        for _, v3 := range v2 {
10          if v4, ok := v3.(map[string]interface{}); ok {
11            getLikes(v4)
12          }
13        }
14    case int:
15      for k2, v5 := range likeSemantics {
16        for _, name := range v5 {
17          if k == name {
18            return map[string]interface{}{k2: v2}
19          }
20        }
21      }
22    }
23  }
24  return
25}