I've created an API in Go that, upon being called, performs a query, creates an instance of a struct, and then encodes that struct as JSON before sending back to the caller. I'd now like to allow the caller to be able to select the specific fields they would like returned by passing in a "fields" GET parameter.
This means depending on the fields value(s), my struct would change. Is there any way to remove fields from a struct? Or at least hide them in the JSON response dynamically? (Note: Sometimes I have empty values so the JSON omitEmpty tag will not work here) If neither of these are possible, is there a suggestion on a better way to handle this?
A smaller version of the structs I'm using are below:
type SearchResult struct {
Date string `json:"date"`
IdCompany int `json:"idCompany"`
Company string `json:"company"`
IdIndustry interface{} `json:"idIndustry"`
Industry string `json:"industry"`
IdContinent interface{} `json:"idContinent"`
Continent string `json:"continent"`
IdCountry interface{} `json:"idCountry"`
Country string `json:"country"`
IdState interface{} `json:"idState"`
State string `json:"state"`
IdCity interface{} `json:"idCity"`
City string `json:"city"`
} //SearchResult
type SearchResults struct {
NumberResults int `json:"numberResults"`
Results []SearchResult `json:"results"`
} //type SearchResults
I then encode and output the response like so:
err := json.NewEncoder(c.ResponseWriter).Encode(&msg)
Here is how I defined my structure.
type User struct {
Username string `json:"username" bson:"username"`
Email string `json:"email" bson:"email"`
Password *string `json:"password,omitempty" bson:"password"`
FullName string `json:"fullname" bson:"fullname"`
}
And inside my function set user.Password = nil for not to be Marshalled.
Another way to do this is to have a struct of pointers with the ,omitempty tag. If the pointers are nil, the fields won't be Marshalled.
This method will not require additional reflection or inefficient use of maps.
Same example as jorelli using this method: http://play.golang.org/p/JJNa0m2_nw
Take three ingredients:
The reflect package to loop over all the fields of a struct.
An if statement to pick up the fields you want to Marshal, and
The encoding/json package to Marshal the fields of your liking.
Preparation:
Blend them in a good proportion. Use reflect.TypeOf(your_struct).Field(i).Name() to get a name of the ith field of your_struct.
Use reflect.ValueOf(your_struct).Field(i) to get a type Value representation of an ith field of your_struct.
Use fieldValue.Interface() to retrieve the actual value (upcasted to type interface{}) of the fieldValue of type Value (note the bracket use - the Interface() method produces interface{}
If you luckily manage not to burn any transistors or circuit-breakers in the process you should get something like this:
func MarshalOnlyFields(structa interface{},
includeFields map[string]bool) (jsona []byte, status error) {
value := reflect.ValueOf(structa)
typa := reflect.TypeOf(structa)
size := value.NumField()
jsona = append(jsona, '{')
for i := 0; i < size; i++ {
structValue := value.Field(i)
var fieldName string = typa.Field(i).Name
if marshalledField, marshalStatus := json.Marshal((structValue).Interface()); marshalStatus != nil {
return []byte{}, marshalStatus
} else {
if includeFields[fieldName] {
jsona = append(jsona, '"')
jsona = append(jsona, []byte(fieldName)...)
jsona = append(jsona, '"')
jsona = append(jsona, ':')
jsona = append(jsona, (marshalledField)...)
if i+1 != len(includeFields) {
jsona = append(jsona, ',')
}
}
}
}
jsona = append(jsona, '}')
return
}
Serving:
serve with an arbitrary struct and a map[string]bool of fields you want to include, for example
type magic struct {
Magic1 int
Magic2 string
Magic3 [2]int
}
func main() {
var magic = magic{0, "tusia", [2]int{0, 1}}
if json, status := MarshalOnlyFields(magic, map[string]bool{"Magic1": true}); status != nil {
println("error")
} else {
fmt.Println(string(json))
}
}
Bon Appetit!
I also faced this problem, at first I just wanted to specialize the responses in my http handler. My first approach was creating a package that copies the information of a struct to another struct and then marshal that second struct. I did that package using reflection, so, never liked that approach and also I wasn't dynamically.
So I decided to modify the encoding/json package to do this. The functions Marshal, MarshalIndent and (Encoder) Encode additionally receives a
type F map[string]F
I wanted to simulate a JSON of the fields that are needed to marshal, so it only marshals the fields that are in the map.
https://github.com/JuanTorr/jsont
package main
import (
"fmt"
"log"
"net/http"
"github.com/JuanTorr/jsont"
)
type SearchResult struct {
Date string `json:"date"`
IdCompany int `json:"idCompany"`
Company string `json:"company"`
IdIndustry interface{} `json:"idIndustry"`
Industry string `json:"industry"`
IdContinent interface{} `json:"idContinent"`
Continent string `json:"continent"`
IdCountry interface{} `json:"idCountry"`
Country string `json:"country"`
IdState interface{} `json:"idState"`
State string `json:"state"`
IdCity interface{} `json:"idCity"`
City string `json:"city"`
} //SearchResult
type SearchResults struct {
NumberResults int `json:"numberResults"`
Results []SearchResult `json:"results"`
} //type SearchResults
func main() {
msg := SearchResults{
NumberResults: 2,
Results: []SearchResult{
{
Date: "12-12-12",
IdCompany: 1,
Company: "alfa",
IdIndustry: 1,
Industry: "IT",
IdContinent: 1,
Continent: "america",
IdCountry: 1,
Country: "México",
IdState: 1,
State: "CDMX",
IdCity: 1,
City: "Atz",
},
{
Date: "12-12-12",
IdCompany: 2,
Company: "beta",
IdIndustry: 1,
Industry: "IT",
IdContinent: 1,
Continent: "america",
IdCountry: 2,
Country: "USA",
IdState: 2,
State: "TX",
IdCity: 2,
City: "XYZ",
},
},
}
fmt.Println(msg)
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
//{"numberResults":2,"results":[{"date":"12-12-12","idCompany":1,"idIndustry":1,"country":"México"},{"date":"12-12-12","idCompany":2,"idIndustry":1,"country":"USA"}]}
err := jsont.NewEncoder(w).Encode(msg, jsont.F{
"numberResults": nil,
"results": jsont.F{
"date": nil,
"idCompany": nil,
"idIndustry": nil,
"country": nil,
},
})
if err != nil {
log.Fatal(err)
}
})
http.ListenAndServe(":3009", nil)
}
Other answers already explained that this isn't possible using the encoding/json package features directly.
Instead, I'd like to offer an alternative solution using a third-party library. Disclaimer: I am the author of the library.
https://github.com/wI2L/jettison
Jettison is a faster and flexible alternative to encoding/json, while still retaining a behavior identical to the standard library by default.
You can use the DenyList option to filter-out fields that should be omitted from the result. Note that the feature is limited, and only works for 1st-level fields.
I just published sheriff, which transforms structs to a map based on tags annotated on the struct fields. You can then marshal (JSON or others) the generated map. It probably doesn't allow you to only serialize the set of fields the caller requested, but I imagine using a set of groups would allow you to cover most cases. Using groups instead of the fields directly would most likely also increase cache-ability.
Example:
package main
import (
"encoding/json"
"fmt"
"log"
"github.com/hashicorp/go-version"
"github.com/liip/sheriff"
)
type User struct {
Username string `json:"username" groups:"api"`
Email string `json:"email" groups:"personal"`
Name string `json:"name" groups:"api"`
Roles []string `json:"roles" groups:"api" since:"2"`
}
func main() {
user := User{
Username: "alice",
Email: "[email protected]",
Name: "Alice",
Roles: []string{"user", "admin"},
}
v2, err := version.NewVersion("2.0.0")
if err != nil {
log.Panic(err)
}
o := &sheriff.Options{
Groups: []string{"api"},
ApiVersion: v2,
}
data, err := sheriff.Marshal(o, user)
if err != nil {
log.Panic(err)
}
output, err := json.MarshalIndent(data, "", " ")
if err != nil {
log.Panic(err)
}
fmt.Printf("%s", output)
}
The question is now a bit old, but I came across the same issue a little while ago, and as I found no easy way to do this, I built a library fulfilling this purpose.
It allows to easily generate a map[string]interface{} from a static struct.
I created this function to convert struct to JSON string by ignoring some fields. Hope it will help.
func GetJSONString(obj interface{}, ignoreFields ...string) (string, error) {
toJson, err := json.Marshal(obj)
if err != nil {
return "", err
}
if len(ignoreFields) == 0 {
return string(toJson), nil
}
toMap := map[string]interface{}{}
json.Unmarshal([]byte(string(toJson)), &toMap)
for _, field := range ignoreFields {
delete(toMap, field)
}
toJson, err = json.Marshal(toMap)
if err != nil {
return "", err
}
return string(toJson), nil
}
You can use tagging attribute "omitifempty" or make optional fields pointers and leave those you want skipped uninitialized.
I didn't have the same problem but similar. Below code solves your problem too, of course if you don't mind performance issue. Before implement that kind of solution to your system I recommend you to redesign your structure if you can. Sending variable structure response is over-engineering. I believe a response structure represents a contract between a request and resource and it should't be depend requests.(you can make un-wanted fields null, I do). In some cases we have to implement this design, if you believe you are in that cases here is the play link and code I use.
type User2 struct {
ID int `groups:"id" json:"id,omitempty"`
Username string `groups:"username" json:"username,omitempty"`
Nickname string `groups:"nickname" json:"nickname,omitempty"`
}
type User struct {
ID int `groups:"private,public" json:"id,omitempty"`
Username string `groups:"private" json:"username,omitempty"`
Nickname string `groups:"public" json:"nickname,omitempty"`
}
var (
tagName = "groups"
)
//OmitFields sets fields nil by checking their tag group value and access control tags(acTags)
func OmitFields(obj interface{}, acTags []string) {
//nilV := reflect.Value{}
sv := reflect.ValueOf(obj).Elem()
st := sv.Type()
if sv.Kind() == reflect.Struct {
for i := 0; i < st.NumField(); i++ {
fieldVal := sv.Field(i)
if fieldVal.CanSet() {
tagStr := st.Field(i).Tag.Get(tagName)
if len(tagStr) == 0 {
continue
}
tagList := strings.Split(strings.Replace(tagStr, " ", "", -1), ",")
//fmt.Println(tagList)
// ContainsCommonItem checks whether there is at least one common item in arrays
if !ContainsCommonItem(tagList, acTags) {
fieldVal.Set(reflect.Zero(fieldVal.Type()))
}
}
}
}
}
//ContainsCommonItem checks if arrays have at least one equal item
func ContainsCommonItem(arr1 []string, arr2 []string) bool {
for i := 0; i < len(arr1); i++ {
for j := 0; j < len(arr2); j++ {
if arr1[i] == arr2[j] {
return true
}
}
}
return false
}
func main() {
u := User{ID: 1, Username: "very secret", Nickname: "hinzir"}
//assume authenticated user doesn't has permission to access private fields
OmitFields(&u, []string{"public"})
bytes, _ := json.Marshal(&u)
fmt.Println(string(bytes))
u2 := User2{ID: 1, Username: "very secret", Nickname: "hinzir"}
//you want to filter fields by field names
OmitFields(&u2, []string{"id", "nickname"})
bytes, _ = json.Marshal(&u2)
fmt.Println(string(bytes))
}
use `json:"-"`
// Field is ignored by this package.
Field int `json:"-"`
// Field appears in JSON as key "myName".
Field int `json:"myName"`
// Field appears in JSON as key "myName" and
// the field is omitted from the object if its value is empty,
// as defined above.
Field int `json:"myName,omitempty"`
// Field appears in JSON as key "Field" (the default), but
// the field is skipped if empty.
// Note the leading comma.
Field int `json:",omitempty"`
You can use the reflect package to select the fields that you want by reflecting on the field tags and selecting the json tag values. Define a method on your SearchResults type that selects the fields you want and returns them as a map[string]interface{}, and then marshal that instead of the SearchResults struct itself. Here's an example of how you might define that method:
func fieldSet(fields ...string) map[string]bool {
set := make(map[string]bool, len(fields))
for _, s := range fields {
set[s] = true
}
return set
}
func (s *SearchResult) SelectFields(fields ...string) map[string]interface{} {
fs := fieldSet(fields...)
rt, rv := reflect.TypeOf(*s), reflect.ValueOf(*s)
out := make(map[string]interface{}, rt.NumField())
for i := 0; i < rt.NumField(); i++ {
field := rt.Field(i)
jsonKey := field.Tag.Get("json")
if fs[jsonKey] {
out[jsonKey] = rv.Field(i).Interface()
}
}
return out
}
and here's a runnable solution that shows how you would call this method and marshal your selection: http://play.golang.org/p/1K9xjQRnO8
Source: Stackoverflow.com