Go, Golang Unmarshal JSON sent by POST method?

stergios

Hi,

Has anyone created the Go struct necessary to unmarshal the JSON sent via the POST method? Would you be willing to share it?

I've been using the default GET method for server side processing with a Go backend, but the parsing is error prone and I would prefer to have an unmarshal library do this for me.

If I have to invent the wheel here, is there a spec for the JSON sent by the POST method? I see the server side processing page [https://datatables.net/manual/server-side], but that seems more suited to the GET method. I imagine the correlation to POST is pretty high, but a document meant for a JSON endpoint would be useful.

Of course, I'll run lots of examples being sure to capture the JSON of various configurations, keeping a keen eye on the server side processing page all the while. But if I could stand on the shoulders of anyone who's already blazed this trail, that would be awesome!

Thanks,

Stergios

kthorngren

I don't know anything about Go but the parameters described in Server Side Processing doc are the same for GET and POST requests. You can use the browser's network inspector with this example to see the parameters sent in a SSP POST request. Does this help?

Kevin

allan

I'm afraid I also haven't tried Go yet (it's on my list, but the list is ever expanding!), however, rather than reading in HTTP parameters, you could have DataTables send the request information as JSON in the request body:

$('#example').dataTable( {
  "ajax": {
    "url": "data.json",
    "contentType": "application/json",
    "type": "POST",
    "data": function ( d ) {
      return JSON.stringify( d );
    }
  }
} );

(from the last example on ajax.data).

That said, there much be a Go library for handling http parameters... Looking around this article looks like a fairly good introduction to the topic. url.ParseQuery might be what you need for a GET request. I'm guessing there is something similar for request body (i.e. POST) parameters.

Regards,
Allan

stergios

Yep, that's how I have it working as well, except I need to set processData:false to avoid being transformed into a query string (plus I add additional fields).

    "ajax": {
      "url": "/getData",
      "method": "POST",
      "dataType": "json",
      "processData": false, // avoid being transformed into a query string,
      "contentType": 'application/json',
      data: function(d){
        d.startDate = g_cqStartDate ;
        d.endDate = g_cqEndDate;
        return JSON.stringify(d);
      }
    },

I use the echo framework which includes all the usual URI parsing which returns query parameters via a map(string, string). However some of the DataTables parameters get mapped into the key name (this is due to array serialization I believe). For example, the 5h column has the following parameter names associated with it.

KEY:                           VALUE:
columns[4][search][regex]:     [false]
columns[4][search][value]:     []
columns[4][searchable]   :     [true]

So now you have to search the parameter key space for existence check. Ick! That's why I'm switching over to the POST method.

Here's the first json blob below sent by DataTables. That's straight forward enough to map into a proper golang data structure required by unmarshalling. Hopefully there are no "surprise" fields added as other DataTables functionality is invoked. There are no other fields mentioned on https://datatables.net/manual/server-side so I'm optimistic about this being straight forward.

By documenting the JSON blob sent via POST you intrinsically support all other programming languages. Even the ones you have no documentation on as all(?) languages seem to have json support these days. So here's my contribution to get the ball rolling! (But really, DataTables already has exceptional documentation, which is one more big reason I am a happy customer.)

{
   "columns" : [
      {
         "data" : "",
         "name" : "",
         "orderable" : false,
         "search" : {
            "regex" : false,
            "value" : ""
         },
         "searchable" : false
      },
      {
         "data" : "Id",
         "name" : "",
         "orderable" : false,
         "search" : {
            "regex" : false,
            "value" : ""
         },
         "searchable" : true
      }, /* ... other columns deleted */
   ],
   "draw" : 1,
   "endDate" : "2021-01-01",
   "length" : 5,
   "order" : [
      {
         "column" : 0,
         "dir" : "asc"
      }
   ],
   "search" : {
      "regex" : false,
      "value" : ""
   },
   "start" : 0,
   "startDate" : "2008-01-01"
}

allan

Many thanks! I guess the server-side processing documentation sort of says that, but it needs to be inferred from the HTTP parameters. I’ll have a look at adding a json schema for it.

Most of the extensions and plug-ins won’t add or alter that structure at all. SearchPanes can require extra information though, which is documented here.

If you make a Go map for this, I’m sure others would appreciate it you are willing to share it?

Allan

stergios

I'll share the POST solution I come up with. It's a back burner task for me as I already have GET working for some time now. But every time I go look at the code I panic just a bit inside and say to myself "there must be a better way".

Figure a couple of weeks or so. BTW, I had not noticed the SearchPanes feature before. I'll see if I can address that too; no promises!

Stergios

stergios

Well, looks like I have some early success in the effort. Here's some working code below. Feel free to ask questions.

    type SearchStruct struct {
        Regex bool          `json:"regex"`
        Value string        `json:"value"`
    }
    type Column struct {
        Data int            `json:"data"`
        Name string         `json:"name"`
        Orderable bool      `json:"orderable"`
        Search SearchStruct `json:"search"`
        Searchable bool     `json:"searchable"`
    }
    type Order struct {
        Column int          `json:"column"`
        Dir string          `json:"dir"`
    }
    type DTJson struct {
        Columms []Column    `json:"columns"`
        Draw int            `json:"draw"`
        Length int          `json:"length"`
        Orders []Order      `json:"order"`
        Search SearchStruct `json:"search"`
        Start int           `json:"start"`
    }

    body, err := ioutil.ReadAll(c.Request().Body)
    if err != nil {
            panic(err)
    }
    fmt.Printf("Request Body:\n%s\n", body)

    var d DTJson
    err = json.Unmarshal(body, &d)
    if err != nil {
        panic(err)
    }
    fmt.Println(spew.Sdump(d))

stergios

A quick test to verify that it worked is to reproduce the json from the go data structure.

    jsonBlob, err := json.MarshalIndent(d, "", "  ")
    if err == nil {
        fmt.Printf("%s\n", jsonBlob)
    } else {
        fmt.Printf("Error: %v\n", err)
    }

allan

That's superb - thanks for sharing this! It will really help with my own adventures in Go .

Allan