Reducing JSON size for Editor

burnchar

Is it possible to load initial table data for Editor without sending the name of each column for every row?

The AJAX source example uses a format like so:

[code]
"aaData": [
[
"Trident",
"Internet Explorer 4.0",
"Win 95+",
"4",
"X"
]
[/code]

The "Joined Tables" example sends every column name N times, where N is the number of rows:
[code]
"aaData": [
{
"DT_RowId": "row_1",
"first_name": "Quynn",
"last_name": "Contreras",
"dept": {
"id": "1",
"name": "IT"
},
"access": [
{
"id": "1",
"name": "Printer"
},
{
"id": "3",
"name": "Desktop"
},
{
"id": "4",
"name": "VMs"
}
],
"extra": {}
}
[/code]



This triples or quadruples my data size, leading to loads of more than 500KB in some cases!
Compression helps somewhat, but then the browser itself becomes a bottleneck sifting through that much data in Javascript ... Especially glacially slow browsers like IE8.


Is it possible to send the JSON to DataTables/Editor without the names?
Those names of course are used to reference the data by name in the "fields" section of Editor initialization for example, so maybe something like this? (refactor the names to a single spot):

[code]
"fields": ["DT_RowId", "first_name", "last_name", "dept", "access", "extra"],
"aaData": [
{
"row_1",
"Quynn",
"Contreras",
{
"id": "1",
"name": "IT"
},
[
{
"id": "1",
"name": "Printer"
},
{
"id": "3",
"name": "Desktop"
},
{
"id": "4",
"name": "VMs"
}
],
{}
}
[/code]

(Ideally the contents of joined table specifiers could be refactored too, e.g. the "id" and "name" parts)

Is anything that can reduce data size like this currently possible or planned for future versions?

allan

> Is it possible to send the JSON to DataTables/Editor without the names?

Yes - absolutely. Simply use mData in DataTables ( `dataProp` in Editor - https://editor.datatables.net/fields/#dataProp ) to point to the array index that you want from the data source for the field.

My examples use object based data in most cases, since it is much easier to understand and track the data through the flow - but there is no reason why arrays can't be used to reduce the size if you want. You just need to keep track of the indexes rather than the object keys.

Regards,
Allan

burnchar

In all my use of DataTables, I had no idea it would be so easy. Editor's elegant and flexible design is a marvel.
Thanks!

allan

Thank you - I'm blushing... :-)

Allan

burnchar

I thought I'd post some sample code segments to help out others. The following code uses joined tables with index-based JSON.


The first section is the Editor initialization:

[code]
// Editor initialization
editor = new $.fn.dataTable.Editor({
"ajaxUrl": {
"create": "POST "~/API/SomeTable",
"edit": "PUT "~/API/SomeTable",
"remove": "DELETE "~/API/SomeTable"
},
"domTable": "#editorview",
"fields": [
// Notice that dataProp can be an integer, but when it represents
// an object dataProp must be a string to represent the hash key ("id")
// Same goes for the DataTables initialization below
{ "name": "serial_no", "dataProp": 1, "label": "Ser#" },
...
{ "name": "status_code_fk", "dataProp": "12.id", "label": "Status", "type": "select" },
{ "name": "group_code_fk", "dataProp": "13.id", "label": "Group", "type": "select" },
...
]
});
[/code]

DataTables init. Here you use mData rather than dataProp.
It wasn't immediately clear to me that the name passed to `editor.field()` is the verbatim text used in the name field above, not the `dataProp` value (as is done when not using dataProp, when it defaults to the name) or the name with the dotted JavaScript notation, e.g. "status_code_fk.id":

[code]
// DataTables initialization snippet
DataTableObject = $('#editorview').dataTable({
"sDom": '<"H"lfr<"br /">T>t<"F"ip>',
"sAjaxSource": "~/API/SomeTable?parameter=value",
"aoColumns": [
{ "mData": 1 },
...
{ "mData": "12.name", "sDefaultContent": "" },
{ "mData": "13.name", "sDefaultContent": "" },
],
{
"fnInitComplete": function (settings, json) {
// Notice no ".id" in the field name, because we pass the
// literal text used in "fields" above
editor.field('status_code_fk').update(json.status_code);
editor.field('group_code_fk').update(json.group_code);
editor.field('vendor_fk').update(json.vendor);
}
});
[/code]

The JSON doesn't change much other than that the name for each trunk value is removed, making it an array data structure rather than a hash table. I didn't experiment with refactoring the "id" and "name" parts because I suspect that doing so would not yield a sufficient the benefit/time tradeoff. If anyone else does the work, please post an example for everyone:

[code]
{
"aaData": [
[
// Most values like below are sent without any extra syntax.
// The joined table values below are slightly more complex.
10098.0,
...
{
"id": 1,
"name": "Idaho"
},
{
"id": 42,
"name": "Some select box value"
},
...

"status_code": [{
"value": 1,
"label": "Idaho"
},
{
"value": 2,
"label": "California"
},
{
"value": 3,
"label": "Arizona"
}, ...
]
}
[/code]

allan

Thanks for sharing this with us!

> It wasn't immediately clear to me that the name passed to editor.field() is the verbatim text used in the name field above, not the dataProp value

I'm working on the new documentation for Editor 1.3 at the moment, and I'm just about to start on the initialisation properties. I've tried to make this clear in the API documentation that has been written, but I will be doubly careful with the init properties.

One thing that might help is that Editor is going to use `fields.data` in preference to `fields.dataProp` (but fully backwards compatible!) similar to DataTables' `columns.data` in 1.10. Trying to harmonise the two a bit more...

Allan

burnchar

Oh that wasn't meant as a slight on your documentation, but on my Javascript skills. I learned Javascript through use of DataTables and Editor. Other confusions included "3.id" and such, which is very weird coming from C++ and x86 assembly language. It's just a Javascript quirk, and a pretty cool one at that.

Your new changes sound really good though. Thank you for your efforts.
Just let me know when you think the time is write for .NET examples/docs.

allan

No slight taken - I'm on a bit of a mission with the documentation... :-)

Allan

burnchar

Allan -- How do I set DT_RowID with my simple array?
I just started testing and found that after submission, I get "Requested unknown parameter '1' from the data source for row 0"

Perhaps I assumed that Editor would use the first column (probably a bad assumption) for the ID.
Since a simple array is being used, I can't have an element named "DT_RowId" because no names exist.

The documentation for idSrc mentions that it supports dotted notation. Is there a way to tell it to read the first (0) column from each array? I've tried setting it like so:

`"idSrc": { "dataProp": 0 },`

...Which I didn't really expect to work, but I am unsure what to use since I do not know the state during DT initialization.
What am I missing?

allan

> How do I set DT_RowID with my simple array?

You don't - since you are returning an array, you can't sent object parameters. There are two options:

1. idSrc as you mention - set it like `idSrc: 0` - that should be all that is needed
2. You could return an object with integers as the keys. Still more space required than an array though.

Allan

burnchar

I had tried that first, but a piece of code I hadn't modified for the change from object to array masked that it worked fine. Thank you for setting me straight, since once I had fixed the code in question, I might have had in mind the `"idSrc"` : 0 didn't work and would have had to troubleshoot all over again.



Reminder to those reading this thread: If you are generating your own JSON, be sure to remember to alter the code that generates the server response to your edit! Mine was still sending nested objects.