Reducing JSON size for Editor
Reducing JSON size for Editor
Is it possible to load initial table data for Editor without sending the name of each column for every row?
The AJAX source example uses a format like so:
[code]
"aaData": [
[
"Trident",
"Internet Explorer 4.0",
"Win 95+",
"4",
"X"
]
[/code]
The "Joined Tables" example sends every column name N times, where N is the number of rows:
[code]
"aaData": [
{
"DT_RowId": "row_1",
"first_name": "Quynn",
"last_name": "Contreras",
"dept": {
"id": "1",
"name": "IT"
},
"access": [
{
"id": "1",
"name": "Printer"
},
{
"id": "3",
"name": "Desktop"
},
{
"id": "4",
"name": "VMs"
}
],
"extra": {}
}
[/code]
This triples or quadruples my data size, leading to loads of more than 500KB in some cases!
Compression helps somewhat, but then the browser itself becomes a bottleneck sifting through that much data in Javascript ... Especially glacially slow browsers like IE8.
Is it possible to send the JSON to DataTables/Editor without the names?
Those names of course are used to reference the data by name in the "fields" section of Editor initialization for example, so maybe something like this? (refactor the names to a single spot):
[code]
"fields": ["DT_RowId", "first_name", "last_name", "dept", "access", "extra"],
"aaData": [
{
"row_1",
"Quynn",
"Contreras",
{
"id": "1",
"name": "IT"
},
[
{
"id": "1",
"name": "Printer"
},
{
"id": "3",
"name": "Desktop"
},
{
"id": "4",
"name": "VMs"
}
],
{}
}
[/code]
(Ideally the contents of joined table specifiers could be refactored too, e.g. the "id" and "name" parts)
Is anything that can reduce data size like this currently possible or planned for future versions?
The AJAX source example uses a format like so:
[code]
"aaData": [
[
"Trident",
"Internet Explorer 4.0",
"Win 95+",
"4",
"X"
]
[/code]
The "Joined Tables" example sends every column name N times, where N is the number of rows:
[code]
"aaData": [
{
"DT_RowId": "row_1",
"first_name": "Quynn",
"last_name": "Contreras",
"dept": {
"id": "1",
"name": "IT"
},
"access": [
{
"id": "1",
"name": "Printer"
},
{
"id": "3",
"name": "Desktop"
},
{
"id": "4",
"name": "VMs"
}
],
"extra": {}
}
[/code]
This triples or quadruples my data size, leading to loads of more than 500KB in some cases!
Compression helps somewhat, but then the browser itself becomes a bottleneck sifting through that much data in Javascript ... Especially glacially slow browsers like IE8.
Is it possible to send the JSON to DataTables/Editor without the names?
Those names of course are used to reference the data by name in the "fields" section of Editor initialization for example, so maybe something like this? (refactor the names to a single spot):
[code]
"fields": ["DT_RowId", "first_name", "last_name", "dept", "access", "extra"],
"aaData": [
{
"row_1",
"Quynn",
"Contreras",
{
"id": "1",
"name": "IT"
},
[
{
"id": "1",
"name": "Printer"
},
{
"id": "3",
"name": "Desktop"
},
{
"id": "4",
"name": "VMs"
}
],
{}
}
[/code]
(Ideally the contents of joined table specifiers could be refactored too, e.g. the "id" and "name" parts)
Is anything that can reduce data size like this currently possible or planned for future versions?
This discussion has been closed.
Replies
Yes - absolutely. Simply use mData in DataTables ( `dataProp` in Editor - https://editor.datatables.net/fields/#dataProp ) to point to the array index that you want from the data source for the field.
My examples use object based data in most cases, since it is much easier to understand and track the data through the flow - but there is no reason why arrays can't be used to reduce the size if you want. You just need to keep track of the indexes rather than the object keys.
Regards,
Allan
Thanks!
Allan
The first section is the Editor initialization:
[code]
// Editor initialization
editor = new $.fn.dataTable.Editor({
"ajaxUrl": {
"create": "POST "~/API/SomeTable",
"edit": "PUT "~/API/SomeTable",
"remove": "DELETE "~/API/SomeTable"
},
"domTable": "#editorview",
"fields": [
// Notice that dataProp can be an integer, but when it represents
// an object dataProp must be a string to represent the hash key ("id")
// Same goes for the DataTables initialization below
{ "name": "serial_no", "dataProp": 1, "label": "Ser#" },
...
{ "name": "status_code_fk", "dataProp": "12.id", "label": "Status", "type": "select" },
{ "name": "group_code_fk", "dataProp": "13.id", "label": "Group", "type": "select" },
...
]
});
[/code]
DataTables init. Here you use mData rather than dataProp.
It wasn't immediately clear to me that the name passed to `editor.field()` is the verbatim text used in the name field above, not the `dataProp` value (as is done when not using dataProp, when it defaults to the name) or the name with the dotted JavaScript notation, e.g. "status_code_fk.id":
[code]
// DataTables initialization snippet
DataTableObject = $('#editorview').dataTable({
"sDom": '<"H"lfr<"br /">T>t<"F"ip>',
"sAjaxSource": "~/API/SomeTable?parameter=value",
"aoColumns": [
{ "mData": 1 },
...
{ "mData": "12.name", "sDefaultContent": "" },
{ "mData": "13.name", "sDefaultContent": "" },
],
{
"fnInitComplete": function (settings, json) {
// Notice no ".id" in the field name, because we pass the
// literal text used in "fields" above
editor.field('status_code_fk').update(json.status_code);
editor.field('group_code_fk').update(json.group_code);
editor.field('vendor_fk').update(json.vendor);
}
});
[/code]
The JSON doesn't change much other than that the name for each trunk value is removed, making it an array data structure rather than a hash table. I didn't experiment with refactoring the "id" and "name" parts because I suspect that doing so would not yield a sufficient the benefit/time tradeoff. If anyone else does the work, please post an example for everyone:
[code]
{
"aaData": [
[
// Most values like below are sent without any extra syntax.
// The joined table values below are slightly more complex.
10098.0,
...
{
"id": 1,
"name": "Idaho"
},
{
"id": 42,
"name": "Some select box value"
},
...
"status_code": [{
"value": 1,
"label": "Idaho"
},
{
"value": 2,
"label": "California"
},
{
"value": 3,
"label": "Arizona"
}, ...
]
}
[/code]
> It wasn't immediately clear to me that the name passed to editor.field() is the verbatim text used in the name field above, not the dataProp value
I'm working on the new documentation for Editor 1.3 at the moment, and I'm just about to start on the initialisation properties. I've tried to make this clear in the API documentation that has been written, but I will be doubly careful with the init properties.
One thing that might help is that Editor is going to use `fields.data` in preference to `fields.dataProp` (but fully backwards compatible!) similar to DataTables' `columns.data` in 1.10. Trying to harmonise the two a bit more...
Allan
Your new changes sound really good though. Thank you for your efforts.
Just let me know when you think the time is write for .NET examples/docs.
Allan
I just started testing and found that after submission, I get "Requested unknown parameter '1' from the data source for row 0"
Perhaps I assumed that Editor would use the first column (probably a bad assumption) for the ID.
Since a simple array is being used, I can't have an element named "DT_RowId" because no names exist.
The documentation for idSrc mentions that it supports dotted notation. Is there a way to tell it to read the first (0) column from each array? I've tried setting it like so:
`"idSrc": { "dataProp": 0 },`
...Which I didn't really expect to work, but I am unsure what to use since I do not know the state during DT initialization.
What am I missing?
You don't - since you are returning an array, you can't sent object parameters. There are two options:
1. idSrc as you mention - set it like `idSrc: 0` - that should be all that is needed
2. You could return an object with integers as the keys. Still more space required than an array though.
Allan
Reminder to those reading this thread: If you are generating your own JSON, be sure to remember to alter the code that generates the server response to your edit! Mine was still sending nested objects.