How to use defered rendering with source/array.txt to speed IE8 up, gives data=null, nothing found
How to use defered rendering with source/array.txt to speed IE8 up, gives data=null, nothing found
I have to speed things up for IE8 citrix/xen users on datatables with +7.200 rows.
I already disabled highlighting, but want also use "bDeferRender": true
I already read this, but I don't understand what should be in array.txt. It's empty here.
http://www.datatables.net/release-datatables/examples/ajax/defer_render.html
http://datatables.net/forums/discussion/4739/x
http://www.datatables.net/forums/discussion/5261/defered-rendering-with-local-ajax-source/p1
I already disabled highlighting, but want also use "bDeferRender": true
I already read this, but I don't understand what should be in array.txt. It's empty here.
http://www.datatables.net/release-datatables/examples/ajax/defer_render.html
http://datatables.net/forums/discussion/4739/x
http://www.datatables.net/forums/discussion/5261/defered-rendering-with-local-ajax-source/p1
This discussion has been closed.
Replies
http://www.codeunit.co.za/2011/02/16/jquery-datatables-with-lots-of-columns-crashes-in-ie/
But I want to use it locally and not on a webpage served i.e. by an apache webserver running on 127.0.0.1. I just read the file from disk. But I guess in array.txt I have to put in the json?
I'm out of lunch on this one, sorry. I thought the javascript would read the data in, save it in array.txt as a cache to speed it up. I didn't thought initially on putting something in source/array.txt or even on creating such empty file.
I got the 7.200 table lines like this:
[code]
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
List
.FixedHeader_Cloned th { background-color: white; }
//<![CDATA[
$(document).ready(function() {
$('#example').dataTable( {
"bProcessing": true,
"sAjaxSource": "sources/arrays.txt",
"bDeferRender": true
} );
} );
$(document).ready( function () { var oTable = $('#example').dataTable(
{
/* http://www.datatables.net/forums/discussion/4634/disable-bpaginate-and-blengthchange/p1
http://datatables.net/usage/options#iDisplayLength
*/
"bSortClasses": false, // This is presented as a feature switch as it can increase processing time (while classes are removed and added)
// so for large data sets you might want to turn this off.
// http://www.datatables.net/usage/features#bSortClasses
// http://www.datatables.net/release-datatables/examples/ajax/defer_render.html
// "bProcessing": true, // Enable or disable the display of a 'processing' indicator when the table is
// being processed (e.g. a sort). This is particularly useful for tables with
// large amounts of data where it can take a noticeable amount of time to sort the entries.
/*
"sAjaxSource": "sources/arrays.txt",
"bDeferRender": true, // Beating the IE Dead Horse http://datatables.net/forums/discussion/4739/x
*/
"bLengthChange": true,
"iDisplayLength": 17,
"aLengthMenu": [[ 17, 25, 50, 100, 200, 400, 800, -1], [ 17, 25, 50, 100, 200, 400, 800, "Alle"]],
//"sPaginationType": "listbox",
//"sPaginationType": "input",
//"sPaginationType": "scrolling",
//"sPaginationType": "four_button",
//"sPaginationType": "extStyle",
"sPaginationType": "full_numbers",
/* EN original
"oLanguage": {
"sLengthMenu": "Display _MENU_ records per page",
"sZeroRecords": "Nothing found - sorry",
"sInfo": "Showing _START_ to _END_ of _TOTAL_ records",
"sInfoEmpty": "Showing 0 to 0 of 0 records",
"sInfoFiltered": "(filtered from _MAX_ total records)"
}
*/
//DE
"oLanguage": {
// http://datatables.net/usage/i18n#oLanguage.sLengthMenu
/*"sLengthMenu": ''+
'10<\/option>'+
'25<\/option>'+
'50<\/option>'+
'100<\/option>'+
'200<\/option>'+
'400<\/option>'+
'800<\/option>'+
'Alle<\/option>'+
'<\/select> Zeilen/Seite.',
^^-- Alternative Methode zur nächsten Zeile und "aLengthMenu": [[25, 50, 100, -1], [25, 50, 100, "Alle"]],
*/
"sLengthMenu": "_MENU_ Zeilen/Seite.",
"sZeroRecords": "Leider nichts gefunden.",
"sInfo": "_START_ - _END_ von _TOTAL_ Einträgen.",
"sInfoEmpty": "Zeige 0 - 0 von 0 Einträgen.",
"sInfoFiltered": "(gefiltert von _MAX_ Einträgen insgesamt)",
"sSearch": "Suche:",
"sUrl": "",
"oPaginate": {
"sFirst": "|<",
"sPrevious": "<",
"sNext": ">",
"sLast": ">|"
}
}
}
);
new FixedHeader( oTable );
} );
//]]>
<< other Lists
List
Item 1
Item 2
Item 3
Item 4
Item 5
Item 6
Item 7
Item 8
Item 1
Item 2
Item 3
Item 4
Item 5
Item 6
Item 7
Item 8
00001
00001 Item 2
00001 Item 3
00001 Item 4
00001 Item 5
00001 Item 6
00001 Item 7
ID0001
[...]
07200
07200 Item 2
07200 Item 3
07200 Item 4
07200 Item 5
07200 Item 6
07200 Item 7
ID7200
[/code]
Help to point me into the right direction would be much appreciated.
I tried to get some more info abot ajax and arrays.txt, and found this:
http://datatables.net/release-datatables/examples/data_sources/ajax.html
http://datatables.net/release-datatables/examples/ajax/sources/arrays.txt
It would be nice to add some example lines of the content of array.txt. Though, I'm not a programmer and therefor got slight problems to understand the concepts behind datatables...
I guess now I got a starting point.
As of my understanding I now have to build the array in array.txt myself, isn't it?
So that makes it necessary to change the script that parsed my .csv table and build the html table that way,. that it build array.txt according the example content above, right? That would be a lot of work I guess ... Or should it create the array.txt automatically from the html table?
http://www.cparker15.com/utilities/csv-to-json/
http://area23.brightbyte.de/csv2wp.php
plus, you can download it and use the code on your own PHP site (if you're running PHP)
I need that data in that array prior to use "bDeferRender": true, anyway if I get ist as a text file (arry.txt) or by POST (server processing), right? So It's not created by the javascipt in the first place while loading the page the first time?
As I understand till now is, that I need an array like in examples/ajax/sources/arrays.txt
[code]
{ "aaData": [
["Trident","Internet Explorer 4.0","Win 95+","4","X"],
["Trident","Internet Explorer 5.0","Win 95+","5","C"],
[...]
["Other browsers","All others","-","-","U"]
] }
[/code]
or transfere that aaData (object?) per POST and "bServerSide": true, through php or other serverside scripting and e.g. usage of an database e.g. MySQL. In my case I don't need serverside processsing for now, because everything is done locally by accessing the file without webserver, directly from disk.
Hm, I did already some processing on the *.cvs to get the html exactly with that infos I need (that means I parsed that cvs and added urls that fit to that data and so on, by parsing with a bash script using sed, awk and regular expressions). So It might be easier for me to have something that turns my html back into *.cvs :(
Or is there a good html parser that could turn my html into that array format? And how to handle those URLs in that array?
Should it look this way? (already urlencoded?)
[code]
{ "aaData": [
["00001",00001 Item 2","00001 Item 3","00001 Item 3","00001 Item 4","00001 Item 5","00001 Item 6","00001 Item 7","ID00001"],
[...]
["07200",07200 Item 2","07200 Item 3","07200 Item 3","07200 Item 4","07200 Item 5","07200 Item 6","07200 Item 7","ID07200"]
] }
[/code]
you didn't keep a copy of the original csv?
Ok, It should be quite easy to get the bash script producing the array.txt rather that list.html with those Data HTML table.
I now put the last Datablock into the array.txt (without that [...]), and get now:
[quote]
DataTables warning: JSON data from server could not be parsed. This is caused by a JSON formatting error.[/quote]
and
[quote]
DataTables warning (table id = 'example'): Cannot reinitialise DataTable.
To retrieve the DataTables object for this table, please pass either no arguments to the dataTable() function, or set bRetrieve to true. Alternatively, to destory the old table and create a new one, set bDestroy to true (note that a lot of changes to the configuration can be made through the API which is usually much faster).
[/quote]
That way get filled by javascript and array.txt?
[code]
Live example
Rendering engine
Browser
Platform(s)
Engine version
CSS grade
Rendering engine
Browser
Platform(s)
Engine version
CSS grade
[/code]
(See HTML-Table I posted earlier)
firebug > console > JSON shows this red marked:
[code]
GET http://192.168.150.11/list/_html/sources/arrays.txt?_=1313765614362
200 OK
63ms
ParameterHeaderAntwortJSON
Nach Schlüssel sortieren
aaData
[["00001", "00001 Item 2", "00001 Item 3", 6 mehr...], ["07200", "07200 Item 2", "07200 Item 3", 6 mehr...]]
0
["00001", "00001 Item 2", "00001 Item 3", 6 mehr...]
0
"00001"
1
"00001 Item 2"
2
"00001 Item 3"
3
"00001 Item 3"
4
"00001 Item 4"
5
"00001 Item 5"
6
"00001 Item 6"
7
"00001 Item 7"
8
"ID00001"
1
["07200", "07200 Item 2", "07200 Item 3", 6 mehr...]
0
"07200"
1
"07200 Item 2"
2
"07200 Item 3"
3
"07200 Item 3"
4
"07200 Item 4"
5
"07200 Item 5"
6
"07200 Item 6"
7
"07200 Item 7"
8
"ID07200"
[/code]
http://stackoverflow.com/questions/993970/what-do-i-need-to-escape-in-my-html-json-response
http://jsonlint.com/
Ok, I'll have to look later into that issue again. I ran out of time...
[code]
{ "aaData": [
["00001<\/a>","00001 Item 2<\/a>","00001 Item 3<\/a>","00001 Item 3<\/a>","00001 Item 4<\/a>","00001 Item 5","00001 Item 6","00001 Item 7","ID00001"],
["07200<\/a>","07200 Item 2<\/a>","07200 Item 3<\/a>","07200 Item 3<\/a>","07200 Item 4<\/a>","07200 Item 5","07200 Item 6","07200 Item 7","ID07200"]
] }
[/code]
So I have to create the whole array that way? I'll see if I can rewrite the bash script that parses the cvs in order to get the proper array.txt.
The other attempt might be to parse the HTML Table to get the proper JSON array. We'll see...
I have some code that might convert your table to JSON array for you (part of a project I'm working on - see http://tote-magote.blogspot.com/2011/08/datatables-work.html )
Can't guarantee it will work, but you can paste your table into MCE (or click HTML button and input the source), then click the J button to convert to JSON
http://www.beg.utexas.edu/_work/dt/VisualDT/tinymce/examples/table_only.html
[quote]Only posts from tinymce.moxiecode.com are accepted.[/quote]
Though I don't know how to use your application. I don't know what to do with:
[quote][Show] [Hide] [Bold] [Get contents] [Get selected HTML] [Get selected text] [Get selected element] [Insert HTML] [Replace selection] [/quote]
EDIT: I now noticed that I missed the 'J'-button to convert the table into JSON.
Obviously It didn't worked for the whole 10MiB table, but I could parse 10.000 Lines of table data (892 Table rows). So I'll do it step by step ;)
Ooops, updated IE8 and windows updates on that new installed winxp vm and now it takes only 15 secs to render the table. That's awesome! Ok under ff and so on (also IE9) it takes under one second.
fbas, thank you again! Also thanks to the datatable team! You rock!
One question that's not directly related to datatables: How can one disable the script delay warning of IE8, that asks to stop the script?
Yes, I already thought, that a serverside processing would help here, but I don't have a good starting point how to achieve that. Will e.g. the apache processing the JSON by javascript (Though, I can't think of it)? Or do one need a completely other approach wit any other serverside scripting like php, python or perl?
Can you point me to some enlighten docs?
I found this:
http://www.codeunit.co.za/2011/02/16/jquery-datatables-with-lots-of-columns-crashes-in-ie/
I'm more concerned about the content of server_processing_post.php and how to achieve to get the JSON data per php from a mysql database, and how to structure that database, because I'm not very experienced in php/mysql :(
I guess this got to do something with that approach:
http://code.google.com/p/datatables-serverside-php-oo/
http://www.sprymedia.co.uk/dataTables/example_add_row.html
http://www.sprymedia.co.uk/dataTables/example_dynamic_creation.html
http://www.datatables.net/release-datatables/examples/data_sources/js_array.html
What docs do I need to understand the basics behind those links and get that serverside thing working?
And would it be for sure faster then processing clientside? How will that data be transfered and rendered? step by step and therefor faster? And if so how can I search through the whole table then? Is that done by serverside processing too (sql)? As you can see I'm not very experienced in php/mysql, though I installed some cms and also a website with some basic php...
the client side only receives a small window of data at a time - one page (number of rows equal to iDisplayLength, can be changed with the Length dropdown list at runtime if you want).
also the sort and filter functionality is run on the database, which is a finely tuned piece of software for these functions.
---- how it works
the mechanics of the server_side implementation are this:
when a user runs a filter, sorts a column, or even at init time when the first data is show, DataTables fires off fnServerData() which assembles a querystring to send to the server that includes the field names (database column names, in order), any global sSearch value, any column-specific sSearch values, sort info, etc.
here's an example I pulled from my javascript debugger of a real server AJAX call. this will look intimidating, but it's all really orderly and Allan posts a server-side script to use that handles this all for you on the server side:
see: http://datatables.net/release-datatables/examples/server_side/server_side.html
[code]
//__my_server__/qa/qafiles_json.php?sEcho=2&iColumns=10&sColumns=fileid%2Cfileid%2Cdir%2Cbasename%2Cfext%2Cfirstfour%2Csize%2Clastmod%2Cqaid%2Cthumb&iDisplayStart=0&iDisplayLength=10&mDataProp_0=0&mDataProp_1=1&mDataProp_2=2&mDataProp_3=3&mDataProp_4=4&mDataProp_5=5&mDataProp_6=6&mDataProp_7=7&mDataProp_8=8&mDataProp_9=9&sSearch=&bRegex=false&sSearch_0=&bRegex_0=false&bSearchable_0=true&sSearch_1=&bRegex_1=false&bSearchable_1=false&sSearch_2=&bRegex_2=false&bSearchable_2=true&sSearch_3=&bRegex_3=false&bSearchable_3=true&sSearch_4=&bRegex_4=false&bSearchable_4=true&sSearch_5=&bRegex_5=false&bSearchable_5=true&sSearch_6=&bRegex_6=false&bSearchable_6=true&sSearch_7=&bRegex_7=false&bSearchable_7=true&sSearch_8=&bRegex_8=false&bSearchable_8=true&sSearch_9=&bRegex_9=false&bSearchable_9=true&iSortingCols=3&iSortCol_0=2&sSortDir_0=desc&iSortCol_1=3&sSortDir_1=asc&iSortCol_2=4&sSortDir_2=asc&bSortable_0=true&bSortable_1=false&bSortable_2=true&bSortable_3=true&bSortable_4=true&bSortable_5=true&bSortable_6=true&bSortable_7=true&bSortable_8=true&bSortable_9=true&kw=false
[/code]
here are the querystring values broken up by line (thanks to my debugger, no work on my part. it's all shown and copyable easily in chrome)
for a summary of these fields, see http://datatables.net/usage/server-side
[code]
sEcho:2
iColumns:10
sColumns:fileid,fileid,dir,basename,fext,firstfour,size,lastmod,qaid,thumb
iDisplayStart:0
iDisplayLength:10
mDataProp_0:0
mDataProp_1:1
mDataProp_2:2
mDataProp_3:3
mDataProp_4:4
mDataProp_5:5
mDataProp_6:6
mDataProp_7:7
mDataProp_8:8
mDataProp_9:9
sSearch:
bRegex:false
sSearch_0:
bRegex_0:false
bSearchable_0:true
sSearch_1:
bRegex_1:false
bSearchable_1:false
sSearch_2:
bRegex_2:false
bSearchable_2:true
sSearch_3:
bRegex_3:false
bSearchable_3:true
sSearch_4:
bRegex_4:false
bSearchable_4:true
sSearch_5:
bRegex_5:false
bSearchable_5:true
sSearch_6:
bRegex_6:false
bSearchable_6:true
sSearch_7:
bRegex_7:false
bSearchable_7:true
sSearch_8:
bRegex_8:false
bSearchable_8:true
sSearch_9:
bRegex_9:false
bSearchable_9:true
iSortingCols:3
iSortCol_0:2
sSortDir_0:desc
iSortCol_1:3
sSortDir_1:asc
iSortCol_2:4
sSortDir_2:asc
bSortable_0:true
bSortable_1:false
bSortable_2:true
bSortable_3:true
bSortable_4:true
bSortable_5:true
bSortable_6:true
bSortable_7:true
bSortable_8:true
bSortable_9:true
kw:false[/code]
these key/value pairs are generated automatically based on your DataTable columns. you can add custom key/value items to the AJAX call, though, as I did with the "kw" parameter above.
the server_processing script will then read these parameters and compose the selection query from the database, limiting the number of rows to return by iDisplayLength (and starting at iDisplayStart).
I'll continue in the next post
this is a huge part of the performance benefit of using server_side processing. Javascript, being interpreted on the client side, is not a very efficient language compared to hard-compiled languages like C, C++, etc, that MySQL is written in. And the search, sort, limit (paging) routines in MySQL have been finely tuned, tested and retested, and are vastly better than anything Javascript would be able to do, especially with larger datasets (search, sort routines scale super-linearly -- adding one new item makes it take longer than just one unit of time).
Allan's server processing script will take care of all the default parameters in the fnServerData call and compose the SQL for you, will return the data from db into arrays, will pack it in JSON for you and return it to the client program. You don't have to concern yourself with this.
-------------- setting up your db
All that remains for you, if you want to move to a server-side architecture, is to place your data into a database and initialize your DataTables with the column names to match your database column names.
If you have MySQL and PHP, chances are you have PHPMyAdmin installed on your webserver, which gives you a web interface into MySQL and makes creating a db and importing data easy. MySQL can import from CSV files, too.
You DO have to set up:
1) the database (a database, at this point, is just a named container where you will place tables. I'm pretty sure in MySQL a "database" is just a folder in the file system that holds table files)
2) a table - the table will have fields/columns for your data (base it on your CSV file), using column names, and each having a data type associated with it (in the simplest cases you will probably use just INT and VARCHAR (string) types and never worry about a thing)
3) optionally create indexes for some columns (indexing adds a tiny need for storage in your db, but enhances performance by pre-sorting a given column (or set of columns), usually into a binary tree)
Then import your data (CSV). If you're importing from CSV, make sure your database columns are set up in the same order as your CSV file.
These are actually pretty simple steps to undertake, and I can help you further with them if you want to go down this route.