Scroller Extra w/ server-side processing

CGRemakes

I've been using "infinite scrolling" for my program that can potentially return large data sets of 30,000+ using server side processing. Works great, other than I'd like a little better performance. From what I understand the Scroller extra is supposed to be a superior replacement for infinite scrolling. Is that true? On the demo, it loads a server-side dataset of 10,000 items very quickly, but I don't know how much processing goes into that ajax file for the demo. Mine will be dynamically generated, so there may be some extra overhead. If there is a perfomance boost with Scroller, I'll continue to look into that option. In my attempts to try it, I've been having a little trouble getting the program to work properly. I think I've disabled all the infinite scrolling option, so as to not have conflicts. I'm a little unclear if there are any changes that need to be made to my server-side script as far as limiting with Scroller vs infinite scrolling. What do I need to change when converting infinite scrolling to Scroller? The following is my datatable initialization (sorry, I have a lot of plugins and things):

[code]

oTable = $('#claimTable').dataTable({
"fnDrawCallback": function ( oSettings ) {

applyEditableText($('.editable_text'));
applyEditableSelect($('.editable_select'));
applyEditableDate($('.editable_date'));

if($('input[name="claim_id[]"]').length > 0)
{
$('.select_all').removeAttr('disabled');
$('.deselect_all').removeAttr('disabled');
$('.export').removeAttr('disabled');
}
else
{
$('.select_all').attr('disabled', 'disabled');
$('.deselect_all').attr('disabled', 'disabled');
$('.export').attr('disabled', 'disabled');
}

get_checked_count();

$('input[name="claim_id[]"]').click(function(){

get_checked_count();
});
},
"fnInitComplete": function(){

var oSettings = this.fnSettings();

$('tfoot input:visible').each(function(){

var index = $("tfoot input").index(this);
$(this).val(oSettings.aoPreSearchCols[index].sSearch);

if(this.value == "")
{
this.className = "search_init";
this.value = asInitVals[$("tfoot input").index(this)];
}
});

new FixedColumns(oTable);
},
"aoColumnDefs": [
{ "bSortable": false, "aTargets": [ 0 ] },
{ "sSortDataType": "dom-text", "aTargets": [ '_all' ] }
],
"bProcessing": false,
"bStateSave": false,
"bServerSide": true,
"sAjaxSource": "/members/get_claim_data.php",
/*"fnServerData": fnDataTablesPipeline,*/
"oScroller": {
"loadingIndicator": true
},
"sDom": 'rT<"clear">itS',
"oTableTools": {
"sSwfPath": "js/swf/copy_cvs_xls.swf",
"aButtons":[
{
"sExtends": "text",
"sButtonText": "Select All",
"fnClick": function(nButton, oConfig, nRow){

$('.checkbox').prop('checked', true);
get_checked_count();
},
"fnInit": function(nButton){

$(nButton).addClass('select_all');
}
},
{
"sExtends": "text",
"sButtonText": "Unselect All",
"fnClick": function(nButton, oConfig, nRow){

$('.checkbox').prop('checked', false);
get_checked_count();
},
"fnInit": function(nButton){

$(nButton).addClass('deselect_all');
}
},
{
"sExtends": "download",
"sButtonText": "Export",
"sUrl": "/members/get_claim_data.php?type=csv"
},
{
"sExtends": "text",
"sButtonText": "Duplicate Selected",
"fnClick": function(nButton, oConfig, nRow){

$('input:checked').each(function(){

row_copy_num = $(this).parent().find('.row_num').html();
var qty = prompt("How many times to copy row #" + row_copy_num + "?", "1");
qty = Number(qty);

if(qty > 0)
window.open('/members/copy_claim_rows.php?claim=' + $(this).val() + '&qty=' + qty, '_blank', 'width=800,height=400,resizable=1');
});
},
"fnInit": function(nButton){

$(nButton).addClass('copy');
}
},
{
"sExtends": "text",
"sButtonText": "Process Selected",
"fnClick": function(nButton, oConfig, nRow){

var claim_ids = $('input[name="claim_id[]"]:checked').serialize();
var row_nums = $('input[name="claim_id[]"]:checked').parent().find('input[name="row_num[]"]').serialize();

$('#error_section').html('');

$.ajax({
type: 'POST',
url: '/members/claims2.php',
data: { claim_ids: claim_ids, row_nums: row_nums },
dataType: "json",
success: function(data){

$('#error_section').append('

allan

> From what I understand the Scroller extra is supposed to be a superior replacement for infinite scrolling. Is that true?

Correct :-)

> On the demo, it loads a server-side dataset of 10,000 items very quickly, but I don't know how much processing goes into that ajax file for the demo.

Try using the Webkit Inspector tools - the Network tab for example will show much much time a server takes to respond.

> I'm a little unclear if there are any changes that need to be made to my server-side script as far as limiting with Scroller vs infinite scrolling.

In theory - no there should be no changes needed, other than to disable infinite scrolling and enable Scroller. Having said that, you are using FixedColumns which is not currently supported by Scroller (indeed, it isn't supported in infinite scrolling mode, but it might just work - I haven't actually tried that!).

Allan

TheSisb

So the FixedColumn patch by John Haltas ( https://github.com/DataTables/FixedColumns/commit/178749d1a8cf21e12c8b4b861249eb82f5216097 ) hasn't fixed the incompatibility between the two plugins?

I might have a dive into it myself soon, I need to fix both the left and right most columns of my table while using scroller...

CGRemakes

Not sure. This is a very trimmed down version of my program. It just returns the row # and 5 generic row, but other than that, it's pretty accurate to the actual program (it even includes processing time). The actual program is behind a login and contains data I'd rather not display, but shouldn't matter for this test:

http://www.mtncom.net/members/test.html

Is this how everything's supposed to perform? It just seems kinda flakey with how it displays the 1st column. Sometimes it removes it and refreshes properly, other times, it just leaves the column, then when it displays the data, it removes the first column.

TheSisb

If I drag your demo's scrollbar down far enough for it to start processing, when the processing is complete I lose the left most column until I scroll again.
I'm pretty sure this is the kind of stuff he meant by it not working correctly.

allan

> So the FixedColumn patch by John Haltas (...) hasn't fixed the incompatibility between the two plugins?

I'd forgotten about that (too much coding going on around here ;-) ). Yes, that looks like it probably does address the compatibility issue.

Personally I wouldn't use infinite scrolling in a project myself (with the sole exception of infinite data such as Twitter) - I'd always suggest Scroller over infinite scrolling. Indeed, I'm actively thinking of taking infinite scrolling out of DataTables. It will probably be deprecated in v1.10 and possibly removed in v1.11.

Allan

CGRemakes

Hmm.... that does seem much more like the expected behavior. However, I think infinite scrolling may work better for my needs. My concern is my program allows the user to click on a checkbox in the first column to select which rows they want to process. With inifinite scrolling, once that's checked, it will remain checked as people scroll/up down and view additional data. With scroller, it's constantly refreshing the data displayed, so if they scroll, it loses its check. Yes, I could probably check and uncheck boxes pretty easily programatically as they scroll, but I'm not sure I see enough benefit with Scroller over infinite scrolling to justify the additional work. Nice thing about infinite scrolling is once it's loaded, it's loaded. If I'm mistaken about any of this, I'm certainly open to suggestions. I'm curious if Scroller remembers the selected state built into Datatables as it loads in data?

allan

Infinite scrolling + FixedColumns is going to be a performance killer. Once you get past a certain number of rows, the performance of FixedColumns is terrible since it must assign height for every row (so the drop of in performance is linear). For that reason alone I'd suggest not using infinite scrolling. However, in addition, have you tried scrolling down the page, then returning to the top and checking a check box - then sort or filter the table. The checkbox check will be lost then as well due to the use of server-side processing (each data get must be got from the server and drawn - the client-side doesn't retain any data).

The real problem with infinite scrolling is that DataTabels doesn't really track the rows that have gone before - it appends more and more rows on each draw (basically a normal draw, but without doing a clear).

> With scroller, it's constantly refreshing the data displayed, so if they scroll, it loses its check

If you are using server-side processing, unfortunately yes that is the case. You would need to keep track of what is checked and recheck it on a draw callback. This is true of any DataTable which uses server-side processing.

Allan

CGRemakes

Good points. ;) I'll do some more playing and see what ultimately works better. Sounds like eventually Scroller will be the only option if I want to use the latest and greatest versions as they are updated.

CGRemakes

Ok, working on getting Scroller to work properly. I get some strange results at times. Up in the information section, it says "Showing 514 to 487 of 487 entries". How is that calculated? I looked at the json object I'm returning back to datatables, and it says "iTotalRecords": "487" and "iTotalDisplayRecords":"487", and there are definintely only 487 rows that it displays, cause I have them numbered. Any thoughts? Also, how it displays the data isn't very accurate (even on the demo). For example, on the API demo with scrolling to a certain line #, it says it's going to scroll to line # 1000, but it scrolls to line # 1002, and says it's showing 1,001 to 1,008.

allan

> Showing 514 to 487 of 487 entries". How is that calculated?

Very likely not all row heights in your table are the same height. Scroller makes the assumption that all rows are the same height (it can't know the height or rows which aren't draw).

> Also, how it displays the data isn't very accurate (even on the demo)

Even in this demo: http://datatables.net/extras/scroller/ ? Seems to work as expected for me. What browser / OS are you using?

Allan

CGRemakes

With setting a hard height for the rows, it does seem to be much more like expected. I was referring more to this demo: http://datatables.net/release-datatables/extras/Scroller/api_scrolling.html. It's isn't a "huge" deal, it just usually seems to be off by 1 row. You would think if you said to jump to row 1000, it would list the row starting with 1000. However, in the demo, it jumps to row 1001. I'm using Chrome 22.0.1229.94 and IE 9, and it's that way in both.