Loading speed: AJAX vs DOM source

klippdassie

Hi,

I hope you can excuse a newbie for asking a question, without providing a proper test case at this point .

I have a python/flask app requesting data from an external API (postgres/postgREST) and displaying that as a dataTable. Originally, I simply passed data in a html table to dataTables (i.e. HTML document [DOM]-source), which worked fine.

As I will, eventually, have (many) more than 5000 rows in the table, though, I changed to passing a JSON object to dataTables, instead, which also seemed to work fine, although I noticed the following console warnings:

[Violation] 'setTimeout' handler took xx ms
[Violation] Forced reflow while executing JavaScript took xx ms

Suspecting that passing data as JSON via variable in HTML is just as slow as using a DOM-source, I then changed to making an AJAX request to a Flask endpoint, instead, like so:

function makeDataTbl(table_id, columns) {
    $.fn.dataTable.ext.errMode = 'none';
    // BLAST
    if ( table_id === 'blast_result_table' ) {
        var detNo = 3; var ordNo = 2; var url = '/blast_run';
    }
    // API SEARCH
    else {
        var detNo = 2; var ordNo = 2; var url = '/search_run';
    }
    var dTbl = $('#'+table_id)
        .on('error.dt', function (e, settings, techNote, message) {
            console.log( 'An error has been reported by DataTables: ', message );
            $('#flash_container').html('Sorry, the query was not successful. Please, contact support if this error persists.');
        })
        .DataTable({
        deferRender: true, // Process one page at a time
        autoWidth : false, // Respect CSS settings
        ajax: {
            url: url,
            type: 'POST',
            data: function () {
                return $("#sform").serialize(); } // Includes CSRF-token
        },
        columns : columns,
        columnDefs: [
            { targets: 0, orderable: false, defaultContent: '', className: 'select-checkbox' },
            { targets: detNo, className: 'details-control' }, // Seq expansion
            { targets:[1,7], visible: false }, // Hidden ID & seq
        ],
        select: { style: 'multi', selector: 'td:nth-child(1)' }, // Checkbox selection
        order: [[ ordNo, 'asc' ]],
        // Layout: l=Show.., f=Search, tr=table, i=Showing.., p=pagination
        dom: "<'row'<'col-md-4'l><'col-md-8'f>>" +
        "<'row'<'col-md-12't>>" +
        "<'row'<'col-md-3'B><'col-md-3'i><'col-md-6'p>>",
        buttons: [ 'excel', 'csv' ]
    });
    return dTbl;
}

This also works, but although the console errors are gone, loading actually appears slower than before (i.e. when I used either DOM or JSON object source). Now, there is a flash of the header and footer and a 'Loading' message before the actual data rows are shown on the screen, whereas before, the whole table appeared at once. Please note, that there is no flash of the unstyled tables (FOUC), but simply a delay between styled table 'frame' and the actual data rows are displayed. How can this happen at the same time as the 'setTimeout' warnings disappear? This seems to be independent of how many rows are returned (1-2500 ca in a small test version).

Thanks for your time. Any ideas would be most appreciated!

kthorngren

Now, there is a flash of the header and footer and a 'Loading' message before the actual data rows are shown on the screen

How can this happen at the same time as the 'setTimeout' warnings disappear?

This is the time waiting for the Ajax response. You can use the browser's network inspector to see how long it takes for the response. Ajax is an Asynchronous process so the web page will build and allow for interaction while waiting to the response. Once the response is received Datatables will populate the table.

whereas before, the whole table appeared at once.

I suspect that Flask is populating the template with the table then responding with the web page. I haven't used Flask this way but I suspect there is a longer delay in retrieving the initial web page when using Flask template for the table than when using Datatables to fetch via ajax. Is this correct?

Kevin

klippdassie

Many thanks, Kevin,

Not sure I follow exactly, but when my user clicks a search button, Flask now responds with the html ('/blast' page) containing the table header only (no data is sent this way). DataTables then starts working on styling and displaying, while the ajax POST request is sent to another Flask endpoint ('/blast_run') that returns the table data in the background/asynchronously, and meanwhile the 'Loading' message is shown.

Do you mean that, using DOM or JSON as rows source, the waiting would take place before anything (i.e. headers or data) gets sent to the client, and so I would not see any flash of the 'Loading' message, but perhaps the total waiting time was actually longer (will have a look at Network inspector, but sounds reasonable)?

But what event or lag does the 'setTimeout' measure, then? And is there a way to stop dataTables from displaying headers until all data have been received, to avoid the flicker? Or would you expect Loading time to increase a lot with very high row counts, so that it would be more useful to keep the message?

Maria

kthorngren

Not sure I follow exactly

I think you got it.

But what event or lag does the 'setTimeout' measure, then?

Its probably a page load issue but I'm not sure.

is there a way to stop dataTables from displaying headers until all data have been received, to avoid the flicker?

You can hide the table with CSS then in initComplete show the hidden table. See this example:
http://live.datatables.net/kafuweka/2/edit

Or would you expect Loading time to increase a lot with very high row counts, so that it would be more useful to keep the message?

Yes it will increase with more data. There is a server side process mode that allows for server based paging but you will need to find or write the server side code to handle this [protocol[(https://datatables.net/manual/server-side).

There are server side libraries I've seen (but not used) for Django. Not sure if there are any for Flask.

Kevin

klippdassie

Thanks, again, Kevin

You're right, I thought 'setTimeout' was a dataTables-specific msg because the console references 'datatables.min.js:14', but that is actually the CDN file for jQuery + datatables + ..., and so the warning probably refers to my whole $(document).ready(function().

Regarding server side, I try to avoid it for now, but will keep it in mind for later.

Regarding initComplete, how do I reference the dataTable inside the initialisation function, say if I wanted something like this:

var dTbl = $('#'+table_id)
        .on('error.dt', function (e, settings, techNote, message) {
            console.log( 'An error has been reported by DataTables: ', message );
            ...
        }) 
        .DataTable({
            preDrawCallback: function(){
                $( ??? container()).css('display', 'none');
            },
            initComplete: function(settings, json) {
                $( ??? container()).css('display', 'block');
            },
            ajax: { ...

/ Maria

klippdassie

Hi, again,

This seems to work:

        .DataTable({
        deferRender: true, // Process one page at a time
        autoWidth : false, // Respect CSS settings
        preDrawCallback: function(settings, json){
            table = settings.oInstance.api();
            $(table.table().container()).css('display', 'none');
        },
        initComplete: function(settings, json) {
            table = settings.oInstance.api();
            $(table.table().container()).css('display', 'block');
        },
        ajax: { ...

And, you were right about the total (page) loading time for JSON vs. AJAX-source. According to Network | Timing, JSON: 562 ms vs. AJAX: 12 ms. for the same table with ca. 2500 rows.

Thanks again for really useful input, Kevin.

/ Maria