Confused about deferRender with ajax sourced data

arkfreestyle

Hi,

I'm trying to improve my datatable's loading times (currently 6-7 seconds). I'm using ajax sourced data, my backend is django and djangorestframework-datatables. My endpoint (/api/device_input/) returns 2500 records. Here's my initialization:

$(document).ready(function() {
        $('#example').DataTable( {
            ajax: {
                url: "/api/device_input/?format=datatables",
            },
            columns: [
                {data: 'Device_ID'},
                {data: 'Device_ID_clone'},
                {data: 'Api_key'},
                {data: 'DateAdded'},
                {data: 'humidity'},
                {data: 'heat_index'},
                {data: 'soil_moisture'},
                {data: 'water_flow_avg'},
                {data: 'quality_of_water'},
                {data: 'rain_sensor'},
                {data: 'wind_speed'},
                {data: 'amb_temp'},
                {data: 'wind_direction'},
                {data: 'temperature'},
            ],
            deferRender: true,
            // serverSide: true,
        } );
    } );

I have verified the response from my api is correct. When I toggle deferRender option I'm expecting datatables to load only 10 records to display for the current page but according to chrome's network tab it still gets all 2500 records first and then displays the table (after 6-7 seconds).

If I just enable serverSide in this case, my datatable loading time reduces to milliseconds and it loads 10 records for each page draw. That's exactly what I want, but I want to do it with client side processing if that's possible? I think I may be misunderstanding deferRender and some other things about client side/server side pagination. Are those different?

Is it possible to load my datatable fast without serverSide enabled? Because according to your documentation's recommendations, my datasets are mostly under 50k records, so I wanted to keep things client side. Please help me understand what's happening here! Thank you so much for the awesome plugin. I'm really loving it so far

kthorngren

Server side processing expects the server to return just the rows for the page being displayed. If the page length is set to 10 for example then Datatables sends a request to the server for 10 rows of data (other parameters are sent as well). It is expected that the server respond with only 10 rows. The server side process is explained here:
https://datatables.net/manual/server-side

The deferRender option is not a server side processing option. The behavior is correct that you get all the rows when using deferRender. Datatables takes the ajax response and builds a data cache which it uses for sorting, searching, etc.
Datatables also takes the ajax response to build the HTML table. deferRender controls the number of rows Datatables creates during the table build. This is described in the deferRender docs:

As an example to help illustrate this, if you load a data set with 10,000 rows, but a paging display length of only 10 records, rather than create all 10,000 rows, when deferred rendering is enabled, DataTables will create only 10.

You can't have a mix of server side and client side processing.

2500 records doesn't seem like a lot and you aren't doing anything fancy with the data. You are just displaying what you get. I wouldn't expect to take too long to process and build the table. You will need to determine where the delay is at. I would start by looking at the browser's developer tools to monitor the ajax request and response. You can find instructions in this technote.

You you will be able to watch live the request, the time it takes for the response and once the client has the response how long it takes Datatables to build the table. I suspect that if you don't see any difference between deferRender on and off that the delay is with the server querying for the data. Possibly a network delay.

Please post if you have any questions.

Kevin

arkfreestyle

@kthorngren Hey, thank you for answering. I understand that we can't have server side and client side processing at the same time. As you suggested, I have used chrome's network tab to monitor my ajax requests for different lengths of data, they are definitely what's taking the most time.

When server side processing is enabled, the server only returns records = page length, this is great and minimizes the loading time significantly (From 10 seconds to 250 ms).

What I want to ask now is, is it possible to load records = page length only with client side processing? For example if my ajax request is made for 2500 records, is it possible to delay retrieval of those records and display the table first when we have enough for the first page? You may be correct in suspecting a network delay, but let me try to elaborate my question, currently:

1) Ajax gets 2500 records of data. (This step blocks datatable from loading)
2) Once step 1 is completed (in 10 seconds for example), THEN the datatable is displayed.

Is it possible to do it like this with client side processing:

1) Ajax gets 2500 records of data. As soon as it has gotten records = page length, it first displays the datatable.
2) The remaining data is then loaded in the background.

My main issue is with the delay it takes to display the datatable, that's what I'm trying to fix. If I can do that somehow with client side processing, that'd be amazing! Otherwise, I'll stick to server side processing.

kthorngren

There is no builtin option for this. You would need to build the code for this. One option may be to use ajax to fetch the original set of data, one page, 100 records, whatever. Then in the initComplete function you could call a function where you fetch the remaining rows via jQuery ajax then in the success function use rows.add() to add the remaining rows.

You will need to make sure the ordering of the queries matches that of Datatables so that when the second set of data is fetched it doesn't display records on the first page. Otherwise it might be confusing for the users.

Here is a simple example of using jQuery ajax to add rows.
http://live.datatables.net/hopepicu/72/edit

Kevin

arkfreestyle

@kthorngren Thank you so much for your help Kevin! That answers my question
I'll return with another question if I get stuck again.
Take care.