large results & item templates killing performance

getalex

I am rendering a datatable with development data with zero problems, but in production we are quickly seeing problems and i'm wondering if it's due to large datasets & the use of templates ...

var dt = $('#itemsTable').DataTable({    
    columns: [
        { data: null, defaultContent: '', orderable: false, className: 'details-control' },
        { data: 'Id', title: 'ID' },
        { ... about 5 other columns then a rendered column with actions ... },
        {
            data: null, orderable: false, searchable: false, title: 'Action',
            render: function (data, type, row) {
                var actionsTemplate = '<div style="text-align:center">' + (row.Status() != 'Pending' ? '' :
                    '<i data-toggle="modal" data-target="#modal-confirm-verify" data-itemid="' + row.Id + '" data-itemtitle="' + row.DocumentTitle() + '" class="glyphicon glyphicon-ok"></i>'
                    + '<i data-toggle="modal" data-target="#modal-confirm-rejection" data-itemid="' + row.Id + '" data-itemtitle="' + row.DocumentTitle() + '" class="glyphicon glyphicon-remove"></i>')
                    + '</div>';
                return actionsTemplate;
            }
        }
    ],
    order: [1, 'asc']
});

Then I also have implemented the click event to expand and show collapsible details per row:

function renderRowDetails(d) { 
   // simple implementation as documented in examples
}

The typical table ends up with 500 to 1000 items, and takes forever to load ... I am thinking about 'defer render' options and was exploring this but then my understanding is that search would not work against page 2, 3, 4, etc ...

Am I doing something wrong here? i wonder if this is an issue with my underlying dataframework using KnockoutJS or broader issues as in expected performance with large results as I mention.

kthorngren

Where is the data coming from?

You config snippet doesn't show ajax or data so I assume you are loading it directly into HTML then Datatables is reading the data from there when it starts. If this is the case then deferRender won't help as it either needs an ajax or Javascript data source.

You can use your browser's developer tools to see how long it takes to fetch the data.

I would try the following to get an idea of how long Datatables is taking to load:

var startTime = Date.now();  //Get initial time

var dt = $('#itemsTable').DataTable({   
    columns: [
        { data: null, defaultContent: '', orderable: false, className: 'details-control' },
        { data: 'Id', title: 'ID' },
        { ... about 5 other columns then a rendered column with actions ... },
        {
            data: null, orderable: false, searchable: false, title: 'Action',
            render: function (data, type, row) {
                var actionsTemplate = '<div style="text-align:center">' + (row.Status() != 'Pending' ? '' :
                    '<i data-toggle="modal" data-target="#modal-confirm-verify" data-itemid="' + row.Id + '" data-itemtitle="' + row.DocumentTitle() + '" class="glyphicon glyphicon-ok"></i>'
                    + '<i data-toggle="modal" data-target="#modal-confirm-rejection" data-itemid="' + row.Id + '" data-itemtitle="' + row.DocumentTitle() + '" class="glyphicon glyphicon-remove"></i>')
                    + '</div>';
                return actionsTemplate;
            }
        }
    ],
    order: [1, 'asc'],
      initComplete: function() {
        //output time difference once DT init is complete
        console.log('DT init complete in ', Date.now() - startTime + ' milliseconds.');
      }
});

Let us know what you find.

Kevin

getalex

Kevin - thanks for the suggestion, I'm loading the datatable through ajax but in a later call when the user clicks a button, executing a function (see below)

the initComplete is a good idea but in my case the table is initially empty, so the console log reporting shows a few milliseconds, it's not clear how to capture the true load time since it's event triggered.

function loadTableData() {

    var params = { ... };

    $.ajax({
        type: "GET",
        data: params,
        contentType: "application/json; charset=utf-8",
        url: "/Path/GetTableData",
        success: function (items) {

            // clear all former datatable contents
            viewModel.dataTableItems.removeAll();

            // Convert the data set into observable objects, and will also add the initial data to the table
            ko.mapping.fromJS(
                items,
                {
                    key: function (data) {
                        return ko.utils.unwrapObservable(data.Id);
                    },
                    create: function (options) {
                        return new Item(options.data, dt);
                    }
                },
                viewModel.dataTableItems
            );
        }
    });
}

kthorngren

I'm not familiar with your framework but my guess is ko.mapping.fromJS() adds the data to the table then viewModel.dataTableItems initializes the updated table. I would calculate the time it takes the loop to execute and calculate how long it takes viewModel.dataTableItems to execute. Something like this maybe?

            var startTime = Date.now();

            ko.mapping.fromJS(
                items,
                {
                    key: function (data) {
                        return ko.utils.unwrapObservable(data.Id);
                    },
                    create: function (options) {
                        return new Item(options.data, dt);
                    }
                },

                console.log(' ko.mapping.fromJS() complete in ', Date.now() - startTime + ' milliseconds.');

                startTime = Date.now();
                viewModel.dataTableItems
                console.log('DT init complete in ', Date.now() - startTime + ' milliseconds.');

Also did you take a look at your Dev Tools > Network to see how long it takes to fetch "/Path/GetTableData"?

Kevin

getalex

Kevin thank you very much for the interaction here - I moved the datetime calculation to the success callback on my ajax call and found what I suspected, a long load time:

Loading just 460 result records took : 136694 milliseconds - that's almost 2 minutes 30 seconds!

The ajax call itself to get the results isn't slow to respond, adding a date comparison before the mapping function is approximately 600ms and then it goes into that long load time populating the observableArray, using KnockoutJS here...

getalex

I'm using the official knockout mapping plugin, and following the KnockoutJS example found here on DataTables integration samples ... for small test results it was working very well, but now that I'm hitting hundreds, the performance issues are surfacing.

I need to be able to handle row edits, so have stored the results in an observable array, and then using a binding pattern to clear the underlying array, call the ajax results and map the results of new objects in the array to listen for edits, individually send updates as needed, until the next load is called with new parameters which result in different records altogether to view/edit. just a bit of background for implementation.

I'm wondering if moving the ajax call to the datatable initialization and adding deferRender would solve the mapping performance issue, I used to think so but not sure at this point.