Adding callback / holding current pagination to $.fn.dataTable.pipeline script

tdmohr

I'm using the $.fn.dataTable.pipeline script (https://datatables.net/examples/server_side/pipeline.html) and am really missing the ability to add a callback and hold the pagination position on table reload. Ideally we could add similar functionality to the script as we can when using the plain ajax reload method (https://datatables.net/reference/api/ajax.reload()).

Is there any possibility we could shoehorn this functionality into the dataTable pipeline workflow?

Tim

kthorngren

Both the callback and staying on the same page seem to work here with the pipelining script:
https://live.datatables.net/dijuqeti/1/edit

Maybe I'm misunderstanding the question. Can you update the test case to show the issue you are trying to solve?

Kevin

tdmohr

Hi Kevin,

Thank you very much for your reply and thanks for making the test case.

I've been using the clearPipeline function, which appears to be required to draw changes made to the data into the table.

sspDataTable.clearPipeline(function(json) {
    console.log('json', json);
}).draw();

I just tested with ajax.reload and it does appear that whilst the callback fires, and pagination is maintained, the table is not redrawn with the changed data.

I'm not sure if its possible to modify the test script to show this as the data needs to be changed.

I've been working on modifying the pipeline script to allow this, and will happily share it once I'm confident its actually working correctly.

Thanks,
Tim

kthorngren

I'm not sure if its possible to modify the test script to show this as the data needs to be changed.

Unfortunately not.

I just tested with ajax.reload and it does appear that whilst the callback fires, and pagination is maintained, the table is not redrawn with the changed data.

Do the changes show if you reload the page? Use the browser's network inspector to verify the XHR request and what is in the response.

I've been using the clearPipeline function, which appears to be required to draw changes made to the data into the table.

That is using draw() which will fetch the data from the server. Shouldn't be any different that ajax.reload(). However in the place of ajax.reload() you could use the draw() API passing in either page or false, depending on your needs, to refresh the data from the server and stay on the same page.

Kevin

tdmohr

Do the changes show if you reload the page? Use the browser's network inspector to verify the XHR request and what is in the response.

Yes. The table is updated after page reload.

To clarify, when I pass false into the draw function, pagination is maintained, but no changes are shown until page reload.

sspDataTable.clearPipeline().draw(false);

kthorngren

Ok. Did you check the XHR response? If the changes are in the response then we will need a test case showing the issue to help debug. If the changes aren't in the response then the server script will need debugging.

Kevin

tdmohr

Hi Kevin,

If the changes are in the response then we will need a test case showing the issue to help debug. If the changes aren't in the response then the server script will need debugging.

I can do one better... below is the pipeline script modified to allow AJAX callback and redraws the data without affecting pagination:

//
// Pipelining function for DataTables. To be used to the `ajax` option of DataTables
//
$.fn.dataTable.pipeline = function ( opts ) {
    // Configuration options
    var conf = $.extend( {
        pages: 5,     // number of pages to cache
        url: '',      // script url
        data: null,   // function or object with parameters to send to the server
                      // matching how `ajax.data` works in DataTables
        method: 'GET' // Ajax HTTP method
    }, opts );

    // Private variables for storing the cache
    var cacheLower = -1;
    var cacheUpper = null;
    var cacheLastRequest = null;
    var cacheLastJson = null;

    return function ( request, drawCallback, settings ) {
        var ajax          = false;
        var requestStart  = request.start;
        var drawStart     = request.start;
        var requestLength = request.length;
        var requestEnd    = requestStart + requestLength;
        var successCallback = settings.successCallback;

        if (settings.resetPaging === undefined) {
            settings.resetPaging = true; // Initialize resetPaging value if not set
        }

        if ( settings.clearCache ) {
            // API requested that the cache be cleared
            ajax = true;
            settings.clearCache = false;
        }
        else if ( cacheLower < 0 || requestStart < cacheLower || requestEnd > cacheUpper ) {
            // outside cached data - need to make a request
            ajax = true;
        }
        else if ( JSON.stringify( request.order )   !== JSON.stringify( cacheLastRequest.order ) ||
            JSON.stringify( request.columns ) !== JSON.stringify( cacheLastRequest.columns ) ||
            JSON.stringify( request.search )  !== JSON.stringify( cacheLastRequest.search )
        ) {
            // properties changed (ordering, columns, searching)
            ajax = true;
        }

        // Store the request for checking next time around
        cacheLastRequest = $.extend( true, {}, request );

        if ( ajax ) {
            // Need data from the server
            if ( requestStart < cacheLower ) {
                requestStart = requestStart - (requestLength*(conf.pages-1));

                if ( requestStart < 0 ) {
                    requestStart = 0;
                }
            }

            cacheLower = requestStart;
            cacheUpper = requestStart + (requestLength * conf.pages);

            request.start = requestStart;
            request.length = requestLength*conf.pages;

            // Provide the same `data` options as DataTables.
            if ( typeof conf.data === 'function' ) {
                // As a function it is executed with the data object as an arg
                // for manipulation. If an object is returned, it is used as the
                // data object to submit
                var d = conf.data( request );
                if ( d ) {
                    $.extend( request, d );
                }
            }
            else if ( $.isPlainObject( conf.data ) ) {
                // As an object, the data given extends the default
                $.extend( request, conf.data );
            }

            return $.ajax( {
                "type":     conf.method,
                "url":      conf.url,
                "data":     request,
                "dataType": "json",
                "cache":    false,
                "success":  function ( json ) {
                    cacheLastJson = $.extend(true, {}, json);

                    if (!settings.resetPaging) {
                        // Maintain the current page data
                        var currentPageDataStart = requestStart % (requestLength * conf.pages);
                        var currentPageDataEnd = currentPageDataStart + requestLength;
                        json.data = json.data.slice(currentPageDataStart, currentPageDataEnd);
                    } else {
                        if (cacheLower != drawStart) {
                            json.data.splice(0, drawStart - cacheLower);
                        }
                        if (requestLength >= -1) {
                            json.data.splice(requestLength, json.data.length);
                        }
                    }

                    drawCallback( json );

                    if (typeof successCallback === 'function') { // Check if the successCallback is a function
                        successCallback(json); // Call the successCallback with the JSON response
                    }
                }
            } );
        }
        else {
            json = $.extend( true, {}, cacheLastJson );
            json.draw = request.draw; // Update the echo for each response
            json.data.splice( 0, requestStart-cacheLower );
            json.data.splice( requestLength, json.data.length );

            drawCallback(json);
        }
    }
};

// Register an API method that will empty the pipelined data, forcing an Ajax
// fetch on the next draw (i.e. `table.clearPipeline().draw()`)
$.fn.dataTable.Api.register( 'clearPipeline()', function (successCallback, resetPaging = true) {
    return this.iterator( 'table', function ( settings ) {
        settings.clearCache = true;
        settings.successCallback = successCallback; // Add successCallback to the settings object
        settings.resetPaging = resetPaging; // Add resetPaging to the settings object
    } );
} );

Table reload is triggered like:

sspDataTable.clearPipeline(function(json) {
    console.log(json);
}, false).draw(false); // Retaining pagination needs to be passed in as false to the clearPipeline() and draw() functions

Hope this helps someone in the future.

Kind regards,
Tim

colin

@tdmohr Nice, thanks for posting,

Colin