Ajax callback dataSrc confusion

skydiver

Hi, I need to perform pre/post ajax processing on a per-request basis and therefore need to use the "ajax: function(data, callback, settings)" variation.

So instead of defining:

new DataTable( {
    ...
    ajax: {
        url: ...
        dataSrc: function( response ) ...
    }
} );

I'd like it the options to be wrpped inside a custom function.
But in order for it to look as close to the simple object definition as possible, the callback will receive a single options object as well.

new DataTable({
    ...
    ajax: dtAjax({
        url: ...
        dataSrc: function( response ) ...
    })
});

Where dtAjax() would look something like:

var dtAjax = function( options ) {
    return function( data, dataTablesCallback, settings ) {
        options.data = $.extend( {}, options.data, data );

        // ... custom operations

        $.ajax( options )
            .done( function( response ) {
                // ... custom operations

                response = processResponse( options, response );

                dataTablesCallback( response );
            } )
            .fail( /* custom operations */ )
    };
}

In our case, options.dataSrc usually contains a callback, so my idea is to write processResponse() as follows:

var processResponse = function( options, response ) {
    if ( typeof options.dataSrc === 'function' ) {
        var processedData = options.dataSrc( response );
        if ( settings.sAjaxDataProp ) {
            response[ settings.sAjaxDataProp ] = processedData;
        }
        else {
            response = processedData;
        }

        return response;
    }
}

The above seems to be working.
However, I dislike like the fact that I'm messing with DataTable's internals.
The settings object may change, and there are probably other data bits that are not being taken care of in the above solution.

My question is therefore:
Is there a better/official way to perform DataTable's post-processing on the returned response?

Thank you.

allan

Hi,

If you are providing your own function to make the Ajax call, then currently that's probably as good as it gets since I didn't really expect this to be done externally.

You can use preXhr and xhr to pre and post process the data to and from the server. There are also jQuery options such as dataFilter which can be used.

What exactly do you need to do to the data?

Allan

skydiver

Hi Allan, thanks for replying.
I was worried that this would be the spirit of the reply.

The table's data is being fetched by an async process on a different machine so I need to defer the ajax promise's resolution until the async server will finish processing.

The returned data may also carry other instructions for the client and this is all done inside a module that wraps the $.ajax() request with its own set of promises and handlers. It is being use across the entire application for making all ajax calls.
That's why I need full control over DataTable's ajax call.

The events you refer to seem to be in the scope of the DOM, not in the scope of the ajax call. I don't think I can wire them as promise resolution handlers.

It seems like the "ajax: callback()" approach is the one I should use, but the callback is being invoked only after performing the ajax request so I have no control over the request itself.
IMO, a more inherent problem with this approach is that DataTables doesn't offer the client a way to deal with its proprietary attributes, such as dataSrc.

After going deeper through the code, I think a solution that will let the client gain control over the call being made, while still letting DataTables process the response the way it see fit, would be to let the client supply an alternative to "$.ajax", at least in the scope of _fnBuildAjax, e.g.:

new DataTables({
    ...
    ajax: ... normal options (e.g. an object with potential dataSrc)
    ajaxXHR: myAjax
    ...
});

Where "myAjax" would return a promise (that internally would manipulate the request, call $.ajax, potentially defer the call, process the response, etc., etc.), that will be transparent to DataTables, which will process the resolved/rejected promise as if it were a plain ajax promise.

... your thoughts?

skydiver

Hi @allan, I'd love to get your reply on the above.
Thanks.

allan

Hi,

Sorry I missed your reply before.

This point seems like the key one, but I don't quite understand I'm afraid:

It seems like the "ajax: callback()" approach is the one I should use, but the callback is being invoked only after performing the ajax request so I have no control over the request itself.

If you use ajax as a function, you are in complete control of the Ajax fetch to get more data. What I would actually suggest doing, is very similar to your dtAjax parameter above, but if you need control over where the data is, rather than attempting to use DataTables dataSrc option for the Ajax, define your own by passing in the parameter where it would be:

var dtAjax = function( prop, options ) {
    return function( data, dataTablesCallback, settings ) {
        options.data = $.extend( {}, options.data, data );
 
        // ... custom operations
 
        $.ajax( options )
            .done( function( response ) {
                // ... custom operations
 
                response = processResponse( options, response );
 
                dataTablesCallback( response[ prop ] );
            } )
            .fail( /* custom operations */ )
    };
}

Then your table initialisation would be something like:

ajax: dtAjax( 'myData', { ... } )

does that make sense and am I on the right track?

Allan

skydiver

Hi @allan,

Your suggestion is not on track because it's following my incomplete solution from the original post.
This is the manual way using some proprietary format of my own.
(*) Ultimately I'd like to let DataTable use it's native handling as much as possible.

I'll start by clarifying this one:

It seems like the "ajax: callback()" approach is the one I should use, but the callback is being invoked only after performing the ajax request so I have no control over the request itself.

What I meant was that by using ajax: function( data, callback, settings ), I am losing the ability to supply custom DataTable properties such as the dataSrc element.
dataSrc may contain a simple property-name string (e.g. "data"), but it may also be a function( response ) callback.

By using the ajax: function( data, callback, settings ) method, I am left to handle dataSrc on my own. Meaning: I'll need to check and handle all the cases (whether it's a simple property or a function, or an aaData B.C. case, etc) on my own.
If there-are / will-be any other special DataTable properties, then I am not even aware of them. Since DataTable's internal code is subject to change, I may also miss future modifications.

Now, my goal is to adhere to the the original ajax: { object } options as much as possible while having control over the way the $.ajax() call is being performed.

The way I see it, there's a solution that will let me benefit of both worlds: being able to pass the ajax options normally but also being able to override/extend DataTable's internal $.ajax() calls with my own custom implementation.

My suggestion is to expose a new property (e.g. the above suggest ajaxXHR) that will let the end user supply an alternative to $.ajax().

This way, I will be able to still pass an ajax: { object } with all the ajax + DataTables options exactly as they are being processed right now, and also have control over the way the $.ajax() call is being processed.

---

This is somewhat a complex subject and I'm not sure if I explained it properly, so I'll give a code example, which will hopefully clarify my point:

////////////////
// myCode.js:
////////////////
var myAjaxCustomOptions = {
        secret: "abcdefg",
        ...
    },

    myAjaxFunc = function( options ) {
        options = $extend( {}, options, myAjaxCustomOptions );
        return $.ajax( options )
                .fail( function( jqXHR, textStatus, errorThrown ) {
                    // Do some custom failure handling, which is common to all requests
                } );
    },

    table = new DataTable( {
        ...
        ajax: {
            url: ...
            dataSrc: function( json ) {
                // Some data manipulation here
            }
        },
        ajaxXHR: myAjaxFunc,
        ...
    } );


///////////////////////////
// jquery.dataTables.js:
///////////////////////////
    function _fnBuildAjax( oSettings, data, fn ) {
        var ajaxXHR = _fnAjaxXHROptions( oSettings ) || $.ajax; // Get a user supplied ajaxXHR or default to jQuery's $.ajax
        ...

        // Instead of
        // oSettings.jqXHR = $.ajax( $.extend( baseAjax ... ) );
        // The code will now use ajaxXHR:
        oSettings.jqXHR = ajaxXHR( $.extend( baseAjax ... ) );
    }

allan

Thank you - I understand now. One of my plans for the next major version of DataTables (2) is to abstract out the Ajax functionality so you could replace it with your own Ajax method (e.g. the fetch API). I think that might be useful for you here, although that is likely several months away from actual implementation I'm afraid.

That said, I don't want to have DataTables just then replicate jQuery's own Ajax functionality, and it is that reason why at the moment you do need to handle the JSON return on your own - I'm not yet sure if that will change or not. There shouldn't be any need to handle aaData or data like DataTables core does - you are in control of the server-side, so have your callback handler only deal with the data type that you will be sending (e.g. ignore aaData since you won't be sending that).

Allan

skydiver

OK, just to summarize, my goal is to be able to decorate the ajax request/response with my own embellishments in a way the will remain transparent to DataTables (letting DataTables do its thing as usual, e.g. dataSrc processing).
- This is impossible with the current API.

Adding an $.ajax alternative doesn't sound like major-version-overhaul-material to me, especially when the fetch API is not yet supported by many browsers and might prolong the next major version's release date. But I'm sure there are more burning issues to attend to and anyway - you're the boss

Thank you for the clarifications.
I'll mark the thread as "answered".

allan

Its going into the next version mostly so that the next version actually happens! To date I've just been adding in features as they come along, but I want to break from that a bit, so 1.10 isn't going to see any significant new features. The abstraction of the Ajax stuff isn't "major" itself, but there are other parts which will make it with a v2 name .

You are right though - at the moment I'm afraid that certain things such as dataSrc are internal.

Allan