Support for callbacks and async handling of buttons.exportData()

chancezeus

The current implementation of the export buttons is limited with regards to server side processing (basically, there is no option to handle exporting all data client side).

I have a couple of ideas how to do this and already implemented a version by registering a custom "buttons.exportData()" which does support loading the additional data from the server, but it however must use "async: false" since buttons.exportData() is a synchronous function.

Would it be possible to somehow change this behaviour to support async callbacks instead of synchronous returns (would also help when exporting large datasets since the current (regular) exportData() functionality can then use setTimeout and chunk it's parsing instead of the current fully synchronous implementation).

In addition to the async callbacks it would also be nice to support some sort of "data.header", "data.body", "data.footer" callbacks that (if set) replaces the current logic with regards to fetching this information and allows for the developer to implement custom filtering and/or (like in this case) ajax fetching the data from his server.

allan

How about adding a buttons.exportDataAsync() method? That could return a Promise (assuming you only need to support browsers that support promises) which would only then do whatever output you need.

Allan

chancezeus

That method will still not be used by the (existing) buttons for printing, copy, pdf, excel, csv, etc... The reason I am asking is to have the possibility to extend the current functioning of these methods without having to rewrite their code (from a user perspective). And the current solution was an example, for general use (especially from a user perspective), I'd much rather have callbacks in an config option (similar for the formatting of data) for the method than to override some (internal) api methods directly...

allan

Oh I see what you mean - you want buttons.exportData() to not just get the data from the DataTable as it currently does, but have the ability to go to the server and get more data?

That would be an option, but if you have server-side processing enabled, that would typically suggest you have many tens of thousands, hundreds of thousands or millions of rows. At that point, creating the export file on the client-side is never going to be performant.

Instead, I would suggest you create the file on the server-side and offer it for download. This plug-in button might be useful for you.

Allan

chancezeus

Good suggestion, this however means that the developer must also implement some sort of Excel/PDF/etc build on the server.

Besides that my suggestion is a solution for the situation where someone manages to get (a lot of) data into a single data table (not recommended I know) just by plainly putting it there (or using SSR/AJAX with no length limit) and tries to export that. With the current implementation the exportData will actually block any UI (inter)action (since it does never yield), by turning the exportData fully async even the "regular" implementation (ie. without additional callbacks, etc) can be made more responsive (in UI terms) by chunking and setTimeout. Something like this comes to mind:

var exportData = function (options, callback) {
  /** Current code for gettting header/footer/etc */

  var rowCount = selectedRowIndexes.length;

  var helper = function (offset, results) {
    var size = options.chunk_size || 100;
    var end = Math.min(rowCount, offset + size);

    for (var i = 0; i < end; i++) {
      /* Perform regular fetching/parsing of the (body) rows here */
      results.push(exportRowData);
    }

    if (end < rowCount) {
      if (callback) {
        setTimeout(function() {
          helper(end, results);
        });
      } else {
        return helper(end, results);
      }
    } else if (callback) {
      callback(results);
    } else {
      return results;
    }
  };

  return helper(0, []);
};

This way the function yields often enough for the UI to remain usable (at least show a spinner or something). The "if (callback)" checks allow for the function to remain working like it used before the update (and thus non-async returning the gathered data, this way backwards compatibility stays intact).

allan

the developer must also implement some sort of Excel/PDF/etc build on the server.

Yes. The export options on the client-side all really struggle with large data sets, which will easily lock the browser up if exporting millions of rows (depending on the browser and host computer, with tens of thousands of rows as well).

Doing it in batches as you say would help, but it might still take minutes to generate the file, while a server-side script could do it in a fraction of the time.

I'm not completely against the idea, I'm just not sure how pratical it is.

Allan