How do I use the `ajax` function option properly?

craxal

I'm struggling to understand the proper way to use the function option for ajax. I load data from a REST API that returns batches of results and continuation tokens (the data set can be very large). I cache these results so I don't have to hit the server every time I redraw or change pages. I also support a "load more" feature when I want more data from the server.

The ajax function provides a callback that the documentation says I'm supposed to call when I have all my data. But getting all the data can take a while, so I want to render the data chunks as they come in. I also want to preserve the current page as data comes in.

Here's a simplified example of what I'm working with.

import * as bluebird from "bluebird";

interface Pair {
    name: string;
    value: any;
}

interface ContinuationResult<T> {
    next: any;
    result: T;
}

let table: DataTables.DataTable;
let cache: Pair[] = [];
let recordsPerSecond = 3;
let totalRecords = 45;

async function render(settings: DataTables.Settings, json: any): Promise<void> {
    if (cache.length > 0) {
        // Cache hasn't changed. No need to re-render.
        return;
    }

    let chunk: ContinuationResult<Pair[]> = null;
    do {
        let next = chunk && chunk.next;
        if (next) {
            chunk = await getChunk(next);
        } else {
            chunk = await getChunk();
        }

        // Call the DataTables callback to render the data.
        // It's possible this is the wrong way to do this. In which case, the API should be used instead.
        chunk.result.forEach((row) => {
            cache.push(row);
            table.row.add(row);
        });
        table.draw(false);
    } while (chunk.next);
}

async function getData(data: Object, callback: Function, settings: DataTables.SettingsLegacy): Promise<void> {
    if (cache.length > 0) {
        return callback({ aaData: cache });
    }

    let chunk: ContinuationResult<Pair[]> = null;
    do {
        let next = chunk && chunk.next;
        if (next) {
            chunk = await getChunk(next);
        } else {
            chunk = await getChunk();
        }

        // Call the DataTables callback to render the data.
        // It's possible this is the wrong way to do this.
        cache = cache.concat(chunk.result);
        callback({ aaData: chunk });
    } while (chunk.next);
}

async function getChunk(next: any = 0): Promise<ContinuationResult<Pair[]>> {
    await bluebird.delay(1000);

    let chunk: Pair[] = [];
    for (let i = next; i < next + recordsPerSecond; i++) {
        chunk.push({ name: i, value: Math.random() });
    }

    return { result: chunk, next: next <= totalRecords ? next + recordsPerSecond : null };
}

$(document).ready(() => {
    table = $("#datatable").DataTable({
        ajax: getData,
        // initComplete: render,
        pageLength: 10,
        paging: true,
        pagingType: "full_numbers",
        processing: true,
        serverSide: false,
        stateSave: false,
        columns: [
            {
                name: "Name",
                data: "name"
            },
            {
                name: "Value",
                data: "value"
            }
        ]
    });
});

allan

The ajax option isn't the way to do this - it expects all of the data up front and it can be passed as a single array to the callback function. Here you are loading it in chunks which isn't going to work.

Instead, don't use ajax, just have an empty DataTable initially, and use the rows.add() method to add the chunks as they become available via your own Ajax call.

Allan