Server-Side Processing does not redraw the table

sek001

I have a Python-Flask app in which I am trying to insert a data table with server-side processing and deferred loading. The Flask template generates the first ten rows of the table as html and inserts the number of rows and draw number into DOM elements for easy access. The user has numerous options for changing the data in the table, and there is a button to re-generate the table with the new data. The table initialises correctly, and the ajax request returns a valid json response. However, pagination only works the first time a page button is clicked. The second time, the processing indicator hangs. If I change one of the user-defined settings, it also appears to work, but the table is not redrawn the second time I do it. I have not been able to figure out what is causing these issues, so I'm hoping someone can see what I'm missing. Here is a shortened version of my html code.

<!-- User-defined setting for constructing the table matrix. A number of other 
     settings are omitted. -->
<input name="tokenSize" id="tokenSize" min="1" max="5" step="1" value="1" type="number">

<!-- Trigger button to apply user-defined settings -->
<input class="bttn" id="generateMatrix" value="Generate Table" type="button">

<!-- Placeholders for variables initially assigned in the template -->
<div id="numRows" style="display: none;">{{ numRows }}</div>
<div id="draw" style="display: none;">{{ draw }}</div>

<!-- First ten table rows are assigned in the template -->
<table id="example" class="display table table-bordered table-striped table-condensed">
    <thead>
        <tr>
            <th>token</th>
            {% for label in headerLabels %}
            <th>{{ label }}</th>
            {% endfor %}
        </tr>
    </thead>
    <tbody>
        {% for row in matrix %}
        <tr>
            {% for cell in row %}
                <td>{{ cell }}</td>
            {% endfor %}
        </tr>
        {% endfor %}
    </tbody>
</table>

And the javascript:

$(document).ready(function() {

    // Function to draw the table
    function drawTable() {  
        var myTable = $('#example').DataTable({
            "processing": true,
            "deferLoading": $("#numRows").text(),
            "serverSide": true,
            "paging": true,
            "pageLength": 10,
            "ajax": {
                "type": "POST",
                "url": "/generateMatrix",
                "dataType": "json",
                "contentType": 'application/json; charset=utf-8',
                "data": function (data) {
                    // Get the values of all the form inputs
                    var form = {};
                    $.each($("form").serializeArray(), function (i, field) {
                        form[field.name] = field.value || "";
                    });
                    // Assign parameters for data tables
                    var parameters = {};
                    parameters.draw = (myTable == null) ? { "draw": 1 } : parseInt($("#draw").text());
                    var draw = (myTable == null) ? { "draw": 1 } : { "draw": parseInt($("#draw").text()) };
                    var info = (myTable == null) ? { "start": 0, "end": 10, "length": 10 } : myTable.page.info();
                    $.extend(form, draw);
                    $.extend(form, info);
                    form = JSON.stringify(form);
                    return form;
                },
                "complete": function(response) {
                    console.log("Page Info: "+myTable.page.info());
                    console.log("Response: "+JSON.stringify(response));
                    // Update the draw element in the DOM
                    $("#draw").text(response["draw"]);
                }
            }
        });
    }

    // Initiate the data table on page load
    drawTable();

    // When the Generate Matrix button is clicked, submit the new settings by ajax
    $("#generateMatrix").click(function() {
        drawTable();
    }

});

I have tried to apply the draw() function in numerous places, but nothing seems to work.

glenderson

I think you need to first destroy the table in your click function. You cannot have two instances of the same named table as there are lots of hidden / stored elements in both HTML and memory.

See this link
https://datatables.net/reference/api/destroy()

sek001

Thanks. This didn't work alone, but I think I also need to use empty() to remove the table from the DOM and then append a new one with the new first ten rows before re-initialising Datatables. This will involve some tweaking of my server-side code, so I'll leave this open and tag it answered if and when I get it working.

sek001

Still no luck on this one. Here's a simplified version of what I tried.

// When the Generate Matrix button is clicked, submit the new settings by ajax
$("#generateMatrix").click(function() {
    refreshTable();
}

function refreshTable() {
    // Destroy and empty the old table. 
    $('#example').DataTable().destroy();
    $('example').empty();
}

// Get column headers for the new table
$.ajax({
    url: "/test",
}).done(function(response) {
    var response = JSON.parse(response);
   // A string with the <thead> portion of the table
    $('example').append(response["headers"]);
});

var newTable = $('#example').DataTable(config);

So far, so good. A new table is drawn with the correct columns. If my server-side script also returns a data object and I set that as javascript sourced data, the complete table is drawn. However, it's entirely client-side from then on. To get a server-side table with deferred loading, I appended a <tbody> with ten rows after the <thead> in the ajax function above. I then tried to initialise the data table like this:

$('#example').DataTable({
    "serverSide": true,
    "deferLoading": 10,
    "processing": true,
    "ajax": "/test2"
});

The Python script in test2 returns the rest of the data in json format as follows:

{ 
  "draw": 1,
  "recordsTotal": 20,
  "recordsFiltered": 20,
  "data": [
                {"col1-value", "col2-value", "col3-value"},
                {"col1-value", "col2-value", "col3-value"}
    ]
 }

I believe this is correct. But the data is not written to the table. Instead, I get a mysterious error: TypeError: e[i] is undefined. I can't figure out what is causing it. Any ideas would be greatly appreciated.

allan

{"col1-value", "col2-value", "col3-value"},

That doesn't look like valid JSON to me. JSONLint confirms this.

I think you probably want to use arrays rather than objects since you haven't use columns.data to tell DataTables to read the data from objects.

Also there should be no need to destroy the table for each draw. Simply calling draw() should be all you need to refresh the table's data.

Allan

glenderson

Back to your original code, I think this is all you needed.

The destroy was to completely reinitialize based upon the code that was already written.

    // When the Generate Matrix button is clicked, submit the new settings by ajax
    $("#generateMatrix").click(function() {
       $('#example').DataTable().destroy();
       $('example').empty();  // Not sure if this is even needed
        drawTable();
    }

When I refresh tables, I first determine if the table is even present, then depending upon if table is present, decide if the row that I'm going to refresh is an update which you would use .draw() or a new row using .add(), but that is a far departure from the code that already existed. Some many ways to do the same thing.

sek001

My bad on the json form--thanks for point that out, Alan.

At long last, I appear to have some code that works. Here it is:

$(document).ready(function() {
    // Create a new table with ten rows in the DOM for deferLoading
    drawTenRows()

    // Initialise a server-side data table
    drawTable();

    // When the Generate Matrix button is clicked...
    // and re-create it with the new form settings
    $("#generateMatrix").click(function() {
        // ...destroy and empty the table
        $("#example").DataTable().destroy();
        $("#example").empty();
        // update the draw placeholder
        newDraw = parseInt($("#draw").text()) + 1;
        $("#draw").text(newDraw);
        // Draw the first ten rows
        drawTenRows();
        // Initialise the data table
        drawTable();
        // Refresh the data table
        $("#example").DataTable().draw();
    });
});

// Function to draw the html for deferLoading
/* Needs a separate ajax request if the columns change? */
function drawTenRows() {
    // Create a new table with ten rows
    $("#example").append("<thead></thead>");
    var rows = "{% for row in matrix[0:10] %}<tr>{% for cell in row %}<td>{{ cell }}</td>{% endfor %}</tr>{% endfor %}";
    var tbody = "<tbody>" + rows + "</tbody>";
    $("#example").append(tbody);    
}

// Function to draw the table
function drawTable() { 

    // Get the current draw from the DOM
    var thisDraw = {"draw": parseInt($("#draw").text()) };

    // Define the columns
    /* Needs a separate ajax request if the columns change? */
    var columns = [{"title": "Term"},{% for label in headerLabels %}{"title": "{{ label }}"}{%- if not loop.last %},{% endif %}{% endfor %}];

    // Configure table options
    var config = {
        "processing": true,
        "deferLoading": parseInt($("#numRows").text()),
        "serverSide": true,
        "paging": true,
        "pageLength": 10,
        "lengthMenu": [[10, 25, 50, -1], [10, 25, 50, "All"]],
        "columns": columns,
        "ajax": {
            "beforeSend": function() {},
            "type": "POST",
            "url": "/generateMatrix",
            "contentType": 'application/json; charset=utf-8',
            "dataType": "json",
            "data": function (data) {
                var form = {};
                $.each($("form").serializeArray(), function (i, field) {
                    form[field.name] = field.value || "";
                });
                var draw = { "draw": parseInt($("#draw").text()) };
                var info = (myTable == null) ? { "start": 0, "end": 10, "length": 10 } : myTable.page.info();
                $.extend(form, draw);
                $.extend(form, info);
                form = JSON.stringify(form);
                return form;
            },
            "complete": function(response) {
                // Update the draw element in the DOM
                $("#draw").text(response["responseJSON"]["draw"]);
            }
        }       
    }

    // Initialise the table
    var myTable = $('#example').DataTable(config);
}

All the server-side processing functions seem to work for paging. Clicking the Generate button re-draws the table with the new data. If the drawTenRows() procedure seems to over-complicate things, it is because one of the user options is to pivot the table, which changes the number of columns. From what I understand, it is not possible to change the number of columns once the table is initialised. If this is the case, I can't see any alternative to destroying the table and using a separate ajax call to get the columns array before calling drawTable(). if there are alternatives, I'd be happy for suggestions. But I'll mark this questions "answered".

allan

From what I understand, it is not possible to change the number of columns once the table is initialised

That is correct. If you need that ability, a table destroy and recreate is currently, and unfortunately, the only option.

Allan