Row gets duplicated when I update rows with same id

acanfora

Hello,
I have a problem adding/updating data to DataTables. I am using JSON data, but do not use the ajax API, but rather I get the json data by jquery and pass it to DataTable. After Initial table setup, that works well, I try to add some rows via these calls:

function updateData(data){
    $.each(data.table.data, function(i, v){
        index = table.column(1).data().indexOf(v.id);
        if(index < 0 ){
            table.row.add(v); // row does not exist yet, so new one will be added
            index = table.column(1).data().indexOf(v.id);
            row = table.row(index);
        } else {
            row = table.row(index);
            row.data(v);
        }
    });
    // force new record to show up on top
    table.order([defaultSortColumn, 'asc']).draw();
    table.order([defaultSortColumn, 'desc']).draw();
}

table.data is an array of items of the same structure of the original init table data.
What I get is that the call:

        index = table.column(1).data().indexOf(v.id);

does not find the newly added rows, so the code above is ending up to add duplicate rows when a newly added row gets an update having same id.
My question is: do I need to perform some call in order to update the DataTables internal data cache? Is that one the reason that prevents IndexOf to find the newly added row ids?

Thank you in advance for the help you may give.

acanfora

Anyone?

allan

I've just put together a little example to test this and it appears to work as expected: http://live.datatables.net/midumuxo/1/edit .

Can you link to a test case that shows the problem so I can debug it please.

Allan

acanfora

Hello Mr. Jardine,
thanks for the quick reply. During debug I found that the issue is not with indexOf, but rather with something related to DOM duplication. Will get together a simplified test case and send you the link ASAP.

acanfora

Hello Mr. Jardine,
after two months I got the time to get back to this, sorry for delay.
Basically I have an update function that polls the server for new rows and updates the DataTable in this way (debug code included):

function updateData(data){
    timeoutId = setTimeout(pollEvents, 20000); // wait for the event to be received to avoid cumulative effect over time
    table = $('#packages').DataTable();
    var index; var row;
    $.each(data.table.data, function(i, v){
        index = table.column(1).data().indexOf(v.id);
        if(index < 0 ){
            console.log('didnt find it: ' + v.id);
            table = table.row.add(v).draw(); // row does not exist yet, so new package created
            index = table.column(1).data().indexOf(v.id);
            row = table.row(index);
        } else {
            console.log('found it: ' + v.id);
            row = table.row(index);
            row.data(v).draw();
        }

        delete detailsCache[v.id]; // force data to be reloaded from server
        displayDetails(row.data().id);
        delete eventsCache[v.id]; // force data to be reloaded from server
        displayEvents(row);
    });
}

the bug I am fighting with is that when a new value of the index in column 1 is sent from server, the row is created as expected, but as soon as an update for the same row (same value of column 1) is sent from server, a second row is created in DOM, despite the fact that the existant index is correctly found (see debug lines below). Furthermore, oddly, while the new row holds the new values and the old one retains the old ones, children rows are updated on both old and new row, as you can see in example below:

the function that updates child rows is the displayEvents listed below:

function displayEvents(row){
    var package_id = row.data().id;
    if (! eventsCache[package_id]){
        get('req=package_events&package_id=' + package_id, function(data){
            eventsCache[package_id] = data.package_events || {};
            displayEvents(row);
        });
        return;
    }

    if(isVoid(eventsCache[package_id]))
        return;

    /* always redraw for events update
    if(row.child()){
        row.child.show();
        return;
    }
    */
    var curevents = eventsCache[package_id];
    var tblchild = [];
    $.each(curevents, function(i, v){
        var message;
.....................................................................................
..................................   omissis   ..................................
.....................................................................................
        var cells = [];
        cells.push(' ');
        cells.push(v.datetime);
        cells.push(message);
        var row = cells.join('</td><td>');
        row = '<tr class="eventrow"><td>' + row + '</td></tr>';
        tblchild.push(row);
    });
    row.child(tblchild).show();
    row.nodes().to$().addClass('shown');
}

I tried several times to find the underlying bug without success, any clue?

allan

Could you clarify the child row aspect - is the problem with the child row?

Can you give me a link to the page so I can debug it directly please?

Allan

acanfora

Hi, I apologize for not being able to give you direct access to the page, it is a sensitive system and I have no permissions to do that. Nonetheless I did some further tests excluding the calls to the part of code that updates the child rows, the duplicate row problem persists, so it has nothing to do with child rows.
So the code which is causing the duplication is basically this one:

function pollEvents(){
    get('req=getNewEvents', updateData);
}

function updateData(data){
    console.log('updateData being called');
    timeoutId = setTimeout(pollEvents, 20000); // wait for the event to be received to avoid cumulative effect over time
    table = $('#packages').DataTable();
    var index; var row;
    $.each(data.table.data, function(i, v){
        index = table.column(1).data().indexOf(v.id);
        if(index < 0 ){
            table = table.row.add(v).draw(); // row does not exist yet, so new package created
            index = table.column(1).data().indexOf(v.id);
            row = table.row(index);
        } else {
            row = table.row(index);
            row.data(v).draw();
        }

        //delete detailsCache[v.id]; // force data to be reloaded from server
        //displayDetails(row.data().id);
        //delete eventsCache[v.id]; // force data to be reloaded from server
        //displayEvents(row);
    });
}

The initialization code of DataTable here is the following:

        modal_get('req=package_tracker', function(data){

            tableCache = data.table;
            tableCache.retrieve = true;
            tableCache.order = [defaultSortColumn, "desc"];
            tableCache.pageLength = 15;
            if($.fn.DataTable.isDataTable("#packages"))
                $('#packages').DataTable().clear().destroy();
            $('#packages').dataTable(tableCache);
            table = $('#packages').DataTable();
...................................................................................................
...................................................................................................
...................................................................................................

Here below you can find the payload of get responses being parsed by the updateData function.
the v.id in this case is the "aa8b6ae4-1e00-4fa7-bfd8-2426a345380d" value

first one - new row created correctly:

jQuery111205458382764095633_1432772005236({
   "errors" : [],
   "_authenticated" : "1",
   "table" : {
      "columns" : [
         {
            "className" : "details-control",
            "defaultContent" : "",
            "orderable" : false,
            "data" : null
         },
         {
            "visible" : 0,
            "name" : "id",
            "data" : "id",
            "title" : "id"
         },
         {
            "visible" : 1,
            "name" : "Package Name",
            "data" : "Package Name",
            "title" : "Package Name"
         },
         {
            "visible" : 1,
            "name" : "Status",
            "data" : "Status",
            "title" : "Status"
         },
         {
            "visible" : 1,
            "name" : "Last Change",
            "data" : "Last Change",
            "title" : "Last Change"
         },
         {
            "visible" : 1,
            "name" : "Sender",
            "data" : "Sender",
            "title" : "Sender"
         },
         {
            "visible" : 0,
            "name" : "sender_id",
            "data" : "sender_id",
            "title" : "sender_id"
         }
      ],
      "data" : [
         {
            "Last Change" : "Thu, 28 May 2015 10:14:40 AM",
            "Status" : "DRAFT",
            "Package Name" : "GSL-SGB-1TEST",
            "id" : "aa8b6ae4-1e00-4fa7-bfd8-2426a345380d",
            "sender_id" : "k8MP7RtROSYM",
            "Sender" : "SG Bank"
         }
      ]
   },
   "authenticated" : true,
   "_debug" : "1",
   }
)

second one - duplication occurs:

jQuery111205458382764095633_1432772005236({
   "errors" : [],
   "_authenticated" : "1",
   "table" : {
      "columns" : [
         {
            "className" : "details-control",
            "defaultContent" : "",
            "orderable" : false,
            "data" : null
         },
         {
            "visible" : 0,
            "name" : "id",
            "data" : "id",
            "title" : "id"
         },
         {
            "visible" : 1,
            "name" : "Package Name",
            "data" : "Package Name",
            "title" : "Package Name"
         },
         {
            "visible" : 1,
            "name" : "Status",
            "data" : "Status",
            "title" : "Status"
         },
         {
            "visible" : 1,
            "name" : "Last Change",
            "data" : "Last Change",
            "title" : "Last Change"
         },
         {
            "visible" : 1,
            "name" : "Sender",
            "data" : "Sender",
            "title" : "Sender"
         },
         {
            "visible" : 0,
            "name" : "sender_id",
            "data" : "sender_id",
            "title" : "sender_id"
         }
      ],
      "data" : [
         {
            "Last Change" : "Thu, 28 May 2015 10:14:40 AM",
            "Status" : "DRAFT",
            "Package Name" : "GSL-SGB-1TEST",
            "id" : "aa8b6ae4-1e00-4fa7-bfd8-2426a345380d",
            "sender_id" : "k8MP7RtROSYM",
            "Sender" : "SG Bank"
         }
      ]
   },
   "authenticated" : true,
   "_debug" : "1",
}
)

third one - the duplicated row gets updated, the first one remains unchanged:

jQuery111205458382764095633_1432772005236({
   "errors" : [],
   "_authenticated" : "1",
   "table" : {
      "columns" : [
         {
            "className" : "details-control",
            "defaultContent" : "",
            "orderable" : false,
            "data" : null
         },
         {
            "visible" : 0,
            "name" : "id",
            "data" : "id",
            "title" : "id"
         },
         {
            "visible" : 1,
            "name" : "Package Name",
            "data" : "Package Name",
            "title" : "Package Name"
         },
         {
            "visible" : 1,
            "name" : "Status",
            "data" : "Status",
            "title" : "Status"
         },
         {
            "visible" : 1,
            "name" : "Last Change",
            "data" : "Last Change",
            "title" : "Last Change"
         },
         {
            "visible" : 1,
            "name" : "Sender",
            "data" : "Sender",
            "title" : "Sender"
         },
         {
            "visible" : 0,
            "name" : "sender_id",
            "data" : "sender_id",
            "title" : "sender_id"
         }
      ],
      "data" : [
         {
            "Last Change" : "Thu, 28 May 2015 10:18:16 AM",
            "Status" : "SENT",
            "Package Name" : "GSL-SGB-1TEST",
            "id" : "aa8b6ae4-1e00-4fa7-bfd8-2426a345380d",
            "sender_id" : "k8MP7RtROSYM",
            "Sender" : "SG Bank"
         }
      ]
   },
   "authenticated" : true,
   "_debug" : "1",
}
)

hope this can help/is sufficient to find the problem.

acanfora

Hi,
seems that the origin of the problem is that the .row(index) method used above is not returning the expected row. I changed the function updateData as listed below, and the duplicated row bug cleared.

function updateData(data){
    timeoutId = setTimeout(pollEvents, 20000); // wait for the event to be received to avoid cumulative effect over time
    table = $('#packages').DataTable();
    var index; var row;
    $.each(data.table.data, function(i, v){

        var indexes = table.rows().eq(0).filter(function(idx){
            return table.cell(idx, 1).data() === v.id ? true : false;
        });

        table.rows(indexes).remove().draw();
        table = table.row.add(v).draw();
        indexes = table.rows().eq(0).filter(function(idx){
            return table.cell(idx, 1).data() === v.id ? true : false;
        });
        row = table.rows(indexes).eq(0);

        delete detailsCache[v.id]; // force data to be reloaded from server
        displayDetails(v.id);
        delete eventsCache[v.id]; // force data to be reloaded from server
        displayEvents(row);
    });
}

if you know a more elegant solution, please advise. I think this way to search and select for values is a little bit too convoluted.

allan

Hi,

Thanks for posting back your findings. The code above is treating the each index from looping over data.table.data as the row index. I don't see where data.table.data is defined, but my assumption from your findings is that it is not based on the row index (i.e. rows().data()).

You could use rows().every() to loop over the rows in the table, or iterator().

Or if the above works for you, use that :-)

Allan

acanfora

Hi,
the data.table.data is the data sent from ajax callback, of which you can see an example above, see just the part:

      "data" : [
         {
            "Last Change" : "Thu, 28 May 2015 10:14:40 AM",
            "Status" : "DRAFT",
            "Package Name" : "GSL-SGB-1TEST",
            "id" : "aa8b6ae4-1e00-4fa7-bfd8-2426a345380d",
            "sender_id" : "k8MP7RtROSYM",
            "Sender" : "SG Bank"
         }

then the correspondant row id is searched by the line:

            index = table.column(1).data().indexOf(v.id);

that should give back the right underlying table array row index (and seems that actually it does that). It seems that the problem is with the row()DT method called with the "row index selector" row-selectorDT, passing to it the integerJS array index obtained from the call listed above, that gives back a row in DOM different from the one containing the v.id value in 2nd (index 1) column. By now I managed to solve my problem, but perhaps it would be worth to check this api to see if it is working as expected.

allan

The index selector doesn't have anything to do with the data in the row. It is an internal index maintained by DataTables - row().index() will give the index for a row.

Allan

acanfora

"The index selector doesn't have anything to do with the data in the row. It is an internal index maintained by DataTables "

I already perfectly understood it. Perhaps you had just a quick glance on my posts and missed the point. What I am reporting is that in my former code I was first getting the "internal index" by using the call:

index = table.column(1).data().indexOf(v.id);

then I was trying to use such index to select a DOM row by calling the:

row = table.row(index);

as documented in your APIs. But this call actually does not select the intended row, differently from what documentation says. So I thought that would have been useful to report this problem even if I managed to find a workaround for it. Hope this will help to improve this nice library.

allan

index = table.column(1).data().indexOf(v.id);

But that doesn't return the internal index. It returns the current order index.

column() when not given a selector-modifier object uses the default options for the selector modifier, which is to return in the current display order.

To get the data in index order use the order option of the selector modifier:

index = table.column(1, { order: 'index' } ).data().indexOf(v.id);

Allan

acanfora

Thank you for the tip, it wasn't immediate from documentation. I would suggest an addition to apis to get a fast selection of a row based on content. Also an API providing an out of the box way to reference columns by column name/title besides the already existant one that uses indexes would be handy. For example two apis with imaginary names of query and vert that would enable us to write something like:

rows = table.query('columnname', 'value to search in column');
column = table.vert('columnname/columntitle');

would be very handy. Hope this could be interesting for you. Thank you very much for your attention!

allan

You can use the selectors as functions which allows you to select by data. See the row-selector documentation, specifically the function option.

However, yes, it would be nice if this would be easier to use and is somethingI plan to introduce in future.

Allan