columnControl not populated in ajax.data and stateSaveParams

Wernfried

When you use columnControl together with ajax.data, serverSide: true and/or stateSave: true, stateSaveParams then the columnControl search settings are not included in callback data parameter:

$("#example").DataTable({
  columns: [...],
  columnControl: [ "order", [ 'searchClear',  { 'search' } ] ],

  serverSide: true,
  ajax: {
     url: '/ajax/data',
     contentType: 'application/json',
     submitAs: 'json',
     data: function (data) {
        console.log(data.columnControl); // -> prints 'undefined'
     }
  },

  stateSave: true,
  stateSaveParams: (settings, data) => {
     console.log(data.columnControl); // -> prints 'undefined'
  })
})

The data is properly send for Ajax and state is properly saved, including all columnControl settings, that's working fine. However, if someone likes to manipulate data.columnControl then it is not possible, because they are not accessible.

For me this issue is not critical, because I don't need to manipulate data.columnControl, I only wanted to see it for debugging (where other possibilities are available).

Kind Regards
Wernfried

Wernfried

After some testing, I have to revert my statement "The data is properly send for Ajax and state is properly saved, including all columnControl settings, that's working fine."

Ajax works fine. However, the columnControl state is saved properly, but when you reload the page, i.e. when state is loaded then saved search values are used only for plain searches but searchList are not used.

allan

Are you able to provide a link to a test case showing the issue please?

Thanks,
Allan

Wernfried

Let's split my post and focus to one issue each.

ajax.data not complete

When you use Server-Side processing and columnControl, then sent data is enriched with columnControl data, see Server-side processing

Here full config to demonstrate.

$("#dataTable").DataTable({
  columns: [
     { name: 'id', data: 'id', title: 'ID' },
     { name: 'age', data: 'age', title: 'Age', columnControl: ["order", [{ extend: "searchNumber" }]] },
     {
        name: 'name', data: 'name', title: 'Name',
        columnControl: [
           "order", [{
              extend: "searchList",
              options: [
                 { label: 'Brad', value: 'Bradley Greer' },
                 { label: 'Tiger', value: 'Tiger Nixon' }
              ]
           }]
        ]
     }
  ],
  layout: {
     topStart: { buttons: ["colvis", "ccSearchClear"] },
     topEnd: "search"
  },
  ajax: {
     url: '#',
     contentType: 'application/json',
     submitAs: 'json',
     type: 'POST',
     beforeSend: function (jqXHR, settings) {
        let data = JSON.parse(settings.data);
        console.log(`beforeSend => ${JSON.stringify(data.columns.map(x => x.columnControl), null, ' ')}`)
     },
     data: function (data) {
        console.log(`ajax.data => ${JSON.stringify(data.columns.map(x => x.columnControl), null, ' ')}`)
     }
  },
  serverSide: true
});

Open the page. No data is loaded/shown, but this does not matter. Set some filter, e.g.

console.log in data prints

ajax.data => [
   null,
   null,
   null
]    

but console.log in beforeSend prints

beforeSend => [
   null,
   {
      "search": {
         "value": "1",
         "logic": "greater",
         "type": "num"
      }
   },
   {
      "list": {
         "0": "Bradley Greer"
      }
   }
]

The result should be the same.

The data which is sent to server is 100% correct, however ajax.data does not include the columnControl addon. So, there is no easy way to manipulate columnControl related data before it is send to the server.

Wernfried

Next issue:

stateSaveParams.data not complete

$("#dataTable").DataTable({
   data: [
      { id: 1, age: 31, name: "Tiger Nixon" },
      { id: 2, age: 63, name: "Bradley Greer" }
   ],
   columns: [
      { name: 'id', data: 'id', title: 'ID' },
      { name: 'age', data: 'age', title: 'Age', columnControl: ["order", [{ extend: "searchNumber" }]] },
      {
         name: 'name', data: 'name', title: 'Name',
         columnControl: [
            "order", [{
               extend: "searchList",
               options: [
                  { label: 'Brad', value: 'Bradley Greer' },
                  { label: 'Tiger', value: 'Tiger Nixon' }
               ]
            }]
         ]
      }
   ],
   stateSave: true,
   stateSaveParams: (settings, data) => {
      console.log(`stateSaveParams => ${JSON.stringify(data.columnControl, null, ' ')}`);
   }
}).on('draw', function (e, settings) {
   $.each(localStorage, function (key, value) {
      if (key.startsWith('DataTable')) {
         let data = JSON.parse(value)
         console.log(`localStorage => ${JSON.stringify(data.columnControl, null, ' ')}`);
      }
   })
});

Open the table and do some filtering, like in previous example.

stateSaveParams prints

stateSaveParams => undefined

but in localStorage you have

localStorage => {
 "1": {
  "searchInput": {
   "logic": "greater",
   "type": "num",
   "value": "10"
  }
 },
 "2": {
  "searchList": [
   "Bradley Greer"
  ]
 }
}

The issue is similar to the previous one.
The data which is sent to localStorage is 100% correct, however stateSaveParams.data does not include the columnControl addon. So, there is no easy way to manipulate columnControl related data before it is send to localStorage.

Wernfried

Next issue:

searchList dropdowns are not restored from localStorage

Use the same setup as before, and do some columnControl filtering

Refresh the page.

Filtering for age is restored from localStorage, however name is not:

Please note, in the table the filter is applied properly (you see only one row for Bradley Greer). But the dropdown list in columnControl search is not updated. It may look like only a cosmetic issue. However, when you use server-side processing, then the server has no information about any searchList filtering which was restored from the localStorage.

allan

The data which is sent to server is 100% correct, however ajax.data does not include the columnControl addon.

Interesting one. The issue is that preXhr, which ColumnControl uses to add its data, is executed after the ajax.data function. In this case that is certainly wrong, but I'm hesitant to immediately change the order in case there is something I'm not thinking of that would cause a problem. The documentation for preXhr doesn't say where in the sequence it will trigger, so it is something that could change. I've got a note to look into this more.

What what it is worth though, if you want to submit a JSON string to the server, use ajax.submitAs.

stateSaveParams => undefined

The stateSaveParams callback is happening before the stateSaveParams event. If you do table.on('stateStaveParams', ...) instead then it should work.

I will have a think about how I can have the extension events happen before the developer's listeners. That is an important point there.

searchList dropdowns are not restored from localStorage

This is very much a bug. Thanks for letting me know about it. I've committed a fix and will be releasing ColumnControl 1.2 with the change (and your PR) shortly.

Allan

Wernfried

Thank you, I will test it after Christmas.
The 3rd issue was most relevant for me, thanks for the fix. Issue 1 and 2 are more cosmetic.

Wernfried

Wernfried

Regarding searchList dropdowns are not restored from localStorage

It looks better with ColumnControl 1.2 - as long as you don't use ColReorder extension with changed order of columns.

If you reorder your columns and reload the table, then search conditions are messed up. Please let me know, if you need sample config and data to reproduce the issue.

allan

I'll need to recreate it locally. If it is as simple as ColReorder, ColumnControl searchList and state saving, then I should be able to recreate it. I'll take a look, probably early in the new year.

Allan

Wernfried

I found another issue with ColumnControl and stateSave

• It does not work when you reorder the columns with ColReorder (reported above)
• It does not work for "complex" searchList

I define items with options, it looks like this:

[ 
{ label: '<span class="icon-text"><span class="icon"><i class="mdi--clipboard-clock-outline"></i></span><span>Issued</span></span>', value: "ISSUED" },
{ label: '<span class="icon-text"><span class="icon"><i class="mdi--certificate"></i></span><span>Authorised</span></span>', value: "AUTHORIZED" },
{ label: '<span class="icon-text"><span class="icon"><i class="spinning-gears-18px"></i></span><span>Started</span></span>', value: "STARTED" }
]
​

The state for the seems to be saved correctly. When you reload the page, then last filter is applied properly, but column in header the selected values are not marked.

Wernfried

I created example for testing and reproducing: https://live.datatables.net/rihobaga/1/edit

When you load the page, then search icon of Status is "active"

and this send to server (and the server filters the result accordingly).

{
 "list": {
  "0": "FINISHED_SUCCESSFULLY"
 }
}

so that's all fine.

But the value is not selected, the page should be open as this:

Wernfried

I created the same example but without server-side processing and no state save.

https://live.datatables.net/zamubene/1/edit

As you see, the filtering does not work either:

Table should show 2 rows. While this might be a valid bug, it does not apply to my application, because I use server-side processing.

Best Regards and happy new year
Wernfried

allan

It's not working with the client-side example because the column's render function is always returning an HTML string for the icon. It should only do that for the display data type. For the filter data type you actually want it to return value. You might want that for ordering as well.

I've tweaked the client-side example to do that, which allows it to work as expected.

I believe that is the issue with the server-side processing example as well. Here is the updated test case and it is sending the correct data.

Allan

Wernfried

For client-side I expected such similar answer. I did not had a detailed look into render documentation.

But for server-side, your change does not help. It looks like this:

The search icon (the three lines) are marked as active, the table is filtered correctly but the drop-down does not tick the selected values.

Here a tweaked ajax function to make proper simulation of server:

  ajax: function (data, callback, settings) {     
    console.log('sent to server: ' + JSON.stringify(data.columns[1].columnControl, null, ' '));
    return callback({
       draw: data.draw,
       recordsTotal: 6,
       recordsFiltered: 6,
       data: [
      { "status": "AUTHORIZED", "id": 1 },
      { "status": "AUTHORIZED", "id": 2 },
      { "status": "AUTHORIZED", "id": 3 },
      { "status": "FINISHED_SUCCESSFULLY", "id": 4 },
      { "status": "FINISHED_SUCCESSFULLY", "id": 5 },
      { "status": "CANCELLED", "id": 6 }
       ].filter(function(x) { 
         var values = Object.values(data.columns[1].columnControl.list);
         return values.length == 0 || values.includes(x.status)})
     });
  }

After some more tests I found out:

It works fine when ColumnControl gets the option list from server response (see https://datatables.net/extensions/columncontrol/server-side columns[i][columnControl][list])

But it fails when options list is a static list defined on client-side like this:

 columns: [
  {
     name: 'status', data: 'status', title: 'Status',
     columnControl: [
        "order",
        [{
           extend: "searchList", 
           options: [
              {value: 'AUTHORIZED', label: 'somehting nicely formatted'},
              {value: 'FINISHED_SUCCESSFULLY', label: 'somehting nicely formatted'},
              {value: 'CANCELLED', label: 'somehting nicely formatted'},
           ]
        }]
     ]
  }

For me it's fine, I don't care whether I define the static list at client-side in option columns.columnControl.options or if I put it at the server-side response.

Wernfried

I created two examples to illustrate my previous post.

Working
https://live.datatables.net/jukozoxi/1/edit

Not working
https://live.datatables.net/rihobaga/7/edit

But as said, for me this limitation is fine, there is no urgent demand from my side to get it fixed.

Kind Regards
Wernfried

Wernfried

One more note: Last Working/Not Working statement applies also ColReorder issue.

Short and final summary:

When you use server-side processing and stateSave and searchList columnControl, then you should not use searchList.options - It breaks ColReorder and it fails to show selected values.

searchList values should be provided by server returned data columnControl[columnDataSrc], (or maybe added at ajax.dataSrc - I did not test) then everything is working fine.

It's up to you, whether you are going to fix this issue or not. However, if you don't fix it, then it might be useful to add according note to the documentation.

Kind Regards
Wernfried

allan

Hi Wernfried,

Many thanks for your investigation into this! This is the commit to fix the issue with locally defined options not being shown as selected when state saving is used.

Allan

allan

I haven't yet got a fix for the ColReorder issue yet I' m afraid. I think that is something I'm going to have to make a change in ColReorder to address. Part of the problem is that which plugin runs first (i.e. the load order) means we can have different code paths.

Allan