stateSave: true.. Additional Ajax call

maniiu

Link to test case:
Debugger code (debug.datatables.net):
Error messages shown:
Description of problem:

Hi again..
Still need some help...
I see that when enabling stateSave: true and serverSide:true, datatables send 2 ajax requests.
It would be even ok, but I don't want to allow user to operate on the table till the second ajax is finished.
In reality after the first call the table is fully rendered and the progressing/waiting icons of "processing: true" disappears before the 2nd call is finished..
It would be difficult to post the script without attaching sensible data.. so I hope you guys know a fast solution for me even without it..
Nevertheless here are the first rows of it:

$('#table').DataTable({
stateSave: true,

                        serverSide: true,
                        processing: true,
                        ajax: {
                            url: '{{url('getlist')}}',
                            type: 'GET',
                            data: function (d) {
                                d.skip = d.start;
                                d.start = 0;
                            },

                        },

I use also search panes.

Is there a way to keep the loading bar till the end of server-side processes?
Thanks!

maniiu

I can't figure this out..
I've checked that I am not initializing datatables twice... and I am not.. this is also evident as by just removing stateSave: true Ajax is getting called one time..
Checked the URI and return data from both calls and they are identical... (believing that datatable was calling ajax twice to recovered stored filters.. not the case, the first render is already filtered and identical to the second)
I removed all other scripts on the page.. but problem is still there..

I see the table is rendered twice, by console.logging d (as per above post) I get
draw: 1, columns: Array(12), order: Array(1), start: 0, length: 10, … on the first call
draw: 2, columns: Array(12), order: Array(1), start: 0, length: 10, …... on the second...

If I click on page 2( or 3 or 4 or back to 1) using navigation on the bottom.. I get the results of page 2,retrieved with only one ajax call.. But if i refresh the page (page 2 in this case) it comes back firing to 2 ajax calls..

additional info are that I am using
datatables.net-bs5
datatables.net-searchpanes-bs5
datatables.net-buttons-bs5
datatables.net-select-bs5

Any idea?

maniiu

Last info..
Removing browser history, and refreshing the page fires only one call.. But when I refresh the page ... the 2 come back!
Indeed something related to cookies and Statesave... probably a wanted functionality.. But why the loading bar of "processing: true" disappears when first render is over and isn't visibile during 2nd render?

kthorngren

I'm not sure of the reason but I believe the 2 ajax requests are caused when both stateSave is enabled and SearchPanes is enabled. Looks like, for this case, you can use deferLoading to eliminate the first ajax request. See this example:
https://live.datatables.net/qoliyehi/11/edit

You can comment out deferLoading to see the two ajax requests. You can comment out the dom option to see that server side processing with stateSave uses only one request.

@alan can confirm this is true and explain why the two requests happen.

Kevin

maniiu

Thanks, but unfortunately this seems not to work.. adding deferloading I get an empty table with no data but just the header (table is fully generated with ajax, no <tr> data are included in the html).. I can't see any ajax call fired.. after that ordering, or filtering, or changing page send ajax call and get the table generated.. Then I was trying to look at the differences in between your script and mine.. as you declared <thead> so i try to do the same (I was using just column attribute for that).. The result is a table that still don't fire ajax call, but shows a forever loading bar.. ..
why you get the ajax call in your script and I don't is something I can't expain

allan

adding deferloading I get an empty table with no data but just the header

That is expected. deferLoading, as the documentation explains, means that you have rendered the first page of the DataTable into the HTML so no Ajax call is sent. Then the next interaction with the table will trigger an Ajax load.

That option was really for when search engines didn't execute Ajax. Those days have long gone, so I think that option should no longer be used - it just adds complexity. I'll mark it as deprecated and remove it in v2.

@allan can confirm this is true and explain why the two requests happen.

It happens in your example there because the request to objects.php results in a 307 Redirect to the /api/staff controller. I put the redirect in to allow older Bins to continue working when I updated it relatively recently. The 307 can more or less be ignored.

Allan

kthorngren

@maniiu

table is fully generated with ajax

Can you post your relevant Javascript code showing this and your Datatables initialization?

@allan

It happens in your example there because the request to objects.php results in a 307 Redirect to the /api/staff controller.

Yes I noticed that, but its good to know why. Comment out deferLoading and you will see two sets of requests:
https://live.datatables.net/yuyunavu/1/edit

Like this:

EDIT: Remove the dom option to not initialize SearchPanes and there is one set of requests.

Kevin

allan

Ah! I wasn't seeing it as I didn't have a state saved. If I change page in the table and reload, then I do indeed see the error. Almost certainly a SearchPanes issue. It is due for a bit of a rewrite, so I'll look at this then.

Allan

maniiu78

Here I am back.. needed to register again.. I probably did some mess with posting my last message and my account I've been locked..
Thanks for your support guys.. I would be good to have that fixed..

maniiu

Before getting my account locked, I was trying to post that I've been able to have DeferLoading working..
I found that the reason was "drawCallback": function (settings) {..." that I was using to incercept the SearchPanes filters applied and show them in a div..

It was preventing the table to show.. Once removed that part of script, it was almost working as expected, as, strangely, sometime the table was still empty and some other times full of data.. I haven't found the trigger for that behaviour..

So I came back to the situation with 2 ajax calls, but I am still looking to find a solution to avoid the user interaction with the table during the second call (my calls are also pretty slow, 8 sec each.. I will have to find a way to optimize the query)... In the meantime intercepting the II call and add a loading icon would be perfect.. Any idea on how to achieve that?

kthorngren

my calls are also pretty slow, 8 sec each.

You have server side processing enabled but it sounds like your server script might be returning all rows instead of limiting the rows returned to just the page being sidplayed. Are you using a Datatables supplied server side processing script or does your server script support the SSP protocol.

Kevin

maniiu

nope it returns only the rows that needs to be visualized.. Server side I use my own script based on SSP protocol... works fine, I have just too much data and joins..

kthorngren

It was preventing the table to show

Are you getting errors in the browser's console?

Without seeing what you are doing its hard to say. Can you update my above example to show what you are doing?

sometime the table was still empty and some other times full of data.. I haven't found the trigger for that behaviour..

This we would need to see to help debug. Again look for errors in the browser's console. Also use the browser's network inspector tool to see the XHR requests and responses.

Kevin

maniiu

Thanks kthorngren you are very supportive!

sometime the table was still empty and some other times full of data.. I haven't found > the trigger for that behaviour..

I found a way to replicate the behavior! If you go in your example and remove your navigation data history.. then try to refresh the page you will see that the table is going to be empty.. I come to that as normally was the first visualization of the day giving me an empty table.

"drawCallback": function (settings) {..."
It was preventing the table to show

Forget about that.. not important.. can live without..

I've tried to listen to ajax call on the page.. and show the "processing" but no luck.. I believe that the first ajax call ends after the second call has been fired, so once the first call is finished it push property display:none to the "processing div" before II call ends.

like this: First Ajax call starts -> Processing div set to visible --> Second call Starts -> processing div set to visible (but useless) -> First call End immediately after -> processing div set to display:none--> after 5-6 seconds the second call ends but during this time no processing div is visualized and user can interact with the table -> div set to visible:none (but useless as it was already after the first call ends)..

Maybe I am wrong.. Thoughts?

kthorngren

I see, so if you use state.clear() to clear the saved state then the deferLoading option stops the initial load and there is not a second ajax request. I'm not sure there is an easy way around this until @allan looks at why SearchPanes is causing the second ajax request with stateSave. He may have some thoughts though. This can be seen with this example by clicking the clear state button:
https://live.datatables.net/qoliyehi/17/edit

I believe that the first ajax call ends after the second call has been fired, so once the first call is finished it push property display:none to the "processing div" before II call ends.

Makes sense since these are asynchronous. I believe SearchPanes is causing the second request to be sent independent of the initial table load.

Kevin

maniiu

Yep thanks Kevin... let's wait for @allan..

allan

I'm honestly not immediately sure why it is going wrong here. I think SP is due for a rewrite, mainly for performance, but also tracking things like this down are fairly difficult at the moment. I'm going to start work on that soon, although I expect it to be a multi-week process.

Allan

maniiu

another thing... Hope it may be useful to identify the problem:
I saw that adding deferloading:true then selecting a something in the SearchPanes, then the filter gets applied.. but when you refresh the page, the table remains filtered but the SearchPanes did not shows any selection, nor the possibility to remove the applied filters.. you then need to filter something else and then remove filters to get back to the unfiltered table.. This behavior is not happening when deferloading is set to false, nevertheless unfortunately, the 2 Ajax requests are getting fired..

allan

I've been thinking about deferLoading and SearchPanes (with server-side processing - since that is when it is used) a little more and I don't actually see how it could be made to work at all. SearchPanes needs extra data on load to be able to populate the panes. However, the whole point of deferLoading is to cause the Ajax load to wait until the user interacts with the table.

Allan

maniiu

Yep but the idea was in someway fixing the 2 calls issue, unfortunately it introduces new ones.
As you said, deferloading has not been though to work like that...
while we will be waiting for a permanent fix, I will have to find a way to pull back the "waiting bar" to avoid user interaction with the table..
If and when I will find a viable solution will publish it here as it may be a useful info for others.

Have a good day!

maniiu

ok. here my solution..
I see that the loading bar is generated when processing:true, but as said his visibility is set to none (display:none) as soon as the first ajax call ends but before the II ajax call finishes.
The loading bar ID is generated dynamically: ID=YourTableID_processing

What I've then done, was to try to intercept if a ajax call is pending.
By using a counter which gets incremented every time a new ajax call starts and decremented when the ajax call ends.
Then to intercept when datatable hide the loading bar I needed to set a timer that force a control on the loading bar and set it visible if the counter has not reached 0.
To avoid the user to interact with the table while the ajax calls are pending I also dynamically insert an overlay inside the container div of the page which will disappear when counter will be 0.
Here is the script:

$(document).ready(function () {
let pendingAjaxCalls = 0;
const overlay = $('#overlay');

// Function to create and append the overlay dynamically
function createOverlay() {
const overlay = $('

<

div>', {

id: 'overlay',
css: {
display: 'none',
position: 'absolute',
top: 0,
left: 0,
width: '100%',
height: '100%',
backgroundColor: 'rgba(255, 255, 255, 0.5)',
},
});
$('#container').append(overlay);
}

// Increment the counter when an AJAX call starts
$(document).ajaxSend(function () {
pendingAjaxCalls++;
setTimeout(updateMyTableProcessingVisibility, 10);
});

// Decrement the counter when an AJAX call is completed
$(document).ajaxComplete(function () {
pendingAjaxCalls--;
setTimeout(updateMyTableProcessingVisibility, 10);
});

// Function to update the visibility of MyTable_processing and the overlay
function updateMyTableProcessingVisibility() {
if (pendingAjaxCalls > 0) {
$('#MyTable_processing').show();
$('#overlay').show();
} else if (pendingAjaxCalls === 0) {
$('#MyTable_processing').hide();
$('#overlay').hide();
console.log('disappearing');
}
}
createOverlay();
// Execute the initial check
updateMyTableProcessingVisibility();
});

Hope someone will find it useful!

maniiu

Ciao, any update on that issue? still have 2 calls.. and is pretty annoying as datatables is a great tool, but users must wait a bit too much for the 2 ajax call...
Sorry to bother you guys, but can you point me to a possible solution?

maniiu

I found another solution which I post here for anyone will need a way out from that in the future..
I figured out that storing SearchPanes items in the Browser Session helps to speed up page loading, but I was not happy as 2 ajax calls were in any case duplicating the total time needed.
Then I thought that having the session with a value (for example the number of total items in the table) is also telling me that the page has been loaded at least once..
So I now check simply if I have $_SESSION['totalitems'] variable set, if yes, I load datatable with deferloading:true, if not I leave that empty and let the 2 Ajax calls been fired on the first loading.
<?php if (isset($_SESSION['totalitems'])) print"deferLoading: true,"; ?>

Works like a charme.. Summing up with the 2 solution (Session storage and deferloading) I moved from a loading time of around 10 secs to less than 1 sec..
I really hope that you guys can fix that issue in future releases.
Thanks in any case for the support!
Ciao

allan

Thanks for the update. Good to hear you have a solution!

I think there is a good chance that I'll remove deferLoading in future, and which that won't directly address this issue, it will trigger a different code path to be taken, which I think will in turn resolve it.

Allan

maniiu

Allan any news?

allan

No sorry. This isn't something that I've been able to work on yet. I have however removed deferLoading for DataTables 2, which is still in development.

Allan