Reaction of DataTables server-side on a 302 from server

Marian_Kechlibar

I am experimenting with a Docker-based solution that runs a PHP application over FrankenPHP. FrankenPHP uses Caddy internally as a server.

Unlike Apache, Caddy will often respond to the AJAX request for server-side data by a 302 or 303 redirect, immediately followed by the correct JSON with server-side data. Nevertheless, DataTables doesn't take kindly to the redirect and triggers the error

https://datatables.net/manual/tech-notes/7

indicating that it didn't follow through with the redirect, but ended up in the error handler.

Unfortunately, I cannot link to any test case, given that this is a code under heavy development and not ready to be deployed publicly yet. But I wonder if DT could be made to react to 302/303 redirects gracefully.

kthorngren

Looks like Datatables starts with this base Ajax config:

        var baseAjax = {
            "url": typeof ajax === 'string' ?
                ajax :
                '',
            "data": data,
            "success": callback,
            "dataType": "json",
            "cache": false,
            "type": oSettings.sServerMethod,
            "error": function (xhr, error) {
                var ret = _fnCallbackFire( oSettings, null, 'xhr', [oSettings, null, oSettings.jqXHR], true );
    
                if ( ret.indexOf(true) === -1 ) {
                    if ( error == "parsererror" ) {
                        _fnLog( oSettings, 0, 'Invalid JSON response', 1 );
                    }
                    else if ( xhr.readyState === 4 ) {
                        _fnLog( oSettings, 0, 'Ajax error', 7 );
                    }
                }
    
                _fnProcessingDisplay( oSettings, false );
            }
        };

Datatables uses jQuery ajax() to fetch the data. I suspect there is an HTTP error in the response causing the error handler to be called. The ajax docs state that you can override the ajax options except success. Possibly you can override the error object to diagnose the error. See the jQuery ajax() docs for the parameters passed into the error function.

Let us know what you find.

Kevin

kthorngren

Also I would try using curl to determine if the error is caused by something Datatables is doing.

Kevin

Marian_Kechlibar

So, I added the following error code to my ajax definition:

               ajax: {
                    url: {$link_val},
                    data: {
                        data_type: {$data_type}
                    },
                    error: function (xhr, textStatus, errorThrown) {
                        alert('DataTables AJAX error ' + textStatus + ", error thrown: " + errorThrown);
                    }
                },

and the output is as attached.

I can see the communication in my Developer Tools. The server sends a 303 and the browser replies immediately to this redirect, receiving a well-formed definition of (empty) data set to draw:

It seems to me that my original diagnosis was correct: Caddy, for some reason, replies with a redirect to the original AJAX request from DataTables and DataTables expects a 2xx response instead.

I only don't know why Caddy is doing that. My experience with Caddy as a HTTP server is about three days in total. I used to work with Apache for decades, but Caddy is new to me. I would abandon it instead of trying to solve that, but the trouble is, in my specific situation, its performance is much better than Apache's.

kthorngren

@allan can comment on whether Datatables checks the status code or other checks before throwing the Ajax error. But this example shows a 301 redirect works:
https://live.datatables.net/socirone/1/edit

And this server side processing example shows a 307 redirect works:
https://live.datatables.net/yaroyala/1/edit

This leads me to believe, maybe incorrectly, that Datatables doesn't just only expect 2xx resposnes.

Did you try using curl? Or possibly jQuery ajax()? You may need to pass draw: 1 as a parameter for your server side processing script.

I don't have a way to generate 302 or 303 responses to test with. Might need to wait for Allan to respond if you don't find the same issue with curl.

Kevin

Marian_Kechlibar

What precisely should I try with Curl? I can do it, but I don't know where to begin.

The intriguing part is that this piece of web code has been under development since May 2022. For about 900 days, under Apache, I have literally never seen this behavior ever, on multiple test servers, Docker or not-Docker.

But once I started fiddling with FrankenPHP / Caddy on this Friday, hey presto, it happens all the time.

kthorngren

I'm not familiar with caddy but possibly this is causing the redirect?
https://caddyserver.com/docs/caddyfile/directives/redir

Kevin

Marian_Kechlibar

Well, it doesn't look like that, my Caddyfile is extremely simple. I suspect it has something to do with the total amount of workers in that FrankenPHP instance.

The 302 behavior doesn't start immediately after bootup of the container, only after a few request/response pairs. And it started happening more frequently when I reduced the total amount of workers to 8.

Might be a caching problem, too.

kthorngren

Sorry I can't help more as I don't have a way to generate 302 or 303 responses. @allan will need to comment if the ajax error is due specifically to these status codes.

Maybe Stack OVerflow or the Caddy community can help debug why Caddy is sending the redirects. If the behavior of the web server changes as time goes on then maybe this is the problem that needs to be solved.

Kevin

Marian_Kechlibar

I will try to put some example online... and I will ask at the Caddy community too.

allan

Can you show me the request URL and also the 302 redirect response from the server (ideally all of the headers, but just the Location one will do if needed.

There are some cases whereby a 302 won't be followed by the browser for Ajax - e.g. from an https source being redirected to an http one. That should show as an error on the browser's console I would think.

I'm a little confused as to why some requests to the URL would result in 302, and some wouldn't (presumably resulting in a 200 OK?). Is it the same URL you are requesting? If it gives a 302, where is it sending you - perhaps doing some kind of load balancing?

Allan

Marian_Kechlibar

I will try to put a working example online today or tomorrow. I am now just waiting for someone else to prepare a working Docker server for me, and they promised to do this today.

Marian_Kechlibar

So, I was able to set up a working example, but I have to send you, @allan, a username/password combo. That application isn't prepared for anonymous use, unfortunately.

I sent it to you through a message.

kthorngren

I think you have a misconfiguration of the Caddy web server. Both 301 Moved Permanently and 302 Found require a location header to indicate the URL to redirect to. Here is a 301 response ( I tried to change all domain and IP addresses info ):

Summary
URL: https:// xxx.yyy.net/user/request/list?locale=en&draw=1&columns%5B0%5D%5Bdata%5D=id&columns%5B0%5D%5Bname%5D=&columns%5B0%5D%5Bsearchable%5D=true&columns%5B0%5D%5Borderable%5D=true&columns%5B0%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B0%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B1%5D%5Bdata%5D%5B_%5D=created.display&columns%5B1%5D%5Bdata%5D%5Bsort%5D=created.timestamp&columns%5B1%5D%5Bname%5D=&columns%5B1%5D%5Bsearchable%5D=true&columns%5B1%5D%5Borderable%5D=true&columns%5B1%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B1%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B2%5D%5Bdata%5D=method&columns%5B2%5D%5Bname%5D=&columns%5B2%5D%5Bsearchable%5D=true&columns%5B2%5D%5Borderable%5D=true&columns%5B2%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B2%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B3%5D%5Bdata%5D=source_id&columns%5B3%5D%5Bname%5D=&columns%5B3%5D%5Bsearchable%5D=true&columns%5B3%5D%5Borderable%5D=true&columns%5B3%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B3%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B4%5D%5Bdata%5D=direction&columns%5B4%5D%5Bname%5D=&columns%5B4%5D%5Bsearchable%5D=true&columns%5B4%5D%5Borderable%5D=true&columns%5B4%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B4%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B5%5D%5Bdata%5D=relative_url&columns%5B5%5D%5Bname%5D=&columns%5B5%5D%5Bsearchable%5D=true&columns%5B5%5D%5Borderable%5D=true&columns%5B5%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B5%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B6%5D%5Bdata%5D=duration_s&columns%5B6%5D%5Bname%5D=&columns%5B6%5D%5Bsearchable%5D=true&columns%5B6%5D%5Borderable%5D=true&columns%5B6%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B6%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B7%5D%5Bdata%5D=http_code&columns%5B7%5D%5Bname%5D=&columns%5B7%5D%5Bsearchable%5D=true&columns%5B7%5D%5Borderable%5D=true&columns%5B7%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B7%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B8%5D%5Bdata%5D=files_traffic&columns%5B8%5D%5Bname%5D=&columns%5B8%5D%5Bsearchable%5D=true&columns%5B8%5D%5Borderable%5D=false&columns%5B8%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B8%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B9%5D%5Bdata%5D=actions&columns%5B9%5D%5Bname%5D=&columns%5B9%5D%5Bsearchable%5D=true&columns%5B9%5D%5Borderable%5D=false&columns%5B9%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B9%5D%5Bsearch%5D%5Bregex%5D=false&order%5B0%5D%5Bcolumn%5D=0&order%5B0%5D%5Bdir%5D=desc&order%5B0%5D%5Bname%5D=&start=0&length=50&search%5Bvalue%5D=&search%5Bregex%5D=false&data_type=any&_=1732564707772
Status: 301
Source: Network
Address: 1.2.3.4:443
Initiator: 
jquery.min.js:2:80630

Request
:method: GET
:scheme: https
:authority: xxx.yyy.net
:path: /user/request/list?locale=en&draw=1&columns%5B0%5D%5Bdata%5D=id&columns%5B0%5D%5Bname%5D=&columns%5B0%5D%5Bsearchable%5D=true&columns%5B0%5D%5Borderable%5D=true&columns%5B0%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B0%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B1%5D%5Bdata%5D%5B_%5D=created.display&columns%5B1%5D%5Bdata%5D%5Bsort%5D=created.timestamp&columns%5B1%5D%5Bname%5D=&columns%5B1%5D%5Bsearchable%5D=true&columns%5B1%5D%5Borderable%5D=true&columns%5B1%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B1%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B2%5D%5Bdata%5D=method&columns%5B2%5D%5Bname%5D=&columns%5B2%5D%5Bsearchable%5D=true&columns%5B2%5D%5Borderable%5D=true&columns%5B2%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B2%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B3%5D%5Bdata%5D=source_id&columns%5B3%5D%5Bname%5D=&columns%5B3%5D%5Bsearchable%5D=true&columns%5B3%5D%5Borderable%5D=true&columns%5B3%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B3%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B4%5D%5Bdata%5D=direction&columns%5B4%5D%5Bname%5D=&columns%5B4%5D%5Bsearchable%5D=true&columns%5B4%5D%5Borderable%5D=true&columns%5B4%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B4%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B5%5D%5Bdata%5D=relative_url&columns%5B5%5D%5Bname%5D=&columns%5B5%5D%5Bsearchable%5D=true&columns%5B5%5D%5Borderable%5D=true&columns%5B5%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B5%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B6%5D%5Bdata%5D=duration_s&columns%5B6%5D%5Bname%5D=&columns%5B6%5D%5Bsearchable%5D=true&columns%5B6%5D%5Borderable%5D=true&columns%5B6%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B6%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B7%5D%5Bdata%5D=http_code&columns%5B7%5D%5Bname%5D=&columns%5B7%5D%5Bsearchable%5D=true&columns%5B7%5D%5Borderable%5D=true&columns%5B7%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B7%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B8%5D%5Bdata%5D=files_traffic&columns%5B8%5D%5Bname%5D=&columns%5B8%5D%5Bsearchable%5D=true&columns%5B8%5D%5Borderable%5D=false&columns%5B8%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B8%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B9%5D%5Bdata%5D=actions&columns%5B9%5D%5Bname%5D=&columns%5B9%5D%5Bsearchable%5D=true&columns%5B9%5D%5Borderable%5D=false&columns%5B9%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B9%5D%5Bsearch%5D%5Bregex%5D=false&order%5B0%5D%5Bcolumn%5D=0&order%5B0%5D%5Bdir%5D=desc&order%5B0%5D%5Bname%5D=&start=0&length=50&search%5Bvalue%5D=&search%5Bregex%5D=false&data_type=any&_=1732564707772
Accept: application/json, text/javascript, */*; q=0.01
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Cookie: PHPSESSID=a0rqqj0r0dm2nnbc0dqmo3c5vp; _nss=1
Host: xxx.yyy.net
Referer: https://xxx.yyy.net/user/request/list?locale=en
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2.1 Safari/605.1.15
X-Requested-With: XMLHttpRequest

Response
:status: 301
Alt-Svc: h3=":443"; ma=2592000
Cache-Control: no-store, no-cache, must-revalidate
Content-Encoding: gzip
Content-Length: 2131
Content-Type: application/json; charset=utf-8
Date: Mon, 25 Nov 2024 19:58:28 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Server: Caddy
Set-Cookie: _nss=1; path=/; secure; HttpOnly; SameSite=Strict
Set-Cookie: PHPSESSID=a0rqqj0r0dm2nnbc0dqmo3c5vp; expires=Mon, 09 Dec 2024 19:58:27 GMT; Max-Age=1209600; path=/; secure; HttpOnly; SameSite=Strict
Vary: X-Requested-With, Accept-Encoding
X-Frame-Options: SAMEORIGIN
x-powered-by: PHP/8.3.7

Query String Parameters
locale: en
draw: 1
columns[0][data]: id
columns[0][name]
columns[0][searchable]: true
columns[0][orderable]: true
columns[0][search][value]
columns[0][search][regex]: false
columns[1][data][_]: created.display
columns[1][data][sort]: created.timestamp
columns[1][name]
columns[1][searchable]: true
columns[1][orderable]: true
columns[1][search][value]
columns[1][search][regex]: false
columns[2][data]: method
columns[2][name]
columns[2][searchable]: true
columns[2][orderable]: true
columns[2][search][value]
columns[2][search][regex]: false
columns[3][data]: source_id
columns[3][name]
columns[3][searchable]: true
columns[3][orderable]: true
columns[3][search][value]
columns[3][search][regex]: false
columns[4][data]: direction
columns[4][name]
columns[4][searchable]: true
columns[4][orderable]: true
columns[4][search][value]
columns[4][search][regex]: false
columns[5][data]: relative_url
columns[5][name]
columns[5][searchable]: true
columns[5][orderable]: true
columns[5][search][value]
columns[5][search][regex]: false
columns[6][data]: duration_s
columns[6][name]
columns[6][searchable]: true
columns[6][orderable]: true
columns[6][search][value]
columns[6][search][regex]: false
columns[7][data]: http_code
columns[7][name]
columns[7][searchable]: true
columns[7][orderable]: true
columns[7][search][value]
columns[7][search][regex]: false
columns[8][data]: files_traffic
columns[8][name]
columns[8][searchable]: true
columns[8][orderable]: false
columns[8][search][value]
columns[8][search][regex]: false
columns[9][data]: actions
columns[9][name]
columns[9][searchable]: true
columns[9][orderable]: false
columns[9][search][value]
columns[9][search][regex]: false
order[0][column]: 0
order[0][dir]: desc
order[0][name]
start: 0
length: 50
search[value]
search[regex]: false
data_type: any
_: 1732564707772

I don't see a location header. I don't see one in the 302 response either. It should look something like this:

HTTP/1.1 302 Found
Location: http://some-other-url

The browser is not able to follow the redirect because the location header does not exist. Possibly there is a misconfig causing the 302 response when there shouldn't be a 301 or 302. Or the config issue is with adding the location header. I didn't find anything useful regarding Caddy and the missing location header.

Kevin

kthorngren

See the Redirections in HTTP for info of how they work.

Kevin

kthorngren

Take a look at the browser's network inspector for this example I posted before:
https://live.datatables.net/socirone/1/edit

It has a 301 Moved Permanently response. In the response headers you can see the location header is present:

location: https://live.datatables.net/examples/ajax/data/objects.txt

Kevin

Marian_Kechlibar

Thank you for your analysis. Certainly looks weird. I will try to produce some samples using curl and submit them to a Caddy discussion.

Marian_Kechlibar

Looking into the problem in more detail, it looks like a subtle bug in my worker code, where a Dependency Injection Container doesn't get fully re-set between requests.

Responses like 301 or 302 or 303 shouldn't contain a body, but here, a 3xx response comes with the actual JSON body.

It is very obviously not a DataTables problem, not really even a Caddy problem, but a DI problem. The framework I am using is not yet fully capable of resetting containers. Thank you for leading me to the correct root of the problem.

allan

Thanks for letting us know. Good luck tracking it down.

Allan