Reaction of DataTables server-side on a 302 from server

Reaction of DataTables server-side on a 302 from server

Marian_KechlibarMarian_Kechlibar Posts: 9Questions: 1Answers: 1

I am experimenting with a Docker-based solution that runs a PHP application over FrankenPHP. FrankenPHP uses Caddy internally as a server.

Unlike Apache, Caddy will often respond to the AJAX request for server-side data by a 302 or 303 redirect, immediately followed by the correct JSON with server-side data. Nevertheless, DataTables doesn't take kindly to the redirect and triggers the error

https://datatables.net/manual/tech-notes/7

indicating that it didn't follow through with the redirect, but ended up in the error handler.

Unfortunately, I cannot link to any test case, given that this is a code under heavy development and not ready to be deployed publicly yet. But I wonder if DT could be made to react to 302/303 redirects gracefully.

This question has an accepted answers - jump to answer

Answers

  • kthorngrenkthorngren Posts: 21,391Questions: 26Answers: 4,961

    Looks like Datatables starts with this base Ajax config:

            var baseAjax = {
                "url": typeof ajax === 'string' ?
                    ajax :
                    '',
                "data": data,
                "success": callback,
                "dataType": "json",
                "cache": false,
                "type": oSettings.sServerMethod,
                "error": function (xhr, error) {
                    var ret = _fnCallbackFire( oSettings, null, 'xhr', [oSettings, null, oSettings.jqXHR], true );
        
                    if ( ret.indexOf(true) === -1 ) {
                        if ( error == "parsererror" ) {
                            _fnLog( oSettings, 0, 'Invalid JSON response', 1 );
                        }
                        else if ( xhr.readyState === 4 ) {
                            _fnLog( oSettings, 0, 'Ajax error', 7 );
                        }
                    }
        
                    _fnProcessingDisplay( oSettings, false );
                }
            };
    

    Datatables uses jQuery ajax() to fetch the data. I suspect there is an HTTP error in the response causing the error handler to be called. The ajax docs state that you can override the ajax options except success. Possibly you can override the error object to diagnose the error. See the jQuery ajax() docs for the parameters passed into the error function.

    Let us know what you find.

    Kevin

  • kthorngrenkthorngren Posts: 21,391Questions: 26Answers: 4,961

    Also I would try using curl to determine if the error is caused by something Datatables is doing.

    Kevin

  • Marian_KechlibarMarian_Kechlibar Posts: 9Questions: 1Answers: 1

    So, I added the following error code to my ajax definition:

                   ajax: {
                        url: {$link_val},
                        data: {
                            data_type: {$data_type}
                        },
                        error: function (xhr, textStatus, errorThrown) {
                            alert('DataTables AJAX error ' + textStatus + ", error thrown: " + errorThrown);
                        }
                    },
    

    and the output is as attached.

    I can see the communication in my Developer Tools. The server sends a 303 and the browser replies immediately to this redirect, receiving a well-formed definition of (empty) data set to draw:

    It seems to me that my original diagnosis was correct: Caddy, for some reason, replies with a redirect to the original AJAX request from DataTables and DataTables expects a 2xx response instead.

    I only don't know why Caddy is doing that. My experience with Caddy as a HTTP server is about three days in total. I used to work with Apache for decades, but Caddy is new to me. I would abandon it instead of trying to solve that, but the trouble is, in my specific situation, its performance is much better than Apache's.

  • kthorngrenkthorngren Posts: 21,391Questions: 26Answers: 4,961

    @allan can comment on whether Datatables checks the status code or other checks before throwing the Ajax error. But this example shows a 301 redirect works:
    https://live.datatables.net/socirone/1/edit

    And this server side processing example shows a 307 redirect works:
    https://live.datatables.net/yaroyala/1/edit

    This leads me to believe, maybe incorrectly, that Datatables doesn't just only expect 2xx resposnes.

    Did you try using curl? Or possibly jQuery ajax()? You may need to pass draw: 1 as a parameter for your server side processing script.

    I don't have a way to generate 302 or 303 responses to test with. Might need to wait for Allan to respond if you don't find the same issue with curl.

    Kevin

  • Marian_KechlibarMarian_Kechlibar Posts: 9Questions: 1Answers: 1

    What precisely should I try with Curl? I can do it, but I don't know where to begin.

    The intriguing part is that this piece of web code has been under development since May 2022. For about 900 days, under Apache, I have literally never seen this behavior ever, on multiple test servers, Docker or not-Docker.

    But once I started fiddling with FrankenPHP / Caddy on this Friday, hey presto, it happens all the time.

  • kthorngrenkthorngren Posts: 21,391Questions: 26Answers: 4,961

    I'm not familiar with caddy but possibly this is causing the redirect?
    https://caddyserver.com/docs/caddyfile/directives/redir

    Kevin

  • Marian_KechlibarMarian_Kechlibar Posts: 9Questions: 1Answers: 1

    Well, it doesn't look like that, my Caddyfile is extremely simple. I suspect it has something to do with the total amount of workers in that FrankenPHP instance.

    The 302 behavior doesn't start immediately after bootup of the container, only after a few request/response pairs. And it started happening more frequently when I reduced the total amount of workers to 8.

    Might be a caching problem, too.

  • kthorngrenkthorngren Posts: 21,391Questions: 26Answers: 4,961
    edited November 24

    Sorry I can't help more as I don't have a way to generate 302 or 303 responses. @allan will need to comment if the ajax error is due specifically to these status codes.

    Maybe Stack OVerflow or the Caddy community can help debug why Caddy is sending the redirects. If the behavior of the web server changes as time goes on then maybe this is the problem that needs to be solved.

    Kevin

  • Marian_KechlibarMarian_Kechlibar Posts: 9Questions: 1Answers: 1

    I will try to put some example online... and I will ask at the Caddy community too.

  • allanallan Posts: 63,602Questions: 1Answers: 10,486 Site admin

    Can you show me the request URL and also the 302 redirect response from the server (ideally all of the headers, but just the Location one will do if needed.

    There are some cases whereby a 302 won't be followed by the browser for Ajax - e.g. from an https source being redirected to an http one. That should show as an error on the browser's console I would think.

    I'm a little confused as to why some requests to the URL would result in 302, and some wouldn't (presumably resulting in a 200 OK?). Is it the same URL you are requesting? If it gives a 302, where is it sending you - perhaps doing some kind of load balancing?

    Allan

  • Marian_KechlibarMarian_Kechlibar Posts: 9Questions: 1Answers: 1

    I will try to put a working example online today or tomorrow. I am now just waiting for someone else to prepare a working Docker server for me, and they promised to do this today.

  • Marian_KechlibarMarian_Kechlibar Posts: 9Questions: 1Answers: 1

    So, I was able to set up a working example, but I have to send you, @allan, a username/password combo. That application isn't prepared for anonymous use, unfortunately.

    I sent it to you through a message.

  • kthorngrenkthorngren Posts: 21,391Questions: 26Answers: 4,961
    edited November 25

    I think you have a misconfiguration of the Caddy web server. Both 301 Moved Permanently and 302 Found require a location header to indicate the URL to redirect to. Here is a 301 response ( I tried to change all domain and IP addresses info ):

    Summary
    URL: https:// xxx.yyy.net/user/request/list?locale=en&draw=1&columns%5B0%5D%5Bdata%5D=id&columns%5B0%5D%5Bname%5D=&columns%5B0%5D%5Bsearchable%5D=true&columns%5B0%5D%5Borderable%5D=true&columns%5B0%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B0%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B1%5D%5Bdata%5D%5B_%5D=created.display&columns%5B1%5D%5Bdata%5D%5Bsort%5D=created.timestamp&columns%5B1%5D%5Bname%5D=&columns%5B1%5D%5Bsearchable%5D=true&columns%5B1%5D%5Borderable%5D=true&columns%5B1%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B1%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B2%5D%5Bdata%5D=method&columns%5B2%5D%5Bname%5D=&columns%5B2%5D%5Bsearchable%5D=true&columns%5B2%5D%5Borderable%5D=true&columns%5B2%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B2%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B3%5D%5Bdata%5D=source_id&columns%5B3%5D%5Bname%5D=&columns%5B3%5D%5Bsearchable%5D=true&columns%5B3%5D%5Borderable%5D=true&columns%5B3%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B3%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B4%5D%5Bdata%5D=direction&columns%5B4%5D%5Bname%5D=&columns%5B4%5D%5Bsearchable%5D=true&columns%5B4%5D%5Borderable%5D=true&columns%5B4%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B4%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B5%5D%5Bdata%5D=relative_url&columns%5B5%5D%5Bname%5D=&columns%5B5%5D%5Bsearchable%5D=true&columns%5B5%5D%5Borderable%5D=true&columns%5B5%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B5%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B6%5D%5Bdata%5D=duration_s&columns%5B6%5D%5Bname%5D=&columns%5B6%5D%5Bsearchable%5D=true&columns%5B6%5D%5Borderable%5D=true&columns%5B6%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B6%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B7%5D%5Bdata%5D=http_code&columns%5B7%5D%5Bname%5D=&columns%5B7%5D%5Bsearchable%5D=true&columns%5B7%5D%5Borderable%5D=true&columns%5B7%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B7%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B8%5D%5Bdata%5D=files_traffic&columns%5B8%5D%5Bname%5D=&columns%5B8%5D%5Bsearchable%5D=true&columns%5B8%5D%5Borderable%5D=false&columns%5B8%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B8%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B9%5D%5Bdata%5D=actions&columns%5B9%5D%5Bname%5D=&columns%5B9%5D%5Bsearchable%5D=true&columns%5B9%5D%5Borderable%5D=false&columns%5B9%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B9%5D%5Bsearch%5D%5Bregex%5D=false&order%5B0%5D%5Bcolumn%5D=0&order%5B0%5D%5Bdir%5D=desc&order%5B0%5D%5Bname%5D=&start=0&length=50&search%5Bvalue%5D=&search%5Bregex%5D=false&data_type=any&_=1732564707772
    Status: 301
    Source: Network
    Address: 1.2.3.4:443
    Initiator: 
    jquery.min.js:2:80630
    
    Request
    :method: GET
    :scheme: https
    :authority: xxx.yyy.net
    :path: /user/request/list?locale=en&draw=1&columns%5B0%5D%5Bdata%5D=id&columns%5B0%5D%5Bname%5D=&columns%5B0%5D%5Bsearchable%5D=true&columns%5B0%5D%5Borderable%5D=true&columns%5B0%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B0%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B1%5D%5Bdata%5D%5B_%5D=created.display&columns%5B1%5D%5Bdata%5D%5Bsort%5D=created.timestamp&columns%5B1%5D%5Bname%5D=&columns%5B1%5D%5Bsearchable%5D=true&columns%5B1%5D%5Borderable%5D=true&columns%5B1%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B1%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B2%5D%5Bdata%5D=method&columns%5B2%5D%5Bname%5D=&columns%5B2%5D%5Bsearchable%5D=true&columns%5B2%5D%5Borderable%5D=true&columns%5B2%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B2%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B3%5D%5Bdata%5D=source_id&columns%5B3%5D%5Bname%5D=&columns%5B3%5D%5Bsearchable%5D=true&columns%5B3%5D%5Borderable%5D=true&columns%5B3%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B3%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B4%5D%5Bdata%5D=direction&columns%5B4%5D%5Bname%5D=&columns%5B4%5D%5Bsearchable%5D=true&columns%5B4%5D%5Borderable%5D=true&columns%5B4%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B4%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B5%5D%5Bdata%5D=relative_url&columns%5B5%5D%5Bname%5D=&columns%5B5%5D%5Bsearchable%5D=true&columns%5B5%5D%5Borderable%5D=true&columns%5B5%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B5%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B6%5D%5Bdata%5D=duration_s&columns%5B6%5D%5Bname%5D=&columns%5B6%5D%5Bsearchable%5D=true&columns%5B6%5D%5Borderable%5D=true&columns%5B6%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B6%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B7%5D%5Bdata%5D=http_code&columns%5B7%5D%5Bname%5D=&columns%5B7%5D%5Bsearchable%5D=true&columns%5B7%5D%5Borderable%5D=true&columns%5B7%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B7%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B8%5D%5Bdata%5D=files_traffic&columns%5B8%5D%5Bname%5D=&columns%5B8%5D%5Bsearchable%5D=true&columns%5B8%5D%5Borderable%5D=false&columns%5B8%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B8%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B9%5D%5Bdata%5D=actions&columns%5B9%5D%5Bname%5D=&columns%5B9%5D%5Bsearchable%5D=true&columns%5B9%5D%5Borderable%5D=false&columns%5B9%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B9%5D%5Bsearch%5D%5Bregex%5D=false&order%5B0%5D%5Bcolumn%5D=0&order%5B0%5D%5Bdir%5D=desc&order%5B0%5D%5Bname%5D=&start=0&length=50&search%5Bvalue%5D=&search%5Bregex%5D=false&data_type=any&_=1732564707772
    Accept: application/json, text/javascript, */*; q=0.01
    Accept-Encoding: gzip, deflate, br
    Accept-Language: en-US,en;q=0.9
    Connection: keep-alive
    Cookie: PHPSESSID=a0rqqj0r0dm2nnbc0dqmo3c5vp; _nss=1
    Host: xxx.yyy.net
    Referer: https://xxx.yyy.net/user/request/list?locale=en
    Sec-Fetch-Dest: empty
    Sec-Fetch-Mode: cors
    Sec-Fetch-Site: same-origin
    User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2.1 Safari/605.1.15
    X-Requested-With: XMLHttpRequest
    
    Response
    :status: 301
    Alt-Svc: h3=":443"; ma=2592000
    Cache-Control: no-store, no-cache, must-revalidate
    Content-Encoding: gzip
    Content-Length: 2131
    Content-Type: application/json; charset=utf-8
    Date: Mon, 25 Nov 2024 19:58:28 GMT
    Expires: Thu, 19 Nov 1981 08:52:00 GMT
    Pragma: no-cache
    Server: Caddy
    Set-Cookie: _nss=1; path=/; secure; HttpOnly; SameSite=Strict
    Set-Cookie: PHPSESSID=a0rqqj0r0dm2nnbc0dqmo3c5vp; expires=Mon, 09 Dec 2024 19:58:27 GMT; Max-Age=1209600; path=/; secure; HttpOnly; SameSite=Strict
    Vary: X-Requested-With, Accept-Encoding
    X-Frame-Options: SAMEORIGIN
    x-powered-by: PHP/8.3.7
    
    Query String Parameters
    locale: en
    draw: 1
    columns[0][data]: id
    columns[0][name]
    columns[0][searchable]: true
    columns[0][orderable]: true
    columns[0][search][value]
    columns[0][search][regex]: false
    columns[1][data][_]: created.display
    columns[1][data][sort]: created.timestamp
    columns[1][name]
    columns[1][searchable]: true
    columns[1][orderable]: true
    columns[1][search][value]
    columns[1][search][regex]: false
    columns[2][data]: method
    columns[2][name]
    columns[2][searchable]: true
    columns[2][orderable]: true
    columns[2][search][value]
    columns[2][search][regex]: false
    columns[3][data]: source_id
    columns[3][name]
    columns[3][searchable]: true
    columns[3][orderable]: true
    columns[3][search][value]
    columns[3][search][regex]: false
    columns[4][data]: direction
    columns[4][name]
    columns[4][searchable]: true
    columns[4][orderable]: true
    columns[4][search][value]
    columns[4][search][regex]: false
    columns[5][data]: relative_url
    columns[5][name]
    columns[5][searchable]: true
    columns[5][orderable]: true
    columns[5][search][value]
    columns[5][search][regex]: false
    columns[6][data]: duration_s
    columns[6][name]
    columns[6][searchable]: true
    columns[6][orderable]: true
    columns[6][search][value]
    columns[6][search][regex]: false
    columns[7][data]: http_code
    columns[7][name]
    columns[7][searchable]: true
    columns[7][orderable]: true
    columns[7][search][value]
    columns[7][search][regex]: false
    columns[8][data]: files_traffic
    columns[8][name]
    columns[8][searchable]: true
    columns[8][orderable]: false
    columns[8][search][value]
    columns[8][search][regex]: false
    columns[9][data]: actions
    columns[9][name]
    columns[9][searchable]: true
    columns[9][orderable]: false
    columns[9][search][value]
    columns[9][search][regex]: false
    order[0][column]: 0
    order[0][dir]: desc
    order[0][name]
    start: 0
    length: 50
    search[value]
    search[regex]: false
    data_type: any
    _: 1732564707772
    

    I don't see a location header. I don't see one in the 302 response either. It should look something like this:

    HTTP/1.1 302 Found
    Location: http://some-other-url
    

    The browser is not able to follow the redirect because the location header does not exist. Possibly there is a misconfig causing the 302 response when there shouldn't be a 301 or 302. Or the config issue is with adding the location header. I didn't find anything useful regarding Caddy and the missing location header.

    Kevin

  • kthorngrenkthorngren Posts: 21,391Questions: 26Answers: 4,961

    See the Redirections in HTTP for info of how they work.

    Kevin

  • kthorngrenkthorngren Posts: 21,391Questions: 26Answers: 4,961

    Take a look at the browser's network inspector for this example I posted before:
    https://live.datatables.net/socirone/1/edit

    It has a 301 Moved Permanently response. In the response headers you can see the location header is present:

    location: https://live.datatables.net/examples/ajax/data/objects.txt
    

    Kevin

  • Marian_KechlibarMarian_Kechlibar Posts: 9Questions: 1Answers: 1

    Thank you for your analysis. Certainly looks weird. I will try to produce some samples using curl and submit them to a Caddy discussion.

  • Marian_KechlibarMarian_Kechlibar Posts: 9Questions: 1Answers: 1
    Answer ✓

    Looking into the problem in more detail, it looks like a subtle bug in my worker code, where a Dependency Injection Container doesn't get fully re-set between requests.

    Responses like 301 or 302 or 303 shouldn't contain a body, but here, a 3xx response comes with the actual JSON body.

    It is very obviously not a DataTables problem, not really even a Caddy problem, but a DI problem. The framework I am using is not yet fully capable of resetting containers. Thank you for leading me to the correct root of the problem.

  • allanallan Posts: 63,602Questions: 1Answers: 10,486 Site admin

    Thanks for letting us know. Good luck tracking it down.

    Allan

Sign In or Register to comment.