way to batch importing to stay under the max_input_vars

crcucb

I used https://editor.datatables.net/examples/extensions/import as a reference to create an import process, however I am getting, "Too many rows edited at the same time (tech info: max_input_vars exceeded)."

I may not have the ability to expand max_input_vars because of where the server is hosted.
I am trying to find a workaround without retooling a bunch of stuff.

Is there a way to auto-batch through the data being uploaded so it only uploads X number of records at a time?

allan

Hi,

There is no built in option for that specifically I'm afraid. What would need to happen is to use ajax as a function to split the payload over multiple requests (probably in a queue rather than firing them all off at once) before then finally calling the callback function to tell Editor that it has been updated. You'd need to keep a hold of the responses from the server to build up the data array as well. Using this route you may wish to do a validation submit before a write submit, to make sure all of the data is valid before writing anything to the db.

That all said, there is another option - ajax.submitAs. You could have the client submit the entire data set as the request body, rather than as parameters.

The upshot is that on the server-side you would need to use something like:

parse_str( file_get_contents('php://input'), $postData );
$json = json_decode($postData, true);

Then do ->process($json) rather than ->process($_POST).

The second option is definitely easier! Depending on how much data you are submitting you might need to watch out for post_max_size and also processing duration though.

Allan

crcucb

Is there a working example of utilizing ajax.submitAs and the PHP processing?

allan

I'm afraid there isn't, no. However, the edit.php script in this example (which you would need to access from the download package, it isn't available on that page), uses the php://input method to read a request body.

It should be just as I've described - use:

ajax: {
  submitAs: 'json',
  url: ...
}

And then the PHP code from above. Possible you might need to check for $postData being empty before attempting to JSON decode it.

Allan

crcucb

I am trying to implement this. I am able to read but when I try to save an edit , I get an error. DevMode shows:
HTTP Error 405.0 - Method Not Allowed
The page you are looking for cannot be displayed because an invalid method (HTTP verb) is being used.

Below is the call to put:

        edit: {
            type: 'PUT',
            url: './php/voting2_edit_test.php',
            submitAs: 'json'
        },

below is the php:

<?php

/*
 * Example PHP implementation used for the REST 'create' interface.
 */
session_start();

error_reporting(E_ALL);
ini_set('display_errors', 'On');


include_once 'php_functions.php';
include_once ( "lib/config.php" );

$instanceDateTime = new DateTime();
$instanceDateTime->setTimezone(new DateTimeZone('America/New_York'));


include( "votingupload2_test.php" );


// The REST example uses 'PUT' for the input, so we need to get the 
// parameters being sent to us from php://input
parse_str( file_get_contents('php://input'), $args );

$json = json_decode($args, true);

$editor
->debug(true) // Add this line
  ->process($json)
    ->data();

// If there is an error, indicate it like a REST service would with a status code
if ( isset($data['fieldErrors']) && count($data['fieldErrors']) ) {
    http_response_code( 400 );
}

echo json_encode( $data );

I'm not seeing any errors in the server logs

crcucb

Actually I found this error:

Fatal error: Uncaught TypeError: json_decode(): Argument #1 ($json) must be of type string, array given in C:\ClientSites\da-pool.com\jyorkejudge4.org\php\voting2_edit_test.php:26 Stack trace: #0 C:\ClientSites\da-pool.com\jyorkejudge4.org\php\voting2_edit_test.php(26): json_decode() #1 {main} thrown in C:\ClientSites\da-pool.com\jyorkejudge4.org\php\voting2_edit_test.php on line 26

allan

Did you try checking for the content being empty as I suggested above? That might help:

$json = $args
  ? json_decode($args, true)
  : [];

Allan

crcucb

Yes, but I am not sure if control is being passed to voting2_edit_test.php as I am writing to my log file early in the code, and I don't see any entries in my log.

Below is what my AJAX calls should look like?

const editor = new DataTable.Editor({
  ajax: {
        create: {
            type: 'POST',
            url: './php/voting2_create_test.php',
            submitAs: 'json'
        },
        edit: {
            type: 'PUT',
          submitAs: 'json',
            url: './php/voting2_edit_test.php'
        },
        remove: {
            type: 'DELETE',
            url: './php/voting2_remove_test.php',
            submitAs: 'json'
        }
    },
    

I didn't see submitAs used in the examples: https://editor.datatables.net/examples/advanced/REST.html