datatables and Memcached, Redis

javismiles

Good day,
I use regularly Memcached to cache my database queries, great speed gains,
is it possible to connect memcached with datatables? and if so, any examples?
best

allan

Plain old DataTables doesn't really care where you get your data from, as long as it can read it (typically JSON). So if you can output JSON from your scripts, then yes, it would work just fine.

Allan

javismiles

thank you Allan,
yes I get that, but Im using the datatables PHP editor library, so I guess I would need to understand where I can plug memcached into there, because I know how to do it with my own php scripts, easy, but not sure where that would go within the editor php library, i would have to get inside the code, mmm

javismiles

if I understand well, this is where in the php editor library the json is obtained and sent back to javascript, somehow i would have to plugin though the memcached way before, before the whole process is executed,
the key is here:

$editor->process( $_POST );
$editor->json();

question, if I do

$thejson= $editor->json();

will I get in $thejson the json that is being sent back? because then I could put it then in memcache and then do

///// GET FROM MEMCACHE
$memkey="getdb";
if ($cacheAvailable == true) {
    $fromstore=$memcache->get($memkey); 
    if ($fromstore) {echo $fromstore;return;}
}

   $editor->process( $_POST );
   $tosend=$editor->json();

   if ($cacheAvailable == true) {$memcache->set($memkey, $tosend,$cachedur);}






 */
public function json ( $print=true )
{
    if ( $print ) {
        $json = json_encode( $this->_out );

        if ( $json !== false ) {
            echo $json;
        }
        else {
            echo json_encode( array(
                "error" => "JSON encoding error: ".json_last_error_msg()
            ) );
        }

        return $this;
    }
    return json_encode( $this->_out );
}

javismiles

I Think I managed to install the memcache, I have to confirm that its working, but no errors, for those that may be interested in it, I put the code here, as this, if it works, its a huge benefit when used with any DB stuff:

define('MEMCACHED_HOST', '127.0.0.1');
define('MEMCACHED_PORT', '11211');
$memcache = new Memcached;
$cacheAvailable = $memcache->addServer(MEMCACHED_HOST, MEMCACHED_PORT);
$cachedur=30;

then at bottom of php:

$memkey="db".$etype;
if ($cacheAvailable == true) {
$fromstore=$memcache->get($memkey);
if ($fromstore) {echo $fromstore;return;}
}

$editor->process( $_POST );
$tosend=$editor->json();

and then inside Editor.php:

public function json ( $print=true )
{

    global $cacheAvailable;
    global $memcache;
    global $etype;
    global $cachedur;

    if ( $print ) {
        $json = json_encode( $this->_out );

        if ( $json !== false ) {
            echo $json;
        }
        else {
            echo json_encode( array(
                "error" => "JSON encoding error: ".json_last_error_msg()
            ) );
        }

        return $this;
    }

    $memkey="db".$etype;
    $tosend=json_encode( $this->_out );
    if ($cacheAvailable == true) {$memcache->set($memkey, $tosend,$cachedur);}

    return json_encode( $this->_out );
}

javismiles

im just not sure its working, i have to verify it

javismiles

I tested and so far its not working, no errors, but its not caching it, not sure why yet mmm

javismiles

I think im putting the code in the wrong place,
I need guidance please as to which part of the php library is starting and finishing the process of gathering the db queries, thank u so much

javismiles

so it works now, it works,
the key is that I was puttting the code in the wrong place of the json function, this position makes it work well:

public function json ( $print=true )
{

    global $cacheAvailable;
    global $memcache;
    global $etype;
    global $cachedur;

    if ( $print ) {
        $json = json_encode( $this->_out );

        if ( $json !== false ) {


            $memkey="db".$etype;
            $tosend=$json;
            if ($cacheAvailable == true) {$memcache->set($memkey, $tosend,$cachedur);}


            echo $json;
        }
        else {
            echo json_encode( array(
                "error" => "JSON encoding error: ".json_last_error_msg()
            ) );
        }

        return $this;
    }


    return json_encode( $this->_out );
}

allan

Thanks for posting back. Good to hear you've got it working now.

Allan

javismiles

thank you Allan,
yes it works but there is a "but",
for this to work properly, the key has to take into account parameters such as:

$start_p=$_POST['start'];
$order_p1=$_POST['order'][0]['column'];
$order_p2=$_POST['order'][0]['dir'];

$memkey="db".$etype."-".$start_p."-".$order_p1."-".$order_p2;

Applying this works so that you can still paginate and sort, however something weird happens, when changing the sorting of a column, after switching twice, the table stops being updated until the key expires. Looking at the network I can see the json being returned being apparently correct, there seems to be some weird interaction with the javascript and then it doesnt update until the key expires, its kind of weird

$processnow=1;

if ($cacheAvailable == true) {
$fromstore=$memcache->get($memkey);
if ($fromstore) {$processnow=0;
echo $fromstore;
return;}
}

if ($processnow==1){
$editor->process( $_POST );
$editor->json();
}

//////////////////////////////////////////////////////////////////////////////////// Editor.php

public function json ( $print=true )
{

    global $cacheAvailable;
    global $memcache;
    global $etype;
    global $cachedur;
    global $memkey;

    if ( $print ) {
        $json = json_encode( $this->_out );

        if ( $json !== false ) {
            $tosend=$json;
            if ($cacheAvailable == true) {$memcache->set($memkey, $tosend,$cachedur);}

            echo $json;
        }

..... etc

javismiles

yeah, ive tried with Redis also, same problem,
it all works except that when coming and going between ASC and DESC of a sorting row or between pages of pagination area, for some reason, it wont update the second time the same key is requested before the key expires

javismiles

yes, a pity, printing the redis or memcached variables on separate pages I can see that all works perfect on the server side, no probs,
for some reason the javascript doesnt wanna update the rows the second time it goes back to the same case

javismiles

I think i will try my best to see if I can provide you a private link with a test case for you to check if possible this issue and the one of the inline editing on compact mobile phone situation as I dont think i can fix this myself

javismiles

how can I send you a private link in a private message? thank you

javismiles

memcache and redis when bserverside is true really dont work well, when bserverside is false then yes, it seems that it can work ok there, a pity though as I wanted it to work with bserverside true

allan

My guess is that the draw parameter is not being updated to match what DataTables sent from the client-side.

To send a PM in the forum, click my user name above and then the Send message button.

Allan

javismiles

thank you Allan, yes got a few things to show u, will prepare some test case soon and send you via PM, thank u again