Populating datatable with FTP directory images

chirathh

I am populating a datatables table with images on a FTP server. I'm declaring my TRs and TDs and a TR has 5 TDs. I have PHP code to formulate these TRs and TDs. Datatables is working perfectly with this, but I have 80,000 images and it takes forever to load and crashes. When I limit the images to around 4000, it loads just fine. I can't use the server side processing method since the images aren't coming from a sql table. I'm getting the images like this:

function getAllImgs($directory){
          $resizedFilePath = array();
          foreach ($directory AS $dir) {
            foreach (glob($dir . '*.jpg') as $filename) {
                array_push($resizedFilePath, $filename);
            }
        }
          return $resizedFilePath;
        }

I'm then using a foreach loop to add the filepaths to TDs. How can I load the whole 80,000 images without crashing the page?

tangerine

How does it "crash"? What error message?

chirathh

@tangerine it just doesn't load the table because the data is too many. Maybe it will load after a while, but it's not good enough. On the console it shows this error for every image :
GET http://site/directory/image.jpg net::ERR_INSUFFICIENT_RESOURCES
It takes a very long time to load and I've cancelled the loading at around 30,000 errors. When I limit the images, the error doesn't show up at all.

tangerine

That's looking like an "insufficient memory" problem. It is not an error message generated by DataTables.
I have had a lot of trouble myself with PHP's memory-hungry handling of large arrays. You could try increasing available memory with ini_set(().
However, 80,000 records is a lot for DataTables unless you use server-side processing.

chirathh

@tangerine yeah memory problem it is, coz of the sheer number of images. Do you think server side processing can be used for my table, since it's not getting the data from a sql table?

allan

Do you think server side processing can be used for my table, since it's not getting the data from a sql table?

You can potentially implement server-side processing without an SQL server. You just need to meet the requirements here. It isn't something I've tried myself and I don't have an example for it though I'm afraid.

Allan