Datatable taking too long to render the view for 2088 count of data.

narasimhakamath

I have a lot of data that needs to be put in the datatable. The code was written long back that rewriting the code will be a huge task for me to add server side pagination.
So I had applied a filter 'Sub Brand' so that I can reduce the count of data coming in the UI.

The count of data reduced significantly by the ratio, but still there were 2088 data that needs to be populated in the datatable.
However, 2088 was still taking a time of almost 8-10 seconds.

I tried improving the SQL query so that I can just fetch whatever data I need for the particular page. Even though the query executed faster than before, the time taken for the UI to load is still the same, without any significant change.

Could you suggest me a solution for this, excluding server side pagination for now?
If there are no other solutions, that's the one I will probably have to go ahead with, but considering the time and testing it requires, I'm not really sure if my colleagues would suggest that way.

Thanks!

kthorngren

This FAQ presents the options:
https://datatables.net/faqs/index#speed

deferRender may help.

Kevin

narasimhakamath

Hello kthorngren,

I did try those 3 options that were provided in the FAQ section. Unfortunately, none of them seemed to make any difference in the speed.

colin

Hi @narasimhakamath ,

We're happy to take a look, but as per the forum rules, please link to a test case - a test case that replicates the issue will ensure you'll get a quick and accurate response. Information on how to create a test case (if you aren't able to link to the page you are working on) is available here.

Cheers,

Colin

narasimhakamath

Since I was not able to re-create the issue as a test case, due to security reasons and because I'm using CakePHP (which I guess is not supported in those 4 for remotely hosted services) mentioned , I ran the DataTables debugger and got the 6 character debug code. Moreover, when I ran the automation testing, 15 tests passes and 1 was down due to the 100% width styling.

What is happening in my case is that, I am rendering an API response as an HTML tag file, which gets triggered into the DataTable, and then two more API calls are made for the displayed data.

I do have the option to get a JSON response instead of the HTML rendered response, but doing so will require a lot of testing and time as it is a pretty huge module. If no other options are possible, that will be the last plan that we would want to go ahead with.

With the result of the DataTables debugger, I'm sure DataTable is working absolutely fine and that it's the data loading from the API that takes the most of the time, where 3 APIs are fired one after the other on such a huge data.

I do not mean to be impolite without providing test cases as mentioned, but I will be really thankful if you could provide me with a solution regarding the data that is being rendered. I hope you can understand the problem I have with providing the test cases here.

colin

Hi @narasimhakamath ,

The problem is that it's hard to recommend anything without seeing your data and the table declaration. If you're using multiple columns.render for example, that will slow down the initialisation. 2-3k rows isn't a large amount of data, so it's likely to be something that you're doing with that data that's causing the issue. As I said, without seeing it, it's hard to offer solutions.

Cheers,

Colin

narasimhakamath

My API gives a response like this, which is used in another Cake PHP template file (.ctp file).
Once the page loads, two other APIs are fired to check different attributes on the data present in the table.
All this, firing of the API to get the HTML response and the other 2 APIs are the ones that is taking a lot of time.

I have added a JS Bin file might give you an idea on this particular scenario. This is a sample data that comes into the .ctp file to be rendered. Hope this helps.

colin

Thanks @narasimhakamath ,

It's hard to tell from there where the slowness is coming from. It would be worth trying to separate everything out - for example, if you comment out DataTables initialisation so that the table is still created but just not initialised, does it still take a long time?

It would be worth modifying that file so that it all loads up - there's too many errors (missing files for images and libraries) - that way we can tell whether DataTables is the issue.

Cheers,

Colin

narasimhakamath

Thanks @colin ,

Since the codebase is really huge, I couldn't find a way to give you proper information regarding this issue.

How about this?
Suppose I don't want the data to be rendered in the UI, instead I want the data available only when I make a search.

For instance, I don't want the data to be rendered in the UI, but instead I want the corresponding data to be rendered only when I make a search.

The table will be empty with no data, and as I type in the search box provided by DataTables, that is when I want the table to be showing the necessary data related to that search.

If that is possible, this will really help me fasten things up in the UI I hope.

Please let me know if it can be done, or if I'll be wrong in doing so.

colin

Hi @narasimhakamath ,

If you had serverSide processing that could easily done (although you wouldn't need it as the processing would be quick), but with client side processing the table has to be initialised fully for DataTables to know what data there is for the search, so that wouldn't help.

As I said before, if serverSide is a no go, then you should do some profiling to find out what element is running slowly. It shouldn't be DataTables with that number of rows.

Cheers,

Colin

narasimhakamath

I ran the DataTable debugger which I had and ran it on that particular page and this is the response that I got. It takes 10-12 seconds fo the page to be loaded entirely.

Here, 2088 data are rendered in a ctp file which is being imported here.

However, in the below image attached, there are no ctp files being imported or rendered on to the screen. Whatever data that has been coming, is from one master file. 13000 data are rendered in the UI in just seconds.

colin

Hi @narasimhakamath ,

On that bottom image, you've got serverSide enabled, the top image doesn't. With it enabled, only the records being viewed are returned to the client, which means only 10 records are really being displayed - that's why it loads that much faster than the top image where 2088 records are being rendered. As Kevin said in the first reply, deferRender should help...

Cheers,

Colin

narasimhakamath

Hi @colin
So as far as I understood about deferRender, it just displays the data and keeps the rest of the data in the buffer.

In the first screenshot, I'm running the fnRowCallBack to call 3 functions and also rendering two input boxes per row. So in total I will be adding 4000 input boxes for the whole data that I have got.

Is it possible to only render those and make the 3 user defined functions work only for the displayed data and not for the entire 2000 rows?

I believe I can reduce a considerable amount of time by running the row callback function only for the data that I will be displaying.

For the one with server-side pagination, this is what that has been happening, but is it possible to do the same with deferRender as you've mentioned.

Could you please give me a little more knowledge on deferRender and how I can change the DataTable configurations to make this happen.

Thanks,
Narasimha.

kthorngren

Take a look at this thread, it has a discussion regarding deferRender.

rowCallaback should only run for those rows that are being displayed. You can use console.log statements to verify.

You haven't posted your code or provided a test case so we can see what is happening. I would probably start by removing rowCallback and deferRender just to see how long it takes to load the data. If its quicker then start adding pieces back until you find the culprit.

What are you doing in these 3 functions?

Kevin

narasimhakamath

So the fnRowCallback gets executed only for those data that are being displayed in the UI.

What about the rest of the data that hasn't been in the UI yet? Are they also rendered by default or it is rendered only when I go to the next page or when I perform a search operation?

If I could store the rest of the data in the buffer, wouldn't it help me get fasten things up in the page?

narasimhakamath

Can I use the fnDrawCallback to make API calls and fetch only those data that are necessary to be displayed in the page depending upon the page number or the search made?

And then use the fnRowCallback to render only those data which I have received from the fnDrawCallback?

DataTable does that job if I have a server side pagination. But since my code is way too huge, I really don't want to mess things up by changing around the code too much.

If the above method works where I can write a generic API to fetch the required data based on the choice made by the user, I can reduce the time significantly by pulling in less data/load to the specific page.

I will also have to handle the changes made in the row by the user whenever they have entered a value in the input field box that is being generated.

colin

Hi @narasimhakamath ,

This here may help, it's showing when events and callbacks are made. createdRow is called when a row is created, rowCallback is called when a row is drawn - if serverSide is enabled, then these are effectively the same (since everything is recreated on each draw), but, if you're doing local processing, then the more you can put into rowCallback the better, since only the rows being displayed will be effected.

Hope that helps,

Cheers,

Colin