Huge performance decrease after updating to datatables 1.11.3

trongart

I've updated my project from dt-1.10.24 to dt-1.11.3. Keeping everything the same, the overall filter performance has decreased 2-3x, which is a huge difference.

Data based on: Ajax, 9000 rows, 48 columns, serverSide false, deferRender true.

Observations:

• Filters from buttons and searchBuilder take 2-3 seconds longer to filter (1.10.24 almost instant).

• Initial table loading takes 2-4 seconds longer (1.10.24 takes 3 seconds to load).

• No longer possible to see the text in the search filter while typing as filtering takes longer unless I wait for 1 second or more (1.10.24 shows text and results instantly).

As I cannot show this in a test case, is there anything that particularly affects performance in the latest release dt-1.11.3 that I could optimise/deactivate to bring it back to the same speed as 1.10.X? What could I try?

colin

We wouldn't have expected any performance change - as part of our test suite we do run tests nightly to ensure there's no noticeable decrease in performance. Curiously, no-one else has reported an issue like this with the 1.11.x releases - and given how heavily used DataTables is, if it was a generic error I would've expected to hear other similar cases.

If you could create an isolated test case, that would definitely be something we'd be interested in seeing and would be keen to help diagnose.

Colin

trongart

@colin Thank you very much for your feedback. I've sent you a direct message regarding an isolated test case.

allan

Fantastic - thank you for the test case.

I'm reasonably sure that a large part of the performance hit is due to the call of .child.show() on each row inside rowCallback. One of the new features in 1.11 was that we save the state of the child row visibility, so what is happening now is that on each row display, there is that hit, compounded with your custom state save parameters function (delete isn't a fast operation - normally it isn't a problem, but when used lots of times in a loop, the effects are noticeable since the Javascript engine can't optimise for it).

I'm thinking we might need to introduce some kind of throttle into the state saving in DataTables to counter this kind of thing.

To confirm this is the issue, could you possible disable state saving on your 1.11 table and see how that then performs?

Thanks,
Allan

trongart

@allan Thank you so much for looking into this! Is it possible to stop "saving the state of the child row visibility" as part of the stateSave config?

I added 3 more isolated test cases for 1.11 with the following changes in our direct message conversation:
1) "stateSave": false
2) "stateSave": false without custom state save parameters and without initComplete with their triggers
3) "stateSave": true without rowCallback

colin

You could remove that info like you did in your other thread,

Colin

trongart

To summarise: Having discussed this with @sandy and @colin, the conclusion is that the performance decrease in datatables 1.11 to 1.10 comes from using .child.show() on each row inside my rowCallback.

I added a datatables 1.11 test case which uses the same rowCallback option as in my project: live.datatables.net/cijewifi/1/edit

My goal is to have the button "Hide All Child Rows" which shows/hides the child rows and as far as I know this is only possible with the code within rowCallback. The full functionality is included in my test case which performs fast in datatables 1.10, but much slowlier in 1.11 affecting the overall performance.

Is there maybe an alternative way to get the same functionality without having to use rowCallback? If there is, it would resolve the issue with datatables 1.11.

colin

It would probably be more efficient to just open/close the child rows when the button is pressed, rather than considering the state on each draw.

Colin

trongart

@colin this sounds fantastic. Just unclear how this can be done all at once without rowCallback. Would this also work with the format function as in my test case?

colin

This example closes the opened child row, only allowing a single child row to be opened at any one time. You would need to do something like that,

Colin

trongart

@colin Thank you so much for your advice and help here. I managed to get all the required functionality without rowCallback as in this test case: live.datatables.net/gajejege/1/edit

However, there is a considerable performance hit when I click on "Open Child Rows" or "Show All Child Rows" when the table is large (9000+ rows and 15+ columns), the plugin hangs up.

In this thread, @kthorngren suggested to use rowCallback instead as in this test case due to a timing issue with inserting child rows into the dom. This solution performed well and instant with large tables in datatables 1.10.

However, with datatables 1.11 this approach has a performance hit as per @allan so not sure if there is any other way to approach opening all child rows without rowCallback unless there is a way to fix this within the actual datatables 1.11 library.

I would appreciate any suggestions.

kthorngren

I updated the test case to have 10000 rows of data.
http://live.datatables.net/vawetupu/1/edit

Before you run the test case open the browser's console. The createdRow option is iterating all the rows building the child rows. This will cause an initial delay with displaying the table. Now when clicking the button you will see it takes some time for rowCallback to iterate the 10 rows on the page.

Here is the same test case with 1.10.18:
http://live.datatables.net/cijewifi/3/edit

rowCallback is much faster. Not sure what has changed.

One thing that seems to help is to remove the createdRow option and call the format function in rowCallback. I updated my first example with this:
http://live.datatables.net/zuxurece/1/edit

Not sure if its needed but I added row().child.remove() to make sure there are no memory leaks by using dtRow.child(format(dtRow.data()), 'no-padding').show(); in rowCallback.

Kevin

trongart

@kthorngren Really appreciate your help with this! I applied the same changes as in your last example to my project (Ajax, 9000 rows, 48 columns, serverSide false, deferRender true) by removing createdRow and modifying rowCallback. The performance hit is still there, unfortunately coming from rowCallback.

I think it is still necessary to find an alternative way to hide/show child rows without rowCallback. As @allan mentioned, the performance hit on 1.11 derives from "saving the state of the child row visibility". Not sure how to prevent that except for a fix within the 1.11 library.

kthorngren

I missed the comment from Allan about using sateSave. The array of saved child rows does seem to increase as you page through the table.

Maybe an alternative is to use the draw event. In the event use rows().every() to loop through the open rows to close and remove them. Then loop through the rows on the page and open them. With 10 rows displayed this will result in 20 row iterations. It appears that the childRows, in the stateSave object, only increases to 10 (or the page length). Does this help with your solution?
http://live.datatables.net/zuxurece/3/edit

Kevin

trongart

@kthorngren Appreciate the feedback! Made a small change to your test case by getting the needed result with the Hide/Show button: live.datatables.net/cehojipa/1/edit

Applied the same in my project by using draw and removing createdRow and rowCallback. Unfortunately, there is still the same performance kill.

kthorngren

Sounds like there is more to saving the child visibility than just iterating rows that have active child. Thought maybe row().child.remove() might help with that but I guess not

Maybe a flag to stateSave the child rows is best.

Kevin

trongart

@allan Do you think it may be possible to add a stateSave flag for child rows into datatables 1.11 as @kthorngren suggested to resolve this performance kill?

trongart

By the way- The strange thing is also that the performance kill persists when stateSave: false, but the flag might still help.

kthorngren

By the way- The strange thing is also that the performance kill persists when stateSave: false,

You can see the delay here with stateSave false:
http://live.datatables.net/vawetupu/2/edit

Are you still creating all the child rows using createdRow?

Kevin

trongart

No, I removed createdRow. The performance kills occurs both by using the rowCallback solution and also with draw and rows().every() (without createdRow and rowCallback).

kthorngren

Sounds strange. Just for fun have you tried clearing the browser's cache?

Kevin

trongart

Yes, I have and also tried this on separately created pages.

rf1234

Unfortunately I also have serious performance issues after upgrading from Data Tables 1.10.21 to 1.11.4. The load time of one relatively complex but not very large data table (not very many rows) is over 60% longer now - and that is with State Saving turned off!

I only upgraded because I want to use the new Editor with field type "dataTable". The download builder doesn't allow me to select an older version of data tables. Can I use an older version of Data Tables jointly with the latest Editor version? If so, how do I do this?

allan

Have you tried the recently released 1.11.5? It has a punch of performance improvements in it which should bring it (almost) back to the 1.10.x speeds.

Allan

rf1234

ok, will try that first.