How to improve rendering speed with many cols and responsive?

glowfish

Link to test case:

https://estate-insights.com/cockpit_test/test.html

Description of problem:

Hello everyone,

I'm currently facing performance issues (delays) with a) "many" columns (~60-70), b) responsive data table, c) server-side processing and d) row groups. Following my investigation the delays are stemming from the usage of the responsive plugin. Nr of rows varies between 10 and 50.

The more columns I add, the longer the rendering takes.The problem only occurs when responsive is used. At 3 different positions in the code high delays occure. When I say high, I mean 400-1000ms each. Which is from a user experience perspective a quite high delay summing up to 1200-3000ms on top of the ordinary server-side ajax request and rendering of the data table.

All 3 positions in the code seem to trigger the "_fnAdjustColumnSizing" function which leads to the delays.
1) Between the draw-callback and the draw event trigger
2) Between the draw event and the initComplete callback
3) Between the initComplete callback and the processing event (when processing is done)

It seems to me that the column sizing/adjust calculations are executed on ALL columns, not only the ones which are visible. Also, it the same calculation of column width is executed multiple times - which seems unnecessary.

I'm pretty stuck right now and don't see a way to improve performance here. deferRender helped a bit to get current performance measures from even worse performance measures. autoWidth does not have an effect - responsive seems to ignore it.

Please find the quite complex config below.

Any help is very much appreciated!

Kind regards,
Konstantin

colin

Your site is password protected so I can look at the page. That said, this section of the FAQ should help, it discusses various techniques to improve performance,

Cheers,

Colin

glowfish

Sorry, Colin - works now without password protection. Firefox is having some issues, but it works in Chrome and Edge.

BTW: In the console you will see some output: All lines with a leading "B" shows the milliseconds required for executing "_fnAdjustColumnSizing".

Thank you!
Konstantin

colin

It looks to me like the biggest part of the load is before the Ajax success message appears. In the code, you appear to be creating the requirements for the serverSide protocol, rather than the server script doing it. Could you move that load to the server so it sends the data in the correct format?

FYI - The protocol is discussed here. Also see examples here.

Cheers,

Colin

glowfish

Hi Colin,

thanks for getting back!

In this specifc case there is certainly above average load on the server side, yes. There are other examples where the server is pretty fast. But this is a separate issue we are handling.

My questions relates to everything after the Ajax success message. As pointed out in the initial question there are some quite significant delays (or time consuming calculations) during rendering of the responsive table which I would like to solve.

Thanks,
Konstantin

allan

Hi Konstantin,

Thanks for the link and making your site available to us. You are absolutely right - having many columns can be a real performance drag, particularly with Responsive. The way it works is to calculate the column widths and then subtracts each in term from the width available until there is no space left. We could (should) optimise that by doing the calculation and subtraction for each column in turn - I'll look into that.

In the meantime, realistically your users are never going to have 68 columns visible, so are you able to pick a subset - perhaps 5-10, which will be defined in the DataTable? Then you could use a custom renderer for Responsive or do away with Responsive on your table and have custom child rows?

Allan

glowfish

Hi Allan,

thank you for your suggestions!

You are pointing in a direction I already had in my mind: Why are all columns' widths calculated although only a small fraction of them is actually shown? You mentioned a custom renderer for Responsive - any hint how such a renderer would look like to somehow flag columns to be in the details section and not being "width calculated"?

Thanks,
Konstantin

allan

Hi Konstantin,

The key is going to be to not define all 68 columns. Just define 5-10 (or whatever it is you would like to show). For every column you define the calculation will currently be performed, and in general it just takes more time to process even without Responsive.

It is important to know here that even if you don't define the data in a column, DataTables uses the original data object given to it for the row. So if your object as 68 data points, but you only define 5 columns, DataTables will retain all 68 data points, but use only 5 of them. You can see the effect of that in this example. The table only have four columns defined, but the data object for the row has seven data points and Editor uses them.

So what you need to do is just define a few columns for your table and then in your custom renderer use all of the data points in the object to display. It will mean a bit more code than letting DataTables / Responsive show the list, but it will be a heck of a lot faster.

Allan

glowfish

Hi Allan,
that sounds like a splendid approach! Will try and let you know the outcome.
Kind regards,
Konstantin

glowfish

Hi Allan,
so - I believe the results are good now - thanks to your support!
However, the current solution seems a bit "ugly" to me. Problem: In order to force Responsive to show the child row/details including the button in the first column, I need to add an empty column with class="none". And this empty column is deleted in the responsive.display.renderer again.
Is there any way to avoid this and somehow "force" Resposnive to show the child row/details?
Thanks again - that was really helpful! I saved about 2000ms!
Konstantin

colin

I'm not clear why you need an empty column - is there a reason for that? Could you update your site to reflect your current code, please.

Colin

glowfish

Hi Colin,

as Allan suggested I'm only defining the first 6 columns (out of 68). They all fit into the table. I.e. no responsive child row is created and thus no responsive details renderer is executed (where I could add the remaining 62 columns).

What I'm doing right now is creating this artificial empty column with class='none', forcing it to appear in the responsive child row which again triggers the responsive details renderer and, voila, I can add the remaining 62 columns.

Does this explanation help understanding the issue?

Kind regards,
Konstantin

allan

Hi Konstantin,

Rather than using the Responsive extension to render the child rows, you'd be better served using your own child rows in this case I think.

Responsive isn't really designed to be used to show extra data when there are no columns hidden - you'd need to use a hack similar to what you have done to make it do anything.

Allan