A lot of forced reflows slowing down the rendering

sebastianbarth

I am currently trying to improve the performance of the table rendering because we have trouble with long rendering times (5-15s for ca. 5-10k cells, depending on the hardware).

Dependencies:

"datatables.net": "2.1.4",
"datatables.net-dt": "2.1.4",
"datatables.net-fixedcolumns": "5.0.0",
"datatables.net-scroller": "2.4.3",
"datatables.net-scroller-dt": "2.4.3",

Therefore I have profiled it in our use case and found a lot of trashed layouts, mostly caused by styling changes, followed by layout property accesses which in turn require the table to render to provide values based on the new changes, just to be followed by other changes.

While some of the unnecessary layouts are caused by our code (onFirstTableDraw, triggered by 'draw.dt'), mainly by using the DT API (possibly wrongfully), I identified two calls of _fnScrollDraw (one as callback for _fnDraw and one explicit call under _fnInitComplete). Could you shortly explain, why both are necessary there?

Do you see any other issues worth investigating?

allan

Could you shortly explain, why both are necessary there?

Not off the top of my head - sorry. I'll need to remind myself as to what is going on there, and it is possible there are optimisation paths that can be made. I'd welcome a link to a test case showing this please.

Thanks,
Allan

sebastianbarth

I could eliminate our own bottleneck. Flame chart looks like this now:

Now I am working on a preresentative example. On jsfiddle is fine?

sebastianbarth

With the following example (Watched on a 4k screen or scaled accordingly) we have good example for a similar configuration using the same plugins. It lacks the surrounding CSS of our App, as well as some specific styles for the table, some glue code and node manipulation (CSS and content) bound to createCell callback and such.

https://jsfiddle.net/w5406n31/5/

In our tests this renders 6-10 times faster than in our code, mainly due to layouting/recalculation of style taking a tenth of what it takes in our app. Combined with two more forced reflows caused by our code (questions to this later). The render time multiplies quickly. One clear factor is the number of cells rendered per page.

While we are actively trying to find out why our layouting and style calculation is that expensive and if this can be improved at all and how, we also aim to reduce the number of forced reflows as a factor. Here we hope you can provide some hints and possibly improve the library.

We have already marked some blocks on the flame chart as candidates.

---

I guess walking through the issues one by one, makes the most sense.

One big reflow that we want to get rid off is caused by a repositioning of the .dt-scroll-head as first element of the .dt-scroll-body to visually make the scrollbar of the body move up to also cover the vertical range of the header and footer:

We do so with the following pseudo code on the first draw.dt event:

scrollBody.prepend(scrollHead);
scrollBody.append(scrollFoot);

// Move body table down to avoid getting hidden behind header due to scroller plugin making it positioned absolutely
DataTables_Table_1.style.transform = `translateY('${px(scrollHead.getBoundingClientRect().height)}')`;

Naturally this causes a reflow of the whole body table.

How could we do this instead, and most important when? If we can't get around moving elements in the tree we would have to move it somehow near the addition of the cells to the body table so that no layouting in between would be invalidated by it.

Thank you for any hint!
Regards, Sebastian

allan

If you are willing to have the scrollbar go over the header as well - don't enable the scrollX / scrollY options in DataTables at all. Rather make the div wrapper around the table scrollable and make the header elements position: sticky: https://live.datatables.net/maqewebu/1/edit .

You'll save a whole ton of DOM calculations (which is what causes the reflows) and performance will be better.

I haven't been willing to compromise on having the scroll bar go over the header / footer, which is why it is not implemented that way in DataTables core - it still uses the (ancient?) method I came up with about 15 years ago! I don't like it - it makes things slow and column width calculations are a nightmare. But the UI of a scrollbar over an area that doesn't scroll seems wrong to me.

Allan

REJIS

Might be too low level for some, but most of the users that use our software didn't like the checkboxes...tiny and using FontAwesome/Glyphs/etc for those and the sort arrows cause issues every once in a while (Sometimes they are partially loaded in or something and Greek Letters and such show instead of the intended symbols).

I put real checkboxes there which needs to be kept in sync with selection state, and I tie it to a data property so in my data object. I ran into issues with all the redraws manipulating the data property with data() on every selection change (Both by the user and when loading the table) since it would redraw when I saved data back for each row. Instead I now use the rows iterator and get to my data essentially with context.aoData[index]._aData then I set the data and invalidate the row letting it redraw at the end. This sped things up pretty quick having just the one redraw. Might be helpful if there was another version of data() that doesn't redraw to allow us to change multiple pieces of data then draw ourselves at the end. Could be helpful internally to redraw as less as possible too.

Now, the other HTML and Style manipulation being done this wouldn't help on. Browsers really complain about having to render sections again when changed by JavaScript. I try to stay away from inline styles too...especially when done with JavaScript. Always works better to have things in CSS to allow it to layer instead of injecting it at runtime.

allan

@REJIS - I'm not clear on how checkboxes relate to this discussion? Also Select does use real checkboxes (it didn't use to, it has since v2 though).

Allan

sebastianbarth

@allan scrollY seems to be required by the scroller plugin. Could you extend the example, if there is a way?

I haven't been willing to compromise on having the scroll bar go over the header / footer, which is why it is not implemented that way in DataTables core

Having scrollbar cover sticky areas is more expected than having it limited to just the table body, especially accessibility-wise. Having three different tables is the next problem. CSS position:sticky would help accessibility and semantically correct tables, too.

allan

scrollY seems to be required by the scroller plugin

Yes. The whole point of Scroller is that it performs virtual scrolling for large data sets. The blog post introducing Scroller explains how it works.

There is no way at the moment to use Scroller without enabling the scrollY option.

The three table solution is far from ideal, and is easily the part of the DataTables code that I most dislike working with - I'd love to rip it out. However, that scrollbar over the top of the header when the header doesn't move, nope, thus far, as I say, that isn't a UI / UX that I've been willing to implement. To my mind it looks horrible.

Allan