Lower the initialization time for long, scrollable tables

ngerstner

The tables, I want to show on my page, can have a large amount of columns (>300). To have a chance of inspecting them, I enabled scrolling via scrollX. This feature works nicely, however for large tables, the initialization time of the table is huge. In an example of 360 visible columns (having 20.000 rows and showing 5 of them per page), the table took 2 minutes to initialize. There is a note related to the problem in the FAQ (https://datatables.net/faqs/index#speed), but it only states that the problem exists and provides no solution.

My question: Is there a feature that initially only renders the first n columns and loads further columns as a user scrolls through the table? This would be nice, as initializing tables with around 50 columns is quite fast.

Thank you for your time!

colin

You can't stop the columns being loaded, but you can use serverSide to reduce the number of rows. Scroller also works with that, as shown in this example.

Colin

tangerine

it only states that the problem exists and provides no solution.

I don't understand why you say that.
The link at https://datatables.net/faqs/index#speed begins with this:

There are several ways in which you can speed up DataTables.

and goes on with more details.

ngerstner

@colin Thank you for your comment. I know about the serverSide option, but as you wrote, it only reduces the number of rows. I think the number of columns is the critical part in my application, though I will try and toy with reducing the number of rows.

ngerstner

@tangerine The link starts with the advise to enable pagination. Then, the speed problem from my question is mentioned:

Its also worth noting that scrolling (scrollX and scrollY) can have a significant impact on draw performance for tables with lots of columns...

but no speedup advice is given. Afterwards, the different methods to load data are discussed.

Am I missing something in the FAQ?

allan

Is there a feature that initially only renders the first n columns and loads further columns as a user scrolls through the table?

No sorry. We can virtualise the creation of rows when using Scroller as @colin says, but there is no such feature in DataTables at this time for column I'm afraid. The closest would be to use Responsive rather than scrollX (which isn't really the same thing at all).

The real issue with scrollX is that it needs to use Javascript to keep the header and body columns aligned. The process of reading and then writing the widths is really slow, particularly for bigger tables. Also, even if we could virtualise the columns, we'd need to know the width of the columns in advance (similar to how Scroller requires all rows to be the same height).

One option that might be viable for you, might be to just stick the table into an overflow: auto div. Then in the table (without `-init scrollX) is too wide, the browser will still show it scrollable and the header will align correctly.

The only reason we don't do that in DataTables is because it doesn't work for Y scrolling - the header scrolls off! We could use FixedHeader for that, but, it's just another trade-off.

Choices, choices... :-).

Allan

ngerstner

@allan Thank you for your comments! I will give it a shot, as I'm currently not using scrollY. In the meantime, I have implemented a custom "column pagination" in which the full data ist stored in a class and a simplified table is destroyed and recreated if further columns are requested.

I'm pretty sure, my quick implementation is not ideal, but such a feature (maybe with sth smarter than destroying and recreating) could also be something for your library.

Thank you for creating and maintaining DataTables,
Nico