large set of data

jijli

I have a large set of data (about 20,000 rows) saved in json format.
var oTable = $('#example').dataTable( {
"bProcessing": true,
"sAjaxSource": "supplier.txt",
"bDeferRender": true,
:
All sorting, filtering, pagination are made by using datatables in client side. Everything is fine.

The only problem is when the page is first loaded, it is very slow. Nothing displayed in the page but the table header and footer.
I am wondering if we can let it display the first page while it is loading the rest?

I can put total row number in the beginning of the file. But I don't want to do server side processing for sorting and filtering.
Any idea?

Thanks!

allan

The problem with just showing the first page is that you need the full data set to know what the first page is - i.e. taking into account any initial sorting and filtering. It is possible to just load the first 10 records if you know what they are going to be and then make an Ajax call yourself to load the rest of them - that would significantly speed things up (use fnAddData to add new data).

If you are able to give us a link I'd be interested in profile the application so I can see where any speed improvements can be made in DataTables.

Allan

jijli

Hi Allan,
Thanks for the response. I just started using datatables, not familiar with it.

You mentioned I need the full data set to know what the first page is. Currently I just passed the whole data set to the client. If I just passed the first few pages in, when and how I pass the rest data ?
When end user presses the next page, will the data table call another new datasource?

Today I did more research from your site. It seems virtual scrolling is good for big set of data. What do you think to use virtual scrolling or pagination? If I use virtual scrolling, may I still use those core functions (like search in each columns..)?

Also in my datasource, I added some html tag for displaying images. I think it increased the amount of data loading from server side to client side. I am thinking I may do that in the client side by using fnRender.

But anyway, I think if I can just load one page first, and then load the rest in the backgroup, it may solve the problem. If you could help me to do this or give me some reference on doing this, that will be great.


Thanks!