large table (20k rows) is loading approx. 35 seconds
large table (20k rows) is loading approx. 35 seconds
culter
Posts: 102Questions: 24Answers: 0
Hi, I'm happy user of DataTables, but recently I uploaded to my MySQL database significantly more data. I had 400 rows, now I have 20k. I'm using Python/Flask and the page is loading extremely slow. The server load is very low during this process (no server processing is activated).
I'm afraid the problem is with the code. At first the data are loading to html page without DataTables and the DataTables functionality with pagination, column sorting etc. are applied after all the data are uploaded to the browser. I'm using the for loop to add the data from mysql to html table. It wasn't big deal when there were 400 rows, but now it's useless.
I don't know how to inicialize the DataTables to apply at the start or do you know some other way to speed up the loading time? Or do you know about some template with large table to review? Thank you!
This question has an accepted answers - jump to answer
Answers
Hi @culter ,
Looping and adding rows individually is going to be slow. This section of the FAQ should help - it gives suggestions on how to improve performance.
Cheers,
Colin
Thank you colin. So you suggested, that adding rows individually is absolutely useless in this amount of data? I will have about 150k rows in my table in the near future. I've read the FAQ, but I'm not sure which solution is better. I have many server resources. Will it help if I'll let the for loop in html and use the server-side processing or not?
I found this video [https://youtube.com/watch?v=OPE5U25AO0U] where is used json. I think it could be suitable for me to download data from time to time, save it as json and then grab it and display it in html. Do you think it will be usable, I mean that It will display the data within 1-2 seconds? Thank you
Hi @culter ,
Yep, if you ignore DataTables, and just create a table within a browser using a for loop, then that would give you a rough idea of the worst case scenario for the table creation.
Json is the format for the data being sent. Take a look at the examples in this section, you'll see the format that's expected. You will need to have a server-side script but there's example you can use.
Cheers,
Colin