Correct approach for reading around 10000 rows?

Rapid1898

Hello - i have created this solution:
https://homepagelevermann.herokuapp.com/profile
which is reading around 10.000 rows from a mysql-db to a web-app using datatables.

The workflow is as following -
- "normal" GET requesting via server.js => routes => controller
- in the controller the data is read from the mysql-db and stored in an array
- then this array is given to the view (ejs-file) and get rendered

It tooks around 30sec to 60sec till the full data is loaded (and the datatables funcitonality like serach / show entries / page handling) is available.

Is this generally the "correct" approach for reading the data into datatables?
Or am i doing here something generally wrong?

Thanks and KR

tangerine

You already asked this question, and got the answer.
https://datatables.net/forums/discussion/68519/reading-around-10000-datasets-best-practice#latest

Rapid1898

Hello - your are right - i have allready asked the question and its morely an "addon-question"
(sure it maybe would be better to reply in the existing thread and not open a new one - sorry for that...)

Of course i have to dig into the FAQ regarding the speed https://datatables.net/faqs/index#speed

But however - i have now the runtime which is 30-60sec and i only wanted to ask if this is somehow a reasonable loading time or not. And from the experience what would a "normal" loading time for something like that.

tangerine

30-60sec is not a reasonable time for a DataTable of 10,000 rows.
You could try some benchmarking to see where the time is being used.

Rapid1898

OK - thanks for the reply - how can i do a benchmarking to find the source of the delay?

colin

This section of the FAQ should help, it discusses various techniques to improve performance,

Cheers,

Colin

Rapid1898

Ich checked several speedup-possibilities (paging, scroll) - but this makes no difference - it still took around 45sec to load the full table -

My current workflow:
* Currently i am triggering the controller-request (via server.js => routes => controllers)
* which loads the 10000 rows to an array (this took between 3-5 seconds)
* and then this array is given to an ejs-file (with linking to the datatables-file including: "$('#example').DataTable( {") which does the rendering (and this took most of the time around 40sec
* and after that the datatable is available (with search / entries / paging)

Maybe i have to change this to client side processing with ajax sourced data - here my question -
When i understand it right - i still have to read the data with the controller from the db (2nd point above). But how can i give this array / data to the file so it can be loaded like in the example
{
"data": [
// row 1 data source,
// row 2 data source,
// etc
]
}