Millions of rows in searchbuilder values - Possible server side?

mete12

Hello @allan ,

I'm using datatables server side options for a long time. I want to integrate searchPanes to a few tables and searchBuilder to others. Assume in a filter we have 1 million potential values. So what is the scalable approach here? Dumping the values into filter using Ajax method? Would it kill the browser? I have seen examples with 50.000 but they have at most 10.000 values inside filters.

Or is there any method where we can load the filter values dynamically as well. Such as when a search query is provided it is sent to server or if customer scrolls, we load the filter values in batches.

Thanks

allan

I'd say no question - go with server-side processing. If you don't, you need to transfer all of the data to the client-side as a single JSON object / array (I guess you could chunk it if needed, but the user would see it loading in chunks). How big a file would that be? It depends on your data, but maybe 3MB or more? That is the slow part - the actual Javascript execution will be a lot faster, finite still obviously, and more noticeable with more rows, but the data transfer is what will take up the most time I expect.

Allan

mete12

Hey @allan ,

I'm ok to go with the server side option. For table data we are already using server side. I see the documentation as well. My problem here the values inside filters and how to go server side while getting those values in chunk. I don't want to put 1 million product name into that filtering section.

allan

You could disable the pane for that column perhaps? Or perhaps SearchBuilder might be better suited for your use case. Or possibly a custom filter with something like Select2 for auto complete.

Allan

mete12

My existing solution is based on select2. and customer uses those fields for filtering so I cannot disable it.

SearchBuilder has the same problem right? Any way to feed the values of it server side?

Actually we liked the searchbuilder and searchpanes since they are well integrated to the table. Is there a way to use select2 inside searchbuilder for example to make filters scalable?

Edit: I have seen this page https://datatables.net/extensions/searchbuilder/examples/customisation/plugin and checking if I it works for me.

mete12

hey @allan ,

I changed select2 to server side and accept multiple selections. I also changed the datatables to serverside. So how would you change condition functions in this case?

I changed inputValue to send all the selected values to the server. Could you help me with the rest?

https://live.datatables.net/meyuzisa/1/

allan

Yup, so if you have server-side processing enabled, then you need to have the server-side do the filtering. Let's assume you've got your value from Select2, you'll want to redraw the table with that filter applied.

To redraw, call draw(), that (when server-side processing is enabled) will trigger an Ajax request to get the data. You need to send the selected filter value(s) to the server so it can act on it. Use ajax.data to do that. There is an example here.

Allan