Search Builder improvements & React integration

choc

Link to test case: https://datatables.net/extensions/searchbuilder/examples/initialisation/buttonOptions.html
Description of problem:

The current Search Builder is very powerful and can perform very complex searches, as well as support deep grouping searches.

Below is some feedback based on my experience of using it:
1.

The searchBuilder.liveSearch (Example page) currently supports the search until the Search button is pressed. I have noticed some behavior that is not to be expected:

When clicking on the And (Or) logic on the left, the search is performed regardless of liveSearch: false.
Also, if two criteria are present and one criterion is removed or indented, the search is performed again.

I would expect all searches to only be performed when the search button is pressed when using liveSearch: false

2.

The current Search Builder is too complex to use. That doesn't mean it's not good, but from a user's aspect, I would expect it to be easy to understand and simple to use. So I think it would be nice to have a simpler version.
Similar to Alternative Pagination to provide an alternative Search Builder that is free of the depth of the groups (searchBuilder.depthLimit) (indent) to reduce logical complexity and reduce maintenance time.
And a visual example of a simpler version looks like this:

source: https://table.sadmn.com
So each row has its own logic of And or Or instead of having the logic in the group.

3.

The current search generator counts the number of criteria as soon as they are added (regardless of whether the criteria are completely filled in or not). I would suggest to update the count only if the criterion is validated (filled in). This is particularly useful if liveSearch: false is used. So if no criteria are valid, disable the “Search” button and enable it when criteria are valid.

I'm currently working on the React integration for Search Builder, as the current Search Builder is quite complex and I'm considering making the UI clearer by making it less complex.

I managed to extract the column data and condition data for each column and render the components accordingly.

And this is what it would look like when clicking on + Add filter

The number of criteria should be updated in the text of Filter button, I just haven't implemented it yet.
If the criteria are valid, the count in the button text is updated, e.g:

This is currently just a layout and the search function is not yet linked. As I still need some time to figure out how some of the logic in the source code works and what I want to integrate is different from the current logic.

(I guess the CSS for the Search Builder in the example is somehow broken).

I think the Search Builder is a killer that can do such great complex searches. All it needs is a good looking/easy to use interface and it is very suitable for React integration. (headless search builder?)

allan

1) Good point. Sounds like a bug. I've added it to my tracker.

2) I've wondered a lot about the tree view of SearchBuilder and I'm sure there is room for improvement. The main goal was to capture the grouping logic. SearchBuidler can create some complex expressions due to its nesting of multiple groups, and the current UI reflects that. For a single level it is too complex, but for multiple groups / levels I think it works fairly well. But as I say, I'm sure it can be improved - perhaps a more traditional tree view (a la Reddit) might work).

3) Applied criteria count - I agree and that is something I want to address in future.

The error with the layout (buttons overlapping) is one that is fixed in the nightly I think and should be released soon.

Allan

choc

Hi Allan,

Could you give me a hint on how to achieve one-level logic?
For example, AND for the first row, OR for the second row, and so on?
Is it possible to do this with minor modifications?

As for the populated value, in the soruce code: https://github.com/DataTables/SearchBuilder/blob/1c3cd71189edc83adbab148090a4e963f8a86872/src/criteria.ts#L2694

How can I extract the populated value based on the condition selected?

Since I only need to extract the select options for conditions of ["=", "!="], so that I can render the React Select component later.

As for the render of Input components for the conditions of ["starts", "!starts", "contains", "!contains", "ends", "!ends"] and ["<", "<=", ">=", ">"] and ["between", "!between"], I don't need to extract anything.

Currently I add a snippet code after these lines to extract the text value.

    // Initialise the value elements based on the condition
    this.dom.value = [].concat(this.s.conditions[this.s.condition].init(
        this,
        Criteria.updateListener,
        loadedCriteria !== undefined ? loadedCriteria.value : undefined
    ));

    if (loadedCriteria !== undefined && loadedCriteria.value !== undefined) {
        this.s.value = loadedCriteria.value;
    }

    // Extract values
    const populatedValues = this.dom.value.map(valueElement => $(valueElement).text()); // Extract text values
    console.log(populatedValues);
    // [ "Ashton CoxCedric KellyGarrett WintersTiger Nixon"]

But I think this is not a good idea. I still need time to understand the source code.

For extracting column data (source code), it is easier to get the column data because the column data for the select options and data format are quite easy to extract. (these part)

choc

I found there's a possible redundant line in the source code which causes the _populateValue method to be triggered twice.

As it will be initialized later in these lines.

After removing this line (L2385):

this.dom.value = [].concat(this.s.conditions[this.s.condition].init(this, Criteria.updateListener));

from the _clearValue() method. The _populateValue method is not triggered twice and it seems that the interface still works. (I haven't fully tested it yet. But after making some Condition select option changes, I haven't noticed any issues for the populated Value select options).

choc

Regarding the extraction of populated values. I end up adding a new key called values inside here

        this.s = {
            condition: undefined,
            conditions: {},
            data: undefined,
            dataIdx: -1,
            dataPoints: [],
            dateFormat: false,
            depth: depth,
            dt: table,
            filled: false,
            index: index,
            liveSearch: liveSearch,
            origData: undefined,
            preventRedraw: false,
            serverData: serverData,
            topGroup: topGroup,
            type: '',
            value: [],
            values: [] // NEW, I add this to get the select options of populated values
        };

and I push the options into my custom values inside here with:

            if (added.indexOf(val) === -1) {

                that.s.values.push({value: val, text: text}); // NEW, push options

                added.push(val);
                options.push(opt);

                if (preDefined !== null && Array.isArray(preDefined[0])) {
                    preDefined[0] = preDefined[0].sort().join(',');
                }

                // If this value was previously selected as indicated by preDefined, then select it again
                if (preDefined !== null && opt.val() === preDefined[0]) {
                    opt.prop('selected', true);
                    el.removeClass(Criteria.classes.italic);
                    that.dom.valueTitle.removeProp('selected');
                }
            }

And clear the values inside each init functions I need:
e.g., in initSelect and initInput and init2Input and initDate and initNoValue and init2Date with:

            that.s.values = [];

And later on fetch the values using:

                const searchBuilder = dt.settings()[0]._searchBuilder;
                const topGroup = searchBuilder.s.topGroup;
                const values = topGroup.s.criteria[0].criteria.s.values;

(I don't know why the structure looks so nested. But luckily I can still get the values from it.)

Note: I was using the Button plugin with Search Builder and render the React components inside the source code: Buttons/js/dataTables.buttons.js, the variable dt refers to this line

The result:

The biggest challenge now would be the single level logic and the corresponding search.

allan

I have to say, that looks excellent

Allan

choc

@allan Do you have time to have a look at my comments above about the possible redundant code in the source code and question?

Also I found that this search method is called twice after deleting filtering rule.

perhaps also a redundant call? But I haven't figured out why it is called twice and what the purpose of this is.

allan

The floating button is something that I've fixed in a recent commit and will be released soon.

It does sound like the call to search() is redundant. I'll need to make some time to look into that, but I'm not sure when I'll get to that yet I'm afraid. Quite a backlog building up at the moment

Allan

choc

Hi @allan

After the actual implementation, I realized that it is not realistic to have an individual logic in each condition (criterion). It makes no sense to use different logic in each condition (criterion), e.g. an “AND” logic in the first condition and an “OR” logic in the second condition. (Silly me)

So I re-designed the UI as follows:

where it has no sub grouping, the same as using depthLimit: 1

So... I've been thinking too much before.

Thank you for all your support so far. Now I got a working simplified Search Builder for React.

allan

Yeah, the nesting and different logic per group complicate things. It can make for some very powerful queries though. I do like your UI.

Allan