Evaluating a new data object against the DataTables search algorithm

Evaluating a new data object against the DataTables search algorithm

trumpetinctrumpetinc Posts: 4Questions: 1Answers: 0

I am incorporating websocket/STOMP update notifications to make our ajax based datatables more responsive.

As part of this effort, I would like to determine if an updated data object still matches the DataTables search specification (we are using column based search), and do an in-place replacement if the updated data still meets the search criteria (instead of calling draw and triggering an ajax call).

I prefer to just not call draw() for every update (it causes screen jitter and the drop down menus that we have in some column values get rebuilt so they disappear under the user's cursor. It's just not a great experience.

Instead of re-implementing the search logic, I'd ideally like to be able to re-use the DataTables implementation, is there a way to get at that? The key is that this is an ajax driver DT, we just have a few operations where we want to be more surgical than loading the entire table.

BTW, the update algorithm I'm using is as follows:

When we receive a STOMP update notice, it includes the record id that was changed. We loop the existing DataTable rows to see if any of them contain that id. If we find a matching row, we then do an ajax call to get only that record, then do a dtRow.data(newData).invalidate(); This causes only that row to be redrawn.

What I would like to do is detect if the newData no longer matches the search(), and if that happens, then visually mark the row (change the opacity or something) and start a 1500ms interruptible refresh timer. When the UI settles down, the timer will expire, and we would then trigger a draw() to get the updated version of the paged data (effectively removing the row). Same approach for deleted rows.

If this isn't possible, another idea I'm kicking around is to trigger the ajax call, but add an additional parameter for the id of interest (table.ajax.params() + '&id=123'). If that returns no records, then we would use the removal approach I describe above.

Any thoughts?

Answers

  • allanallan Posts: 65,249Questions: 1Answers: 10,814 Site admin

    I'd ideally like to be able to re-use the DataTables implementation, is there a way to get at that?

    Unfortunately no, there isn't an option for that I'm afraid. I do see the utility in that, and possibly the ability to replace the default implementation, but at the moment you'd need to pull the logic out of DataTables and rework it.

    This is the function you want if you did want to head down that road.

    I will look at making it accessible in future.

    Allan

  • trumpetinctrumpetinc Posts: 4Questions: 1Answers: 0

    Thanks. I ultimately decided to go with my "other option" - initiating my own ajax query against only the changed rows, then doing an in-place update. The result is super responsive, and the search syntax is completely consistent with the back-end.

    Here's the implementation in case anyone else would benefit:

    class DataTableRefresher{
        constructor(table, config){
            this.table = table;
            this.getIdForData = this._createIdForDataFn(config.idPath);
            
            // For non-spring mode, use the standard DT exact search syntax ( "123" )
            // In spring DT mode, the +-1 is for Spring DataTables weirdness (123+-1)
            // The + is used for multi-select.
            // Then we have to add a non-existent additional search spec (-1).
            // If we don't do this, then the Spring DataTables implementation assumes that we are doing a LIKE search, so we find more records than we want.
            // Ultimately, Spring's DataTables filtering predicates need to be fixed.
            this.getSearchStringForId = config.springDataTablesMode ? (id) => id + '+-1' : '"' + id + '"';
            
        }
        
        _createIdForDataFn(path){ // spec can be dot separated, etc...
            // Split the path string into an array of individual keys
            const keys = path.split('.');
    
            return (data) => keys.reduce(
                                    (current, key) => current ? current[key] : undefined
                                    , data
                                );
    
        }
        
        _getRowForId(id){
            for (const r of this.table.rows()[0]){
                let dtRow = this.table.row(r);
                if (this.getIdForData(dtRow.data()) === id){
                    return dtRow;
                }
            }
            
            return undefined;
        }
        
        async refreshTableForIds(ids){
            for(const id of ids){
                let dtRow = this._getRowForId(id);
                if (dtRow){
    
                    // make a deep copy of the current search params            
                    let params = {...this.table.ajax.params()};
                    params.columns = [...this.table.ajax.params().columns];
    
                    // then add another search spec to get us our id
                    params.columns.push(
                        {
                            data: this.table.init().dataSource.idPath,
                            searchable: true,
                            search: {
                                value: this.getSearchStringForId(id) 
                            }
                        }
                    );
                    
                    let separator = this.table.ajax.url().includes('?') ? '&' : '?';
                    let url = this.table.ajax.url() + separator + $.param(params); // have to encode the ajax parameters are a proper query string for the URL
                    
                    let dtResult = await doRestCall('GET', url);
                    if(dtResult.data.length){
                        // changed data is still visible in the table (i.e. hasn't been deleted or changed in a way that filters it out)
                        let newData = dtResult.data[0];
    
                        dtRow.data(newData).invalidate();
                        
                        this.table.init().rowCallback?.(dtRow.node(), newData);
    
                    } else {
                        // the changed data is no longer visible in the table
                        dtRow.draw('page');
                    }           
                    
                } else {
                    console.log(`id ${id}  not in table, ignoring the refresh request`);
                }
            }   
        }   
    }
    

    example usage:

    const dtRefresher = new DataTableRefresher(this.api(), { idPath: 'id', springDataTablesMode: true} );
    

    This can then be used in an event handler (in my case, triggered by a websocket STOMP message that contains the list of ids that have changed).

    I'm going to hold off on debouncing the draw() call as this isn't impacting our application much right now, but it would be pretty easy to implement.

  • trumpetinctrumpetinc Posts: 4Questions: 1Answers: 0

    Just realized that there is one line in this code that uses an external function dependency:

    let dtResult = await doRestCall('GET', url);
    

    that will need to be adjusted for the specific ajax call approach.

  • trumpetinctrumpetinc Posts: 4Questions: 1Answers: 0

    Update that includes a default ajax handler (which can be overidden with the optional doAjaxCallFn config value):

    class DataTableRefresher{
        constructor(table, config){
            this.table = table;
            this.getIdForData = this._createIdForDataFn(config.idPath);
            
            // For non-spring mode, use the standard DT exact search syntax ( "123" )
            // In spring DT mode, the +-1 is for Spring DataTables weirdness (123+-1)
            // The + is used for multi-select.
            // Then we have to add a non-existent additional search spec (-1).
            // If we don't do this, then the Spring DataTables implementation assumes that we are doing a LIKE search, so we find more records than we want.
            // Ultimately, Spring's DataTables filtering predicates need to be fixed.
            this.getSearchStringForId = config.springDataTablesMode ? (id) => id + '+-1' : '"' + id + '"';
            
            this.doAjaxCall = config.doAjaxCallFn ? config.doAjaxCallFn : this._defaultDoAjaxCall;
            
        }
        
        async _defaultDoAjaxCall(url){
            const response = await fetch(url);
            if (!response.ok){
                if (response.status == 401){
                    throw new Error("Authorization is expired");
                }
                var body = await response.json();
                console.log(`[${response.status}] - ${body.message}`);
                throw new Error(`[${response.status}] - ${body.message}`);
            }
            const contentType = response.headers.get("content-type");
            if (!contentType || !contentType.includes("application/json")) {
              throw new TypeError("Oops, we haven't got JSON!");
            }
            
            return response.json();
        }
        
        _createIdForDataFn(path){ // spec can be dot separated, etc...
            // Split the path string into an array of individual keys
            const keys = path.split('.');
    
            return (data) => keys.reduce(
                                    (current, key) => current ? current[key] : undefined
                                    , data
                                );
    
        }
        
        _getRowForId(id){
            for (const r of this.table.rows()[0]){
                let dtRow = this.table.row(r);
                if (this.getIdForData(dtRow.data()) === id){
                    return dtRow;
                }
            }
            
            return undefined;
        }
        
        async refreshTableForIds(ids){
            for(const id of ids){
                let dtRow = this._getRowForId(id);
                if (dtRow){
    
                    // make a deep copy of the current search params            
                    let params = {...this.table.ajax.params()};
                    params.columns = [...this.table.ajax.params().columns];
    
                    // then add another search spec to get us our id
                    params.columns.push(
                        {
                            data: this.table.init().dataSource.idPath,
                            searchable: true,
                            search: {
                                value: this.getSearchStringForId(id) 
                            }
                        }
                    );
                    
                    let separator = this.table.ajax.url().includes('?') ? '&' : '?';
                    let url = this.table.ajax.url() + separator + $.param(params); // have to encode the ajax parameters are a proper query string for the URL
                    
                    let dtResult = await this.doAjaxCall(url);
                    if(dtResult.data.length){
                        // changed data is still visible in the table (i.e. hasn't been deleted or changed in a way that filters it out)
                        let newData = dtResult.data[0];
    
                        dtRow.data(newData).invalidate();
                        
                        this.table.init().rowCallback?.(dtRow.node(), newData);
    
                    } else {
                        // the changed data is no longer visible in the table
                        dtRow.draw('page');
                    }           
                    
                } else {
                    console.log(`id ${id}  not in table, ignoring the refresh request`);
                }
            }   
        }   
    }
    

    Basic usage:

    const dtRefresher = new DataTableRefresher(this.api(), { idPath: 'id'} );
    

    Complete configuration example:

    const dtRefresher = new DataTableRefresher(this.api(), { idPath: 'id', springDataTablesMode: true, doAjaxCallFn: (url) => doRestCall('GET', url)} );
    
Sign In or Register to comment.