Natural Sorting not ordering naturally

glimpsed_chaos · July 2023

Link to test case: https://live.datatables.net/jixibate/1/

Error messages shown: None
Description of problem: I am using the natural sorting plugin which is specifically for "Sort data with a mix of numbers and letters naturally." There are even a few comments that specifically mention the same format I need to use here.

And initially it looks like it works. It starts to order 1,2,3, 4a, 4b,4c, 5, 6, etc... However, the natural order in regards to the "alpha" side becomes unordered.

In my example, everything looks good up through 29a,29b and 29c. A few rows later it orders as 35d, 35e, 35a, 35b, 35c. A few row later it is correct again with 37a, 37b, 37c. And broken again after that.

I tried with and without a few extensions I use such as scroller (true and false) to ensure there was not a compatibility issue. But the result is the same either way.

Am I implementing it incorrectly here? Without the plug-in it orders 1, 10, 100, etc.. and with it orders like I stated above so it appears to be right but something is not.

Thank you for your time.

colin · July 2023

Agreed, something looks odd there. Thanks for the test case, I've trimmed it down to be mininum (https://live.datatables.net/jixibate/2/edit) and it's still occurring.

I've raised it internally (DD-2680 for my reference) and we'll report back here when there's an update.

Colin

glimpsed_chaos · July 2023

Thanks Colin!

For now I am just using a SQL ORDER that handles it initially when returning the data. Ordering with the column will throw it off but the user can request the table again. Should not happen often enough to be an issue for us.

glimpsed_chaos · February 16

@Colin

With 2.0 now out and I can see that the natural.js sorting plugin updated to 2.0.0 as well, I thought I would come back to this to check.

I updated references to the nightly builds and the 2.0.0 plugin but it appears the problem still exists. The sorting appears to work but as you scroll down (such as 35a, 35b,etc. - the same as in my original post) the order is not sorted properly. Some are, some are not.

Here's the updated example: https://live.datatables.net/jixibate/3/edit?html,js,output

I don't know if it is relevant but if I take my table to an array, I can then sort that array without issue with using localeCompare such as:

            const sortArray = (sourceArray) => {
                const sortByLocation = (a, b) => a[0].localeCompare(b[0], 'en', { numeric: true });
                return sourceArray.sort(sortByLocation);
            };

allan · February 16

Hi,

I'm afraid I didn't specifically look at the natural sorting plug-in for the 2.0 release, so I'm afraid it will be unchanged in behaviour. Sorry.

It might be that the algorithm being used by the plug-in just doesn't cope with that data. We perhaps need to update it to a different natural sort library. That said, it is surprising, it looks like the sort of data that it should just work with. I didn't write the original algorithm, just put a wrapper around it.

Let me get back to you on this one.

Allan

glimpsed_chaos · February 16

@allan

Not a problem, as I said in my original post, I have a work around for now.

With 2.0 out I am sure you understandably have more urgent items.

allan · February 16

Here we go.

I found this article with a natural sort algorithm which does work with your data set, but then it goes on to mention localeCompare as you did yourself. With that it appears to work very nicely.

With localeCompare widely supported, I think it might be time to update the natural sorting plug-in for DataTables to use this:

DataTable.type('natural', {
  order: {
    asc: function (a, b) {
      return a.localeCompare(b, navigator.languages[0] || navigator.language, {numeric: true, ignorePunctuation: true});
    },
    desc: function (a, b) {
      return a.localeCompare(b, navigator.languages[0] || navigator.language, {numeric: true, ignorePunctuation: true}) * -1;
    }
  },
  className: 'natural-sort'
});

Goodness me it is cleaner than it was before!

Allan

kthorngren · February 16

Looks like Allan's suggestion is the way to go. However I think I found the issue with the original plugin. This regex expression seems to be wrong:

var re = /(^-?[0-9]+(\.?[0-9]*)[df]?e?[0-9]?%?$|^0x[0-9a-f]+$|[0-9]+)/gi

Removing [df]?e? seems to fix the issue:
https://live.datatables.net/jixibate/4/edit

I reduced the data set down to the 35's which were a problem with d-f. More work/testing might need to be done for the regex expression.

Kevin

glimpsed_chaos · February 16

@Allan Looking forward to an update that works.

@kthorngren I saw that you reduced it down to the 35's and I copied your code to the full table I had and that seems to work fine as well - https://live.datatables.net/jixibate/8/edit

kthorngren · February 16

Looking forward to an update that works.

I would replace the natural.js you are using with the code Allan presented above. Just load it locally like Allan's example. Its a much simpler solution and doesn't rely on regex patterns that might be buggy.

Kevin

glimpsed_chaos · February 17

@kthorngren @allan

Allan's example works like a charm however, in my application I am currently on 1.13.7 and we just went live this past week. Cautiously waiting before introduction Datatables 2.0.0 into the mix.

In testing that, I get a DataTable.type is not a function (because that is new in 2.0.0) I disovered. https://live.datatables.net/jixibate/9/

Is there a way to implement Allan's solution into my original 1.13.7 example?

kthorngren · February 17

The DataTable.type() was introduced an 2.0. Instead you can make an ordering plugin with the code. For example:

$.extend( DataTable.ext.type.order, {
    "natural-asc": function ( a, b ) {
        return a.localeCompare(b, navigator.languages[0] || navigator.language, {numeric: true, ignorePunctuation: true});
    },
  
    "natural-desc": function ( a, b ) {
        return a.localeCompare(b, navigator.languages[0] || navigator.language, {numeric: true, ignorePunctuation: true}) * -1;
    }
} );

Test case with 1.13.7.
https://live.datatables.net/zigevalo/1/edit

Kevin

kthorngren · February 17

Just to confirm the ordering plugin works with 2.0:
https://live.datatables.net/durezivi/1/edit

So you won't need to remember to use DataTable.type() but you will have that option.

Kevin

glimpsed_chaos · February 18

@kthorngren Thank you! That works perfect.

allan · February 21

I've just made a commit to update the natural sorting plug-in for DataTables to use localeCompare. That will be in 2.0.1 when it drops.

Allan

glimpsed_chaos · February 21

@allan Thank you!

iyoub · July 25

Hello everyone, I have tried the above solutions, and it does not seem to sort properly. Here is an example where the "resistivity" column is not properly sorted: https://live.datatables.net/zezisebo/2/

kthorngren · July 25

@iyoub You need to load the plugin code. Updated example:
https://live.datatables.net/zezisebo/3/edit

Kevin

iyoub · July 25

@kthorngren Hi Kevin, now "Resistivity" is properly sorted, but "Lifetime" is not sorted properly anymore

kthorngren · July 25

It looks like the problem is with comparing 4.6 (note the space) and 4.23. I space is less than a numeric value. See the console output of this test case:
https://live.datatables.net/zegepava/1/edit

I uses localCompare to sort this array:

['4.689', '4.2', '4.245', '4.6 with ozone cleaning']

Here is the result:

['4.2', '4.6 with ozone cleaning', '4.245', '4.689']

It also shows the result of comparing a space to a numeric:

console.log( " " < "2" );
true

Possibly you just want to apply the natural sorting plugin to the appropriate columns. Let the Lifetime column sort as a string. The updated test case shows this

Kevin

Natural Sorting not ordering naturally

Natural Sorting not ordering naturally

Answers

Howdy, Stranger!

Categories

DataTables

Natural Sorting not ordering naturally

Natural Sorting not ordering naturally

Answers

Howdy, Stranger!

Quick Links

Categories

DataTables