Natural Sorting not ordering naturally

glimpsed_chaos

Link to test case: https://live.datatables.net/jixibate/1/

Error messages shown: None
Description of problem: I am using the natural sorting plugin which is specifically for "Sort data with a mix of numbers and letters naturally." There are even a few comments that specifically mention the same format I need to use here.

And initially it looks like it works. It starts to order 1,2,3, 4a, 4b,4c, 5, 6, etc... However, the natural order in regards to the "alpha" side becomes unordered.

In my example, everything looks good up through 29a,29b and 29c. A few rows later it orders as 35d, 35e, 35a, 35b, 35c. A few row later it is correct again with 37a, 37b, 37c. And broken again after that.

I tried with and without a few extensions I use such as scroller (true and false) to ensure there was not a compatibility issue. But the result is the same either way.

Am I implementing it incorrectly here? Without the plug-in it orders 1, 10, 100, etc.. and with it orders like I stated above so it appears to be right but something is not.

Thank you for your time.

colin

Agreed, something looks odd there. Thanks for the test case, I've trimmed it down to be mininum (https://live.datatables.net/jixibate/2/edit) and it's still occurring.

I've raised it internally (DD-2680 for my reference) and we'll report back here when there's an update.

Colin

glimpsed_chaos

Thanks Colin!

For now I am just using a SQL ORDER that handles it initially when returning the data. Ordering with the column will throw it off but the user can request the table again. Should not happen often enough to be an issue for us.

glimpsed_chaos

@Colin

With 2.0 now out and I can see that the natural.js sorting plugin updated to 2.0.0 as well, I thought I would come back to this to check.

I updated references to the nightly builds and the 2.0.0 plugin but it appears the problem still exists. The sorting appears to work but as you scroll down (such as 35a, 35b,etc. - the same as in my original post) the order is not sorted properly. Some are, some are not.

Here's the updated example: https://live.datatables.net/jixibate/3/edit?html,js,output

I don't know if it is relevant but if I take my table to an array, I can then sort that array without issue with using localeCompare such as:

            const sortArray = (sourceArray) => {
                const sortByLocation = (a, b) => a[0].localeCompare(b[0], 'en', { numeric: true });
                return sourceArray.sort(sortByLocation);
            };

allan

Hi,

I'm afraid I didn't specifically look at the natural sorting plug-in for the 2.0 release, so I'm afraid it will be unchanged in behaviour. Sorry.

It might be that the algorithm being used by the plug-in just doesn't cope with that data. We perhaps need to update it to a different natural sort library. That said, it is surprising, it looks like the sort of data that it should just work with. I didn't write the original algorithm, just put a wrapper around it.

Let me get back to you on this one.

Allan

glimpsed_chaos

@allan

Not a problem, as I said in my original post, I have a work around for now.

With 2.0 out I am sure you understandably have more urgent items.

allan

Here we go.

I found this article with a natural sort algorithm which does work with your data set, but then it goes on to mention localeCompare as you did yourself. With that it appears to work very nicely.

With localeCompare widely supported, I think it might be time to update the natural sorting plug-in for DataTables to use this:

DataTable.type('natural', {
  order: {
    asc: function (a, b) {
      return a.localeCompare(b, navigator.languages[0] || navigator.language, {numeric: true, ignorePunctuation: true});
    },
    desc: function (a, b) {
      return a.localeCompare(b, navigator.languages[0] || navigator.language, {numeric: true, ignorePunctuation: true}) * -1;
    }
  },
  className: 'natural-sort'
});

Goodness me it is cleaner than it was before!

Allan

kthorngren

Looks like Allan's suggestion is the way to go. However I think I found the issue with the original plugin. This regex expression seems to be wrong:

var re = /(^-?[0-9]+(\.?[0-9]*)[df]?e?[0-9]?%?$|^0x[0-9a-f]+$|[0-9]+)/gi

Removing [df]?e? seems to fix the issue:
https://live.datatables.net/jixibate/4/edit

I reduced the data set down to the 35's which were a problem with d-f. More work/testing might need to be done for the regex expression.

Kevin

glimpsed_chaos

@Allan Looking forward to an update that works.

@kthorngren I saw that you reduced it down to the 35's and I copied your code to the full table I had and that seems to work fine as well - https://live.datatables.net/jixibate/8/edit

kthorngren

Looking forward to an update that works.

I would replace the natural.js you are using with the code Allan presented above. Just load it locally like Allan's example. Its a much simpler solution and doesn't rely on regex patterns that might be buggy.

Kevin

glimpsed_chaos

@kthorngren @allan

Allan's example works like a charm however, in my application I am currently on 1.13.7 and we just went live this past week. Cautiously waiting before introduction Datatables 2.0.0 into the mix.

In testing that, I get a DataTable.type is not a function (because that is new in 2.0.0) I disovered. https://live.datatables.net/jixibate/9/

Is there a way to implement Allan's solution into my original 1.13.7 example?

kthorngren

The DataTable.type() was introduced an 2.0. Instead you can make an ordering plugin with the code. For example:

$.extend( DataTable.ext.type.order, {
    "natural-asc": function ( a, b ) {
        return a.localeCompare(b, navigator.languages[0] || navigator.language, {numeric: true, ignorePunctuation: true});
    },
  
    "natural-desc": function ( a, b ) {
        return a.localeCompare(b, navigator.languages[0] || navigator.language, {numeric: true, ignorePunctuation: true}) * -1;
    }
} );

Test case with 1.13.7.
https://live.datatables.net/zigevalo/1/edit

Kevin

kthorngren

Just to confirm the ordering plugin works with 2.0:
https://live.datatables.net/durezivi/1/edit

So you won't need to remember to use DataTable.type() but you will have that option.

Kevin

glimpsed_chaos

@kthorngren Thank you! That works perfect.

allan

I've just made a commit to update the natural sorting plug-in for DataTables to use localeCompare. That will be in 2.0.1 when it drops.

Allan

glimpsed_chaos

@allan Thank you!

iyoub

Hello everyone, I have tried the above solutions, and it does not seem to sort properly. Here is an example where the "resistivity" column is not properly sorted: https://live.datatables.net/zezisebo/2/

kthorngren

@iyoub You need to load the plugin code. Updated example:
https://live.datatables.net/zezisebo/3/edit

Kevin

iyoub

@kthorngren Hi Kevin, now "Resistivity" is properly sorted, but "Lifetime" is not sorted properly anymore

kthorngren

It looks like the problem is with comparing 4.6 (note the space) and 4.23. I space is less than a numeric value. See the console output of this test case:
https://live.datatables.net/zegepava/1/edit

I uses localCompare to sort this array:

['4.689', '4.2', '4.245', '4.6 with ozone cleaning']

Here is the result:

['4.2', '4.6 with ozone cleaning', '4.245', '4.689']

It also shows the result of comparing a space to a numeric:

console.log( " " < "2" );
true

Possibly you just want to apply the natural sorting plugin to the appropriate columns. Let the Lifetime column sort as a string. The updated test case shows this

Kevin