How to make render() work independently when user- or application-filtering?

cFreed

I just fell into a situation where the currently available set of values for type seems not enough to work as simply as it'd be possible.
Here is a pretty simple use case that illustrates the problem.

Having a column (say "allowed") containing a boolean value, which contains "0" or "1" (comes from a DB content).
Obviously it needs to be more friendly displayed so we'll do something like this:

var table = $('#example').dataTable({
  columnDefs: [{
    data: 'allowed',
    render: function (data, type, row, meta) {
      return data == '1' ? 'yes : 'no';
    }
  }]
});

Since we didn't do anything depending on type above, the user can type "yes" or "no" in the dedicated filter area to display only the allowed (respectively not-allowed) rows .

In the other hand, if for some reason the application must directly filter rows by itself, we can do for instance:

table.columns(<col index>).search('yes').draw;

So far so good.

Now if the application works in a multilingual context where a localize() function is available, the first scriptlet above becomes:

var table = $('#example').dataTable({
  columnDefs: [{
    data: 'allowed',
    render: function (data, type, row, meta) {
      return localize(data == '1' ? 'yes : 'no');
    }
  }]
});

Then user filtering keeps working fine, wether using "yes|no", "si|no", "oui|non", and so on.
But... now application filtering works only when the current language is English!

So one immediate obvious solution is to also change the second scriptlet accordingly:

table.columns(<col index>).search(localize('yes')).draw;

Sure then it works fine anew, but here is what I wanted to expose: isn'it too bad to have to use localize() again, while in this case a more immediate and elegant method would be to directly work with the original data value "0|1":

table.columns(<col index>).search('1').draw;

For this to work, it'd suffice to make render() work depending on its type argument.
But it's not currently possible: the "filter" type is the one used by DataTables both when responding to user-filtering and to column.search().
(BTW I guess that maybe user-filtering just uses column.search()...)

Here we retrieve what the doc says about the filter|search ambiguity.
So it'll be great if it could be uncoupled (may be by adding a new type value "search"?).

Thanks for your attention.

cFreed

Erratum

Just under the first scriptlet in the previous post, please read type above instead of type below.

Sorry.

edited by Allan - I've modified your post above to reflect that correction.

allan

But... now application filtering works only when the current language is English!

I don't quite understand that point I'm afraid. The filtering should be language independent - it will be based on whatever is in the table (or what returned by the renderer for a filter type). Can you show me an example of that problem?

Regarding using a search for 0 and 1 as well as yes and no (or their translations), I would suggest something like:

render: function ( data, type, row ) {
  return type === 'filter' ?
    data +' '+ localise( data ) :
    localise( data );
}

or similar. i.e. have both the data values you want to be filtered on available in the search string.

Allan

cFreed

Hi Allan, thanks for your answer.

Aside comment: thanks for correcting my initial post. I hadn't found a way for doing it by myself once posted: did I missed something?
Furthermore I lately noticed that I let another error: in my scriptlets I always forgot parenthesis in .draw().

Now regarding my question, I'm a bit disconcerted by your answer, so I'm afraid I didn't clearly explained.

I don't quite understand that point I'm afraid.

Unless I misinterpret what is "that point" above, what I meant with "now application filtering works only when the current language is English!" is: when render() has been modified to return a localized value, then .search('yes') will not match, for instance, the spanish version si.

Can you show me an example of that problem?

In my mind the use case I exposed is precisely the example of the problem.
Said in other words, it is: we can't make a render() function return different results whether it's invoked in .search() or simple user-filtering process, because both use the same type value "filter".

The filtering should be language independent - it will be based on whatever is in the table (or what returned by the renderer for a filter type).

Here again, I see the need of a distinction depending on what is being processed:

• when .search() (remember it's ran by the application itself), yes it absolutely should be language independent (at least ideally), and so will be based on whatever is in the table (in this case, the boolean value 0 | 1).
• when user-filtering, conversely, it must work on what the user typed, and obviously what he may type is also what he could see displayed... which is language dependent!

Another thing must be pointed out: while .search() can be (and here, is) launched in the context of a unique column so it's not ambiguous, by design user-filtering always works on any visible column.
So in fact their intent is fundamentally different (even if they be common in some cases).
But, and here is the problem at my eyes, this difference can't be well-preserved at the time render() is invoked, because the data it returns can't be maked different.

With my current workaround:

• using .search(localize('yes')) ensures a correct (non-ambiguous) filtering since the value returned by render() can't contain anything else than the localized value of yes or no.
• and the user-filtering process risks only to abusively retain rows where some other similar value resides in another column (but this belongs to the inherent ambiguity of this kind of filtering, and after all keeps absolutely logic at the user's eyes).

In the other hand, no offense but I find that your suggestion to use data + ' ' + localize(data) as returned value from render() is not a better one:

• sure it allows direct use of .search('1') for a correct result when application-filtering
• but it makes user-filtering more widely ambiguous: it may retain rows where another column contains, not only the localized value (as already in the original version), but also the raw data value 0 or 1 which likely may easily happen with columns containing integers.

Finally it's why I imagined it could be of interest to have a new possible value "search" for type, that would be used by render() when .search() is processed, so allowing to work like this:
(here I intentionally write more code than needed, for the sake of clarity)

var table = $('#example').dataTable({
  columnDefs: [{
    data: 'allowed',
    render: function (data, type, row, meta) {
      switch (type) {
        case 'display':
        case 'filter':
        case 'sort':
          return localize(data == '1' ? 'yes : 'no');
          break;
        case 'search':
          return data;
      }
    }
  }]
});

which in turn allows application-filtering to remain the most simple, and language independent:

table.columns(<col index>).search('1').draw();

Hoping to have been more clear... and not too annoying!
Thanks for your attention.

Fred

allan

Hi Fred,

Thanks for your reply. Regarding editing posts - you can do so for 15 minutes after the post. After that you'd need to post a correction as you did.

Regarding the problem - I've created this little example: http://live.datatables.net/joyinohe/1/edit .

It uses a trivial little localize function (you can change the name of the variable it uses to change the "dictionary").

It seems to work as I would expect - you can filter on Si or No as it is setup. You cannot search on the office location ("London" for example).

Can you modify that example, or link me to a page, that shows the issue you describe so I can more fully understand it please?

Regards,
Allan

cFreed

Hi Allan, thanks for your effort.

Here is a modified version of your example: http://live.datatables.net/joyinohe/2/.

_BTW sorry if it doesn't allow edition (I just noticed that "/edit" is omitted in the url above): I'm familiar with jsfiddle and stackOverflow contexts, and was a bit surprised with the somewhat different behaviour of joyinohe.
Particularly I'm not quite sure it auto-saved my version! Let me know.

you can filter on Si or No as it is setup. You cannot search on the office location ("London" for example).

I totally agree, and there is no problem till here.

So in my modified version I added what I previously named application-filtering, which uses search(), and should ideally be language independent, as we both said: hence it should be able to retrieve "London".
And you can see that it doesn't, even if localizing only when type === 'filter': .search() always gets the localized version returned by render().

It's why I talk about an hypothetical "search" value for type, which might be the one render() gets passed when .search()ing, so we could return the original data in this case.

I realize that till now I couldn't clearly transmit my point of view, probably due to my poor English (as a French man I struggle to express myself in that language, and I'm never sure I succeeded).
So, sorry to take your time, and thanks for your patience.

Fred

allan

Hi Fred,

Your English is excellent. Far better than my embarrassing school boy French...! And yes, JSBin is a little different in how it handles URLs - it does auto save.

I think part of the issue here is that the type === 'filter' is actually legacy. For backwards compatibility it didn't change it to be type === 'search', but really, that is what it should be called (since the term "search" is used else where in the API for searching the table)!

If you want the search term to include both the localised and the original data, you can do something like: return localized + ' '+ data;: http://live.datatables.net/joyinohe/3/edit

If we introduce search in addition to filter, it would be another search API that would be required, and I think that would get really confusing.

Regards,
Allan

cFreed

Hi Allan,

I persist to think that, despite of your kind words about my English, till now I failed to really and clearly transmit what I'm looking for.
I'll try again one more time, but please don't feel too annoyed by my insistance!

If we introduce search in addition to filter, it would be another search API that would be required

It's notably by reading this that I think I didn't clearly explain my goal.
It seems to me (maybe naively) that for what I have in mind it'd be sufficient to slightly change the way render() is internally invoked.

Remember that my fundamental intent is only to be able to work with render() in a way that allows to detect if we're in the context of user-filtering (i.e. when the user typed something in the dedicated "filter" area), as opposed to any other render() invocation.

If I correctly understand what you said about the trouble with search|filter terms, it means (and comes from the fact) that in fact user-filtering process internally uses .search(), which in turn invokes render() always with the same type = 'filter', so we can't detect if we're in the user-filtering process or not.
So let's give up my previous suggestion of introducing a supplemental type = 'search'.

And here is the alternative solution I now imagine: would'nt it possible to introduce a context variable (say for instance searchContext), which might be:

• set to undefined or 'unknown' at init time
• set to 'user' at the beginning of the user-filtering process (and reset when it's finished)
• and meantime added as a new member to the meta object transmitted to render()

I think that it's really only a slight change, and without any compatibility issue.
Then inside the code of a render() instance we could know when it comes from the user action, and return a localized value, while returning the raw value in other occurrences.
Something like this (again, more code than needed, to be totally clear):

render: function (data, type, row, meta) {
  if (type === 'filter') {
    // we're in a .search() process...
    if (meta.searchContext === 'user') {
      // ...which was initiated by user action
      return localize(data);
    }
    // ...which was initiated by the application itself
    return data;
  }
}

Last point, and sorry to insist on this: using return localized + ' ' + data; doesn't produce exactly the hoped result.
Indeed it permits to retain the raw values (and so be language independent when we want it), but it does this also when user-filtering, so it makes the set of retained values wider than those displayed, which seems rather strange against what may be expected in this situation.

Again, sorry for my persistance, and thanks for your attention.

Fred

allan

Thanks for following up on this - its fantastic to have a conversation about enhancements to DataTables!

My main issue with what you propose is that the search() and column().search() methods should use the filter orthogonal data type. The search() method calls exactly the same code as the search box at the top of the table.

The issue is compounded further by column().search() since DataTables itself doesn't provide inputs for column search. If you look at this example you will be able to see that it uses column().search(). What you propose would break that if renderers were used to display something different in the table.

In essence there is no knowledge of if a search is triggered by the end user or not, and I think it would be very difficult to determine that. It would also require layering of search, since you would want an API search to be combined with a user search.

I completely agree that DataTables does need to significantly improve its filtering, but I'm not sure that this is the right way to do it I'm afraid. My current thought to allow access to the raw data programmatically is to allow search() to accept a function that can be used to search the data. How to clear that, and how to layer it is currently an open question though.

Allan

cFreed

Hi Allan,

Thanks for following up on this - its fantastic to have a conversation about enhancements to DataTables!

I really appreciate your courtesy effort.

Regarding the rest I must confess I couldn't fully understand your argumentation, since I never dived deep enough in the API.
Nevertheless there is something I precisely realized when looking at the example you pointed out: it's that your understanding of my idea is a bit disturbed by, again, my lack of precision till now.

I guess you probably imagined that inputs such those of this example also belong to what I called "user-filtering", hence your comment:

there is no knowledge of if a search is triggered by the end user or not

But it's not the case! For me, "user-filtering" (sure it's appears badly named now) addresses only the data typed in the dedicated "Filter" area which is part of the core DataTables layout.
The reason of the difference at my eyes is: this area is the only one the developer can't control, while he has hand on the inputs of the example since they're his own creation, and so he can take any needed measures to effectively control all that happen about them.

So as I persisted to imagine that things could work more simply than you fear (and because I'm somewhat obstinate ), I did some partial tests in order to prove that we could in fact determine that user-filtering has been triggered.
And the funny thing is that they lead me to a complete solution working like I want without needing a DataTables enhancement!

In a few words, it works like this (not fully tested yet, but seems that it should actualy succeed):

• define a (now own) variable userFiltering, initially remains undefined
• after DataTable initialization, bind draw event to userFiltering = undefined;, bind keyup event on the dedicated "Filter" area to userFiltering = 'user';
• then I can work in render() as previously exposed

So I can assert I'll not annoy you any longer with this!
Again, thanks for have been so patient.

Fred

allan

For me, "user-filtering" (sure it's appears badly named now) addresses only the data typed in the dedicated "Filter" area which is part of the core DataTables layout.

If you use search() to set a search value, you'll see that is also sets the value in the input element you call "user-filtering": http://live.datatables.net/zaqufowu/1/edit .

The input element and search() do the same thing - indeed the input element is basically:

input.on( 'keypress', function () {
  dt.search( this.value ).draw();
}

That's why I say we can't know if the input is user filtered or not. Its possible to say that if you don't use search() (which might be true in a specific case, but isn't always true). Its even worse when using column().search() since DataTables doesn't provide any input elements for that - you have to use `-api column().search().

Great to hear that you've got a work around though!

Allan

cFreed

Hi Allan,

Thanks for this added information.

Sure I was pretty surprised by this sort of bidirectional relation between table.search() and the "user-filtering" input tag!
Fortunately, this feature doesn't break my workaround, since the keyup event keeps fired only when it's the user that enters something in this area.

I still have (more than) a lot to discover about DataTables

Fred