Scroller pagination issue: offset always 0

tacman1123

Scroller pagination has always bit a bit of a mystery to me. I'd like to be able to configure that page size, it appears to guess the page size based on some hueristic.

Regardless, that's not really my problem. When scrolling through

https://packages.survos.com/

it works fine, you can see in the console that the offset changes.

But when selecting Symfony 7 and marking=valid

https://packages.survos.com/?marking=valid&symfonyVersions=7.0 (alas, due to an unrelated but, query string parameters aren't auto-selecting the facets, so to see the error you have to open the searchPanes and select these 2).

Now scroll through, and and the debug line shows the offset as always 0, so it never gets beyond 135.

I think this use to work, but may not. It seems to work without the facet searches, so maybe I'm encoding something wrong? Or maybe related to the sort?

tacman1123

I made a 45-second video to show the problem: https://www.awesomescreenshot.com/video/26297260?key=16f4eaa82bba8a918a37d2a83cb61fd1

allan

Hi,

Thanks for the link and video.

The number of rows that Scroller selects to get and display is based on scroller.displayBuffer. That is basically the number of rows that fit into the viewport multiplied by that fixed factor.

This is the code that does that.

Regarding the offset, I'll need to come back to that. I'm traveling this week, so it will probably be next week before I get a chance to have proper look at it.

Allan

tacman1123

Thanks -- that explains why when I open the console debugger the number of rows requested changes!

I'm still stuck on the offset, I'd be grateful if you got a chance to look at it. Thanks.

tacman1123

Perhaps there's a version out of sync, I found this when trying to upgrade everything.

[warning] datatables.net-scroller-bs5 requires datatables.net-bs5@1.11 - 2 but version 2.0.3 is installed.
[warning] datatables.net-scroller-bs5 requires datatables.net-bs5@1.11 - 2 but version 2.0.3 is installed.
[warning] datatables.net-scroller requires datatables.net@1.11 - 2 but version 2.0.3 is installed.
[warning] datatables.net-scroller requires datatables.net@1.11 - 2 but version 2.0.3 is installed.

tacman1123

Hi, any suggestions on where I can look to solve this?

allan

datatables.net-scroller requires datatables.net@1.11 - 2 but version 2.0.3 is installed

That doesn't make a particularly huge amount of sense to me. 2.0.3 should be included in that range I thought. I understood it to be 1.11.* through 2.*.

Checking with the npm SemVer Calculator appears to confirm this:

What package manager are you using?

Allan

tacman1123

AssetMapper (from Symfony) which downloads the packages from jsdelivr.

I believe it then scans the downloaded package for dependencies and installs those.

tacman1123

But I'm not sure the problem is the packages, but rather that the callback isn't providing the right offset. Maybe it's a package sync issue, but that might be completely unrelated.

tacman1123

I've added an issue to get rid of the warning in AssetMapper, but I'm almost positive that it's simply a false warning, and not the root of the pagination problem.

https://github.com/symfony/symfony/issues/54610

tacman1123

Are you able to confirm this behavior? I'm stuck on where to even look at this point. I'm wondering if it's something new in version 2, as I think this same code use to work.

allan

Apologies - I was focusing not on the offset issue, but rather the version error. That might well be something in symfony that has different semver handling that node - I'm not sure.

But going back to the offset issue - it looks like you have different row heights in the table, and that isn't valid with Scroller. With Scroller you must have all rows exactly the same height, otherwise it can't perform its virtual scrolling calculations (i.e. it can't say "row x must be exactly at pixel position y, since all rows as height z" and it can't calculate the height of all rows, since 1, it doesn't have all the rows and 2 that would be so slow).

Try adding the nowrap class to your DataTable. That will help in that regard (although you'll have the description column hidden by Responsive I'm afraid).

I'm not certain that will fix it outright, but it might...

Allan

tacman1123

Indeed, that solves the problem, thanks!

Is there a way to simply lock the row height? I can control the image size (though the images are loaded later, obviously, and that's probably contributing to the mis-calculation of the height).

That is, if I have to lock the height of the row, it should be the maximum height of the thumbnail image. Then I'd like description to simply wrap to the available height and show an ellipsis if it doesn't have the space.

allan

I'm not seeing a thumbnail on the page I'm afraid? However, if you have an <img> tag, adding the correct height and width attributes will allow it to render in the space correctly.

One other thing I've realised - the use of the child rows is going to mess up Scroller as well, since that also takes up vertical space in the scrolling container.

You could use a modal? Or if you only have 1000 rows, I wouldn't bother using Scroller. It if was 10k, then sure, it makes more sense (although paging might work for you instead of scrolling?).

Allan

tacman1123

So I still haven't been able to get rid of the offset=0 error.

go to https://packages.survos.com and select Symfony 7.1 from the left menu (under Symfony). Start scrolling down, eventually it'll become blank and all the offsets will be zero in the network log.

BUT it only happens when there's a filter! It appears to work okay if it's everything.

I have the nowrap class on the table, and have locked all the rows to the same height (2 lines). No images.

Any ideas? This is a glaring issue for us, as the whole system is dependent on datatables and facets.

allan

The row heights are not all the same (I see some which are four lines high, some which are two, some three). When using Scroller, it is a core requirement that all rows be of identical height. The way it works is to say "this is the height of the rows, this is the scroll position, therefore I should show row X". If one or more rows are of different heights, that that equation fails. Scroller has to operate that way since it doesn't have all the rows present to be able to calculate the height of each row.

You need to do one of:

1. Make all rows identical in height
2. Remove Scroller
3. Switch to paging instead of scrolling.

Allan

tacman1123

Is there a css class I can use somewhere to force the rows to be the same height? I thought I was doing this, but I think there are some columns where I'm not doing it correctly.

I see now that while I'm using the .line-limit-2 class on every text column, I neglected to import the css file. DOH.

will applying this class to every column force the rows to be the same size?

.line-limit-2 {
    display: -webkit-box;
    -webkit-line-clamp: 2;
    -webkit-box-orient: vertical;
    overflow: hidden;
}

tacman1123

Hmm, I may have misunderstood line-clamp and unstyled / inline list.

In particular, where max is greater than the line clamp, this seems to have a taller row.

My idea was to have the items all on the same line, up to a certain number. I've confusing list-unstyled and list-inline, when I tweak this the offset seems to be set okay, maybe clamping and lists don't work as I think they do.

Anyway, thanks for re-iterating that it's row height, if you have any suggestions how to make it happen on a higher level (rather than individual columns), I'd love to hear it.

            <ul class="list-unstyled">
                {% for item in items|slice(0, max) %}
                <li class="list-inline-item">{{ item }}
                    {{ loop.last and (items|length > max) ? '+%d'|format(items|length - max) }}
                </li>
                {% endfor %}
            </ul>

tacman1123

I'm still stuck, even after removing all formatting in my tables. I have 3 columns with only some text, no lists or any clamping or anything special. The rows are all the same size.

It _only_happens when working with searchPanes. Without it, the offset works as expected.

tacman1123

The link shows just a few columns with no special formatting.

https://packages.survos.com/simple

In the javascript console, you can see that simple scrolling works, and that the offset changes.

But if you select something from the searchPanes, like marking=symfony_ok, the offset never gets above zero and eventually there's no data on the page.

tacman1123

API Platform deprecated something in version 4 that may be related...

tacman1123

Indeed, I think this is not about the row height but rather about the datasource not returning the right offset when doing a facet search. Sigh.

kailashkds0

Hello,

I am using jquery datatable. I am doing infinite scroll.
When I am scrolling without search pages selected.
first page:
https://packages.wip/api/meili/packages?limit=81&offset=0&facets[]=marking&_locale=en&_index=packagesPackage&_style=spreadsheet
When scroll:
https://packages.wip/api/meili/packages?limit=81&offset=46&facets[]=marking&_locale=en&_index=packagesPackage&_style=spreadsheet

But when I select a searchpane:
first page:
https://packages.wip/api/meili/packages?limit=81&facet_filter[]=marking,in,new&offset=0&facets[]=marking&_locale=en&_index=packagesPackage&_style=spreadsheet

When scroll:
https://packages.wip/api/meili/packages?limit=81&facet_filter[]=marking,in,new&offset=0&facets[]=marking&_locale=en&_index=packagesPackage&_style=spreadsheet

I have check in both the cases api response is same (with searchpages without searchpanes) So there is nothing in api response that is a issue.
Further more:

I am using callback datatable to get response via ajax.

apiData.offset = params.start;
this start is somehow set to 0 in case of searchpanes.

ajax: (params, callback, settings) => {
console.error(starting at ${params.start});
this.messageTarget.innerHTML = starting at ${params.start};
this.apiParams = this.dataTableParamsToApiPlatformParams(params, searchPanesRaw);
});

kailashkds0

https://packages.survos.com/
this is live website

kailashkds0

Hello,

I am using jquery datatable. I am doing infinite scroll.
When I am scrolling without search pages selected.
This is live website
https://packages.survos.com/
first page:
https://packages.survos.com/api/meili/packages?limit=0&offset=128&facets[]=marking&_locale=en&_index=packagesPackage&_style=spreadsheet
When scroll:
https://packages.survos.com/api/meili/packages?limit=99&offset=128&facets[]=marking&_locale=en&_index=packagesPackage&_style=spreadsheet

But when I select a searchpane:
first page:
https://packages.survos.com/api/meili/packages?limit=81&facet_filter[]=marking,in,new&offset=0&facets[]=marking&_locale=en&_index=packagesPackage&_style=spreadsheet

When scroll:
https://packages.survos.com/api/meili/packages?limit=81&facet_filter[]=marking,in,new&offset=0&facets[]=marking&_locale=en&_index=packagesPackage&_style=spreadsheet

I have check in both the cases api response is same (with searchpages without searchpanes) So there is nothing in api response that is a issue.
Further more:

I am using callback datatable to get response via ajax.

apiData.offset = params.start;
this start is somehow set to 0 in case of searchpanes.

            ajax: (params, callback, settings) => {
            console.error(starting at ${params.start});
            this.messageTarget.innerHTML = starting at ${params.start};
            this.apiParams = this.dataTableParamsToApiPlatformParams(params, searchPanesRaw);
            });

tacman1123

Here's a simplified example, 1000 rows with the identical marking, so selecting marking=new is the same as not selecting anything.

https://packages.survos.com/

use the scroller, everything's fine.

All the markings are 'new', so selecting it should be identical.

https://packages.survos.com/?marking=new

all the rows are the same height, there's very little data per row.

We are returning the same results, with our without a selected facet, which make sense because the select facet doesn't actually change the results.

I don't think it's a bug in what our data returns, but somehow the params.start we receive in the ajax: callback is always zero when searchpanes has data.

The ajax callback code is at https://github.com/survos/SurvosApiGridBundle/blob/main/assets/src/controllers/api_grid_controller.js#L659-L685

I can share the code that does the filtering, but I don't think it's there, because we are returning identical JSON with our without the filter, which is expected.

Any ideas? I think it might be a bug in scroller with dynamic data.

tacman1123

it's not the row size. when there is searchPane data, it never goes beyond 0

allan

Apologies for not having got back to you on this. It very much does look like a SearchPane / Scroller error. Probably something to do with needing to recalculate sizes. I'll need to get back to you on this - got a fair backlog of support and contract work at the moment. Sorry I don't have an immediate fix.

Allan