Advice for performance on exports?

Advice for performance on exports?

dknifedknife Posts: 14Questions: 4Answers: 1
edited September 2018 in Buttons

I'm having considerable issues with exports on datatables. It's fairly quick below 1000, going up to around 10 seconds on 2000 records. If I go over 3000 records, it takes over 60 seconds to process and download, over 5000 is go and make coffee. This occurs irrespective of Excel, CSV, PDF or Print buttons, they all perform similar.

For the amount of data, I'm exporting only specific columns, a mix of visible and hidden ones. There's a total of 36 columns. An excel export of 3700 records results in a 5.5Mb file.

I've tried some googling and searching here but I haven't really found anything applicable other than a possible issues with exports beyond 10.1.16 and a reference to FileSaver.js which isn't part of any downloads for DataTables/Editor and I assume is built-in. I use the combined download format as well.

Are there any best practices for configuring exports? I'm not really doing anything complex with my button config, merely specifying columns.

{
                text: 'Excel',
                extend: 'excelHtml5',
                exportOptions: {
                    columns: [1,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37],
                    modifier: {
                        page: 'current'
                    }
                },
                title: function() {
                    return getLocationName();
                },
                filename: function() {
                    return getLocationName();
                }
            }

This question has an accepted answers - jump to answer

Answers

  • allanallan Posts: 63,237Questions: 1Answers: 10,418 Site admin
    Answer ✓

    a reference to FileSaver.js which isn't part of any downloads for DataTables/Editor and I assume is built-in

    Its in the Buttons HTML5 export file.

    There is indeed a performance regression in 1.10.17+ that effects the export. The change fixed a bug, but it causes some significant performance overhead. As such, try using 1.10.16 if you aren't already using it.

    Beyond that we'd need a link to the page so it can be performance profiled so see if there is anything else that could be done.

    Allan

  • dknifedknife Posts: 14Questions: 4Answers: 1

    I've confirmed using 1.10.16 that the regression is the cause. Using the older version I was able to process a 3,700 row export to csv in a couple of seconds.

    Is there any indication of when this will be fixed?

    Also is there a way to use the download configuration tool so I can match up the right versions of all the various plugins on the CDN for 1.10.16? I'm currently using a combined concatenated download hosting the files myself.

  • allanallan Posts: 63,237Questions: 1Answers: 10,418 Site admin

    Is there any indication of when this will be fixed?

    Not yet. The problem is that the change fixed a bug, but it causes significant overhead to make sure the data is valid and correct. I need to find a way to address both!

    Also is there a way to use the download configuration tool so I can match up the right versions of all the various plugins on the CDN for 1.10.16?

    No. The download builder uses only the latest versions of each piece of software.

    Allan

  • dknifedknife Posts: 14Questions: 4Answers: 1

    ok thanks. I've forced it to 1.10.16 for now but using the latest version of all the addons and Editor. So far no issues but not sure if there will be a compatibility issue at some point due to possibly an addon looking for something not present in 1.10.16 at which point I guess I'll need to look at dates of releases to match.

    Good luck with the export fix, hope you're able to solve the issue soon.

  • axm4814axm4814 Posts: 1Questions: 0Answers: 0

    Thank you for providing an interim solution of downgrading to 1.10.16, I was stuck for a while.

  • dknifedknife Posts: 14Questions: 4Answers: 1

    Any update on whether this will be fixed anytime soon? This prevents anyone with large table exports from being able to progress past version 1.10.16.

  • colincolin Posts: 15,237Questions: 1Answers: 2,599
    edited March 2019

    Hi @dknife ,

    We hope to release soon, no date yet though I'm afraid. You can use the nightly builds in the meantime, as the issue is resolved in there.

    Cheers,

    Colin

  • dknifedknife Posts: 14Questions: 4Answers: 1

    Excellent! That's all I needed to know, look forward to the release. Thanks!

This discussion has been closed.