Memory leak in fnClearTable()?
Memory leak in fnClearTable()?
I believe I may be seeing a memory leak related to fnClearTable(). I have a simple test page written that sets up sample data and uses fnAddData() to add it to the table. It then uses fnClearTable() to clear the data from the table, sets up a new set of sample data, and uses fnAddData() to put the new data into the table. This process is repeated every 10 seconds. I had the program running for over 24 hours, and periodically took screenshots showing the program, Google Chrome's Task Manager (which shows the program's "private memory" usage), and top (sorted by VIRT memory usage). When I took the first screenshot after 3,116 data reloads, Chrome showed the page's private memory usage as 617,840K. top showed the process's memory usage as 1363m (note that in the screenshots, the process' PID is 2548). Just over 24 hours later, Chrome showed the page's private memory usage as 963,552K while top showed the process' memory usage as 1737m. While this particular test doesn't show extreme growth of memory usage over time, our production application does; when I left it running overnight doing a similar reload process, Chrome had killed the tab sometime during the night.
Has anyone else seen this type of behavior? Is it a memory leak in DataTables? If so, is there any way I can help track it down and get it fixed?
You can download the test program here:
http://dl.dropbox.com/u/5415472/DataTables%20Memory%20Leak%20Test.zip
You can download the set of screenshots taken over the 24-hour test period here:
http://dl.dropbox.com/u/5415472/DataTables%20Memory%20Leak%20Screenshots.zip
Here's the code inline:
[code]
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
DataTables Memory Leak Example
var goTable;
var giColCount = 10;
var giRowCount = 10000;
var giLoadCount = 0;
function fnClearTable() {
goTable.fnClearTable();
}
function fnLoadTable() {
var aaData = [];
for (var iRowIndex = 0; iRowIndex < giRowCount; iRowIndex++) {
aaData[iRowIndex] = [];
for (var iColIndex = 0; iColIndex < giColCount; iColIndex++) {
aaData[iRowIndex][iColIndex] = sprintf("Load #%'05d
Row #%'05d
Column #%'05d", giLoadCount, iRowIndex, iColIndex);
}
}
goTable.fnAddData(aaData);
giLoadCount++;
}
function fnClearAndLoadTable() {
fnClearTable();
fnLoadTable();
}
$(function() {
for (var iColIndex = 0; iColIndex < giColCount; iColIndex++) {
$('#testTable thead tr').append('Column ' + iColIndex + '');
}
goTable = $('#testTable').dataTable({
"bJQueryUI": true,
"sPaginationType": "full_numbers"
});
$('.loadButton').click(function() {
fnLoadTable();
});
$('.clearButton').click(function() {
fnClearTable();
});
setInterval(fnClearAndLoadTable, 10 * 1000);
});
DataTables Memory Leak Example
Load Data
Clear Data
Load Data
Clear Data
[/code]
Has anyone else seen this type of behavior? Is it a memory leak in DataTables? If so, is there any way I can help track it down and get it fixed?
You can download the test program here:
http://dl.dropbox.com/u/5415472/DataTables%20Memory%20Leak%20Test.zip
You can download the set of screenshots taken over the 24-hour test period here:
http://dl.dropbox.com/u/5415472/DataTables%20Memory%20Leak%20Screenshots.zip
Here's the code inline:
[code]
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
DataTables Memory Leak Example
var goTable;
var giColCount = 10;
var giRowCount = 10000;
var giLoadCount = 0;
function fnClearTable() {
goTable.fnClearTable();
}
function fnLoadTable() {
var aaData = [];
for (var iRowIndex = 0; iRowIndex < giRowCount; iRowIndex++) {
aaData[iRowIndex] = [];
for (var iColIndex = 0; iColIndex < giColCount; iColIndex++) {
aaData[iRowIndex][iColIndex] = sprintf("Load #%'05d
Row #%'05d
Column #%'05d", giLoadCount, iRowIndex, iColIndex);
}
}
goTable.fnAddData(aaData);
giLoadCount++;
}
function fnClearAndLoadTable() {
fnClearTable();
fnLoadTable();
}
$(function() {
for (var iColIndex = 0; iColIndex < giColCount; iColIndex++) {
$('#testTable thead tr').append('Column ' + iColIndex + '');
}
goTable = $('#testTable').dataTable({
"bJQueryUI": true,
"sPaginationType": "full_numbers"
});
$('.loadButton').click(function() {
fnLoadTable();
});
$('.clearButton').click(function() {
fnClearTable();
});
setInterval(fnClearAndLoadTable, 10 * 1000);
});
DataTables Memory Leak Example
Load Data
Clear Data
Load Data
Clear Data
[/code]
This discussion has been closed.
Replies
Regards,
Allan
You can download the updated test program here:
http://dl.dropbox.com/u/5415472/DataTables%20Memory%20Leak%20Test%202010-08-18.zip
You can download the set of screenshots taken over the Firefox test period here:
http://dl.dropbox.com/u/5415472/DataTables%20Memory%20Leak%20Test%20Screenshots%20-%20Firefox.zip
Regards,
Allan
Every 15 seconds i loading 2Mb of json, clearing table and adding 3000 rows to it.
Every query increases memory usage by 10Mb
Google chrome tab crashing in 15 minutes
One of the replies:
-------
Your example doesn't contain any leaks -- if you manually collect garbage after the loop has stopped, memory usage drops back to initial value.
I think that GC is just never too aggressive unless the system starts running low on memory.
-------
So I'm not sure what to do about it. There is no leak in DataTables there that I can see, it's just the GC doesn't keep up. Another member of the forum (GregP) is interested in this topic as well - you might be interested in his recent post: http://datatables.net/forums/comments.php?DiscussionID=5118
Allan
Is there a way to manually force garbage collection? I was not aware of that.
My compromise ended up being something that's probably not suited to your goals. In my application, it is likely for the end-user to simply go away from their desk or otherwise stop interacting with the page. I hacked together a pausing API that allows you to either manually pause, programmatically pause, or have idle/away events trigger the pause. When paused, memory consumption does not grow, and if I'm not mistaken the GC will eventually circle around and free up the memory.
It's smack-dab in the middle of "workaround" territory. Although the pause will be useful to us for other reasons, I would still like to find a solution to any memory leaks, full-stop.
"""although make sure you clear any event handlers you've added to the rows / cells in the table!"""
I binded click event to every row adn then deleted it and memory was not freed
Now I'am unbind event handlers from rows before fnClearTable call and memory consumption is stable
Thanks
If you haven't started using .delegate() instead of .click() or other binding methods yet, I suggest you look into it. Delegate to a parent that is not part of the DataTables object and you don't have to manually destroy anything ever.
I've tried to use .delegate() function but memory started to leak again
What I do is
"""$("#table_id").delegate("td", "click", clickhandler);"""
Right after datatable initializaton
I don't clearing event handlers, and i've got memory leak again
I think that happens because browser keeping reference to row from event handler and can't free it's memory...
So even when using delegate I still have to clear event handlers before table fnClearTable
$('#wrapper').delegate('table td', 'click', clickHandler);
or if you couldn't be fussed manually deciding which wrapper to use, this should work in most scenarios. You just go "up" to whatever the parent is before delegating to an element within.
$('#table_id').parent().delegate('table td', 'click', clickHandler);
The wrapper/parent becomes the listener, not the table. To be honest, I haven't tried implementing it myself yet (trying to work on column resizing first) but if you get around to it, I'd love to hear your results. If I get around to it first, I'll let you know how it goes.
I don't really know which events I need to unbind, or even how to unbind them. Would be interested in specific examples of how you got memory consumption stable, if you have them!
I feel dumb for chasing a technical solution for so long now, but I think... I just have my polling interval too fast with no sanity check for completion before starting the next one. I think it's causing some timing to go wonky.
Even dumber: I'm pretty sure Allan suggested this could be my issue ages ago and I forgot about it. ;-) I'll have to implement a better polling mechanism. Instead of an interval between fnDraws, it will be an interval from when Draw completes (no matter how long that takes... could be a second or two depending on connection) to when it next polls for data.
Writing something that takes into account possible errors in the Ajax request... could be more involved than it sounds on the surface... ;-)
Imagine if you could just set "dataSource = "? ;-)
> Imagine if you could just set "dataSource = "? ;-)
Hmmm - sounds like a grand idea for a plug-in that... :-)
Allan