Using AJAX Solr in combination with DataTables
Using AJAX Solr in combination with DataTables
As all my predecessor have said this is a great plugin and I've been using it for months without problems. Thanks for all your work on this!
I'm using AJAX Solr (http://github.com/evolvingweb/ajax-solr) to access my Solr instance and I use it to make the AJAX requests to the Solr server and then use fnClearTable, fnAddData and fnDraw to build the new DataTable (I don't want to use DataTable's server-side support because the format is quite different).
I'm sure there'll be quite a bit more problems in the future with this combination (it seems a bit like a bastardization, both projects weren't quite made for this use case) but at the moment this is the one I have:
I'd like to use DataTables for the nice options like row coloring, the display of sort directions and sorted columns etc. but the problem I currently have is that while a click on the column headers should sort those columns it requires a custom call to AJAX Solr. I had a look at _fnSortAttachListener but that does seem to call _fnSort in every case and that tries to sort the data unless server-side is enabled.
I could enable server-side and provide a fnServerData function that does what I want but is it a problem if I never call the callback (as I want to add the data myself as explained above - different format etc.).
Short version of the question: I want to use server-side processing but don't want to convert the data to the JSON format DataTables requires but use the API (fnAddData) instead to add the data myself.
I'm using AJAX Solr (http://github.com/evolvingweb/ajax-solr) to access my Solr instance and I use it to make the AJAX requests to the Solr server and then use fnClearTable, fnAddData and fnDraw to build the new DataTable (I don't want to use DataTable's server-side support because the format is quite different).
I'm sure there'll be quite a bit more problems in the future with this combination (it seems a bit like a bastardization, both projects weren't quite made for this use case) but at the moment this is the one I have:
I'd like to use DataTables for the nice options like row coloring, the display of sort directions and sorted columns etc. but the problem I currently have is that while a click on the column headers should sort those columns it requires a custom call to AJAX Solr. I had a look at _fnSortAttachListener but that does seem to call _fnSort in every case and that tries to sort the data unless server-side is enabled.
I could enable server-side and provide a fnServerData function that does what I want but is it a problem if I never call the callback (as I want to add the data myself as explained above - different format etc.).
Short version of the question: I want to use server-side processing but don't want to convert the data to the JSON format DataTables requires but use the API (fnAddData) instead to add the data myself.
This discussion has been closed.
Replies
Basically if you want to use server-side processing, you'll need to use the format that DataTables requires. Without that, you'd need to change the core code to cope with whatever data you give it. Can you not use fnServerData with a little customisation to convert from whatever your Ajax library is returning to what DataTables needs?
Regards,
Allan
I had hoped I could avoid the conversion but I'll try it and see if any other problems arise.
Regards,
Allan
My current approach is to write a custom Manager[1] that accepts a callback in addition to the widgets and a custom fnServerData function.
The fnServerData first has to convert the aoData to something that is more easily usable:
[code]
var data = {};
$.each(aoData, function() {
data[this.name] = this.value;
});
[/code]
It then does a couple of things like this: Manager.store.addByValue('q', data["sSearch"] || "*:*");
It adds all the relevant parameters from aoData to the ParameterStore from ajax-solr. The last thing to do is to call Manager.doRequest with a callback function that takes the Solr result and converts it to the expected format: reply["iTotalDisplayRecords"] = data.response["numFound"]
One problem here is: I've got no "iTotalRecords" as Solr currently doesn't provide that value. I'd have to do a search for all documents first and cache that somehow. It's doable but it is also functionality I don't need. I haven't yet looked into if DataTables allows me to somehow disable this. It currently displays something like this:
Showing 1 to 25 of 2812 entries (filtered from undefined total entries).
As an intermediate result of this I can tell that it would be a lot easier if the _fnAjaxUpdate were to provide an associative array with all the necessary values (an array of the sorted columns for example) and let the default fnServerData convert that into the "DataTables internal format". This way custom fnServerData functions would be easier to write.
Even better then would be to be able to make the callback (more) customizable. I might try to just reimplement _fnAjaxUpdateDraw instead of converting the results manually to a format it understands.
My next challenges include adding an autocomplete function to the search field DataTables provides and interpreting the sort parameters.
This is very much a work in progress and I don't know if I ever finish it or if I just leverage the ajax-solr functions. The "only" things I'm using from DataTables at the moment are the formatting/display options and the sort handling/coloring of the columns etc.
Either way I'll keep you updated.
[1] http://evolvingweb.github.com/ajax-solr/docs/symbols/AjaxSolr.AbstractManager.html
Regarding the array of key/value pairs used by _fnAjaxUpdate - it's a huge pain this. It's what jQuery uses for it's array serialisation - hence why it needs to be used here. But aren't you only interested in transforming the return from the server (having had the server deal with the POST/GET parameters) into a 2D Javascript array?
Allan
Sebastiaan
Will be back s00n!
Sebastiaan
AJAX Solr
@import "css/demo/site_jui.css";
@import "css/demo/demo_table_jui.css";
/*
* Override styles needed due to the mix of three different CSS sources! For proper examples
* please see the themes example in the 'Examples' section of this site
*/
.dataTables_info { padding-top: 0; }
.dataTables_paginate { padding-top: 0; }
.css_right { float: right; }
#example_wrapper .fg-toolbar { font-size: 0.8em }
#theme_links span { float: left; padding: 2px 10px; }
<!-- JQuery http://www.jquery.com -->
<!-- JQuery UI http://www.jqueryui.com/ -->
/**/
<!-- CUSTOM AJAX-SOLR DataTable Widget http://www.datatables.net/ -->
<!-- CORE AJAX-SOLR http://evolvingweb.github.com/ajax-solr/ -->
/**/
<!-- Custom AJAX-SOLR Manager for DATATABLES -->
<!-- Custom AJAX-SOLR DataTable Widget -->
/**/
/* Initialize AjaxSolr and do the first request */
var Manager;
$(document).ready(function() {
(function ($) {
$(function () {
Manager = new AjaxSolr.DataTableManager({solrUrl: 'http://localhost:8983/solr/'});
Manager.fields = "field_link_workflow_s,process_id,instance_id,place_id";
Manager.addWidget(new AjaxSolr.DataTableWidget({
id: 'table',
target: '#table'
}));
Manager.init();
Manager.store.addByValue('q', '*:*');
Manager.store.addByValue('fl', Manager.fields);
Manager.store.addByValue('rows', '10');
Manager.doRequest();
});
})(jQuery);
});
[/code]
[code]
AjaxSolr.DataTableManager = AjaxSolr.AbstractManager.extend({
fields: null,
datatable: null,
fnCallback: null,
convertedData: null,
executeRequest: function (servlet) {
var self = this;
if (this.proxyUrl) {
jQuery.post(this.proxyUrl, { query: this.store.string() }, function (data) { self.handleResponse(data); }, 'json');
} else {
jQuery.getJSON(this.solrUrl + servlet + '?' + this.store.string() + '&wt=json&json.wrf=?', {}, function (data) {
self.handleResponse(data);
// COLUMNS
var currentColumns = [];
$.each(self.response.response.docs[0], function(key, row) {
currentColumns[currentColumns.length] = {"sTitle":key};
});
// BUILD THE ROWS
var currentData = [];
$.each(self.response.response.docs, function(key, row) {
var newrow = [];
$.each(row, function(key, value) {
newrow[newrow.length] = value;
});
currentData[currentData.length] = newrow;
});
self.convertedData = {};
self.convertedData["iTotalDisplayRecords"] = self.response.response.numFound;
self.convertedData["iTotalRecords"] = self.response.response.numFound;
self.convertedData["aaData"] = currentData;
self.convertedData["aoColumns"] = currentColumns;
self.fnCallback(self.convertedData);
});
}
},
setCallback: function (dTable, dtFnCallback) {
this.datatable = dTable;
this.fnCallback = dtFnCallback;
}
});
[/code]
[code]
(function ($) {
AjaxSolr.DataTableWidget = AjaxSolr.AbstractWidget.extend({
currentColumns: [{ "sTitle": "Dummy" },{ "sTitle": "Dummy" },{ "sTitle": "Dummy" },{ "sTitle": "Dummy" }],
currentData: [["Some text here or something else",434,"OFF","@#2342"]],
init: function () {
var self = this; // So we can use the this reference from inner function
// FIRST BUILD THE COLUMNS from the first row (change this in the future)
this.currentColumns = [];
$.each(self.manager.fields.split(","), function(key, value) {
if(self.currentColumns.length == 0)
self.currentColumns[self.currentColumns.length] = {"sTitle":value, "sType":"html"};
else
self.currentColumns[self.currentColumns.length] = {"sTitle":value};
});
// Initialize data
$(this.target).dataTable({
"bJQueryUI": true,
"sPaginationType": "full_numbers",
"bProcessing": true,
"bServerSide": true,
"sAjaxSource": null,
"fnServerData": function (sSource, aoData, fnCallback ) {
// Rownumber
var rows = getSetting(aoData, 'iDisplayLength');
self.manager.store.addByValue('rows', rows);
// Filtering
var q = getSetting(aoData, 'sSearch');
if(q.length > 0)
self.manager.store.addByValue('q', q);
else self.manager.store.addByValue('q', '*:*');
// Fields
self.manager.store.addByValue('fl', self.manager.fields);
// Paging
var start = getSetting(aoData, 'iDisplayStart');
self.manager.store.addByValue('start', start);
// Sorting
var sortcolnr = getSetting(aoData, 'iSortCol_0');
var sortcol = "";
if(self.currentColumns[sortcolnr] != undefined)
sortcol = self.currentColumns[sortcolnr].sTitle;
var sortdir = getSetting(aoData, "sSortDir_0");
if(sortcol.length > 0 && sortdir.length > 0)
self.manager.store.addByValue('sort', sortcol + " " + sortdir);
self.manager.doRequest();
self.manager.setCallback(this, fnCallback);
},"aoColumns": this.currentColumns
});
},
afterRequest: function () {
var self = this; // So we can use the this reference from inner function
// FIRST BUILD THE COLUMNS from the first row (change this in the future)
this.currentColumns = [];
$.each(this.manager.response.response.docs[0], function(key, row) {
self.currentColumns[self.currentColumns.length] = {"sTitle":key};
});
// BUILD THE ROWS
this.currentData = [];
$.each(this.manager.response.response.docs, function(key, row) {
var newrow = [];
$.each(row, function(key, value) {
newrow[newrow.length] = value;
});
self.currentData[self.currentData.length] = newrow;
});
}
});
function getSetting(aoData, settingName) {
var returnValue;
$.each(aoData, function(key, value) {
if(value.name == settingName) {
returnValue = value.value;
return false;
}
});
return returnValue;
}
})(jQuery);
[/code]
That's about it. I hope you can somehow use it.
Sebastiaan