Blog

Dynamic Loading. Fetching Unlimited Number of Records from Server Side

Hi everybody. Today we will be speaking about Webix ways to optimize server-client communication in case of huge data. Those of you who are happy to fetch long datasets at once, please raise your hands!

Webix dynamic loading

 

Just as I’ve expected, nobody’s here.  Everybody has gone to dynamic loading which is implemented very easily with Webix data components.

webix.ui({
    view:"datatable", url:"mydata.php", datafetch:100
});

Initially, a limited number of records (e.g. 100) is loaded, while further requests are triggered by scrolling or paging automatically. Guys, isn’t it nice?

Still, such a pattern requires quite a specific format of a server response –  {data:[ /*array of records*/], pos:0, total_count:100  } – and you may ask yourself: “So what should I do, if I cannot tune the response or I don’t know the total length of server-side data?” Indeed, there are lots of web services that do not provide such information.

The article below shows the solution that relies totally on Webix public API and can be built by any of you.

For such cases Webix offers proxy objects that contain custom logic of data loading.The package already includes a number of proxies that serve different purposes, e.g. simplify connector-based loading, enable offline support, connect to indexedDB,  etc.

In the same way we can create a proxy that implements dynamic loading of an “unlimited” number of records without the need to modify server response.

A proxy object must have $proxy parameter set to true and functions that trigger on loading and saving. Its name is prefixed to the loading/saving url or both. Basically, it looks like this:

//define proxy
webix.proxy.myProxy = {
    $proxy:true,
    load:function(view, callback){ ..},
    save:function(view, update, dp, callback){ .. }
}
//use proxy
webix.ui({
    view:"datatable",
    datafetch:100,
    url:”myProxy->mydata.php
});

You can read more about proxy tuning in the documentation.

Here our goal is to redefine the loading pattern. The widget we work with should:

  • load initial data, say, 50 records;
  • each time user scrolls till the end of the dataset fetch a new portion from server.

Let’s implement this using Webix library and PHP backend.

datatable

Loading initial data

We are tuning data loading, so we need to define the load() method. It adds the count parameter to the loading url to pass the number of requested records to server.

//widget configuration
webix.ui({
    view:"datatable",
    datafetch:50, //number of records per portion
    url:”idata->./data_dyn.php
});

//proxy configuration
webix.proxy.idata = {
$proxy:true,
    load:function(view, callback){
        var url = this.source;
        url += (url.indexOf("?") == -1 ) ? "?": "&";
        url += "count="+(view.config.datafetch || 0);
        //url will look like "./data_dyn.php?count=50"
       
        callback.push({ success:this._checkLoadNext});
        webix.ajax(url, callback, view);
    },
    _checkLoadNext:function(text, data, loader){
        if(!data.json().length)
            this.data.url._dontLoadNext = true;
    }
};

In addition to default error and success callbacks, we provide a callback to check whether the server still returns data. If no data has come – finally, it should come to an end –  further requests are senseless, so we set _dontLoadNext flag to true (see its usage below).

Dynamic loading while scrolling

We need to listen to scrolling events in the widget and catch the moment when the dataset is fully scrolled to load the new data:

webix.proxy.idata = {
    $proxy:true,
        load:function(view, callback){
        ...
        view.$ready.push(this._attachHandlers);
    },
    _attachHandlers:function(){
        //this points to widget you work with
        var proxy  = this.config.url;
        if(this.config.columns)
            this.attachEvent("onScrollY", webix.bind(proxy._loadNext, this));
    else
        this.attachEvent("onAfterScroll", webix.bind(proxy._loadNext, this));
    },
    _loadNext:function(){
        var proxy  = this.config.url;
        var contentScroll =  this.getScrollState().y+this.$view.clientHeight;
        var last = this.getItemNode(this.getLastId());
               
        if(last && contentScroll>last.offsetTop && !proxy._dontLoadNext)
            this.loadNext(this.config.datafetch, this.count()+1);
    }
};

All scrollable data management widgets have the onAfterScroll event while the Datatable features a specific pair of them, the onScrollY and onScrollX, depending on the scrolling direction. We need to check it before attaching the handler to make our proxy compatible with all Webix data widgets.

The handler function checks whether the last data item is visible at the moment and, if so, calls the loadNext() method that – watch the hands! –  passes the number of records and offset position to component’s load() method as a third argument. And this method is modified by our proxy.

So, all we need to do is to tune the proxy’s load() to process these parameters:

load:function(view, callback){
    var url = this.source;
    url += (url.indexOf("?") == -1 ) ? "?": "&";

    var details = arguments[2];
    var count = details?details.count:view.config.datafetch || 0;
    var start = details?details.start:0;

    url += "count="+count;
    url += start?"&start="+start:"";
    //url will look like "./data_dyn.php?count=50&start=51"

    callback.push({ success:this._checkLoadNext});
    webix.ajax(url, callback, view);

    view.$ready.push(this._attachHandlers);
}

Basically that’s all with the client side. Your server script should use the count and start parameters for data queries, and you’ll get the needed response.

You can check the live demo or look at the code

As you can see, Webix proxies are the way to a better world where everyone can explore their developer creativity without sticking to this or that tool.

As a home task, I suggest you to connect the above proxy to paging so that nothing can stop you from loading dynamically. Feel free to ask questions in the comments. See you!

Written by
The following two tabs change content below.
Olga Kustova

Olga Kustova

Front-end web developer fond of high-quality code and effective design. Is eager to share all the tips and tricks regarding Webix usage with wide audience.
Share on Google Plus Share on Twitter Share on Facebook Share on Stumbleupon Share on LinkedIn Bookmark on del.icio.us Vote on Reddit