Two-step loading and infinite scrolling for DBPV

Triples describing a DBpedia entity can be divided in two groups: (outgoing) properties of the entity and (incoming) reverse properties. Reverse properties are properties of some other entity which has this entity as the property value.
So a DBpedia entity has a fan-in and a fan-out, which are described by the amount of reverse properties and (forward) properties of the entity. Some entities on DBpedia have a large fan-in (for example, ~50000 other entities link to  dbpedia:United_States).

To get the triples of some entities, the following SPARQL query could be used:

SELECT ?hasprop ?v ?isprop WHERE { {<; ?hasprop ?v} UNION {?v ?isprop <} }

Two-step loading

However, this query would retrieve both outgoing and reverse properties. For entities with a large fan-in this would mean long loading times and a lot of processing at once. For this reason, a two-step loading method is used. In the two-step loading method, there are two queries: one to select the outgoing properties (forward query) and another to select the incoming ones (reverse query) ,both are executed using AJAX, so they run in parallel. Depending on the fan-in and fan-out of the entity (and various other less controllable factors), the responses may arrive in different order. When the fan-in is large, the forward query response is likely to arrive first, however.

Also, the fan-out of DBpedia entities varies less than the fan-in and seems to be relatively low. So processing of the forward query response is most likely to be instant (which can’t be said of the ten thousands of triples that may return with the reverse query).

(In)finite scrolling

Because showing everything at once would impose a heavy load on the browser for entities with high fan-in, infinite scrolling was implemented. However, with a catch, as will be explained shortly.

With infinite scrolling, normally we would load only a limited set of items of some list to show and when the user reaches the end of the list, we would query the server for more items. This is what Facebook and a lot of other sites with endless feeds do.

The catch with infinite scrolling that is currently done for DBPV is that we load everything but don’t show it. The loading part seems to have a very limited impact on the responsiveness of the page (even for dbpedia:United_States). The showing part however (where the view is updated from the model where thousands of triples are added at once), seems to have the highest impact on page response. With these observations in mind, the following infinite scrolling solution is done:

The incoming (reverse) properties of some entity are requested with a reverse query (as usual in two-step loading), but they are stored in a Angular scope variable that isn’t shown. Then when the user scrolls to the bottom (or almost the bottom), a small amount of triples from this hidden list are transferred to the list that’s shown on the page.

This solution is not really how true infinite scrolling should work, but probably provides a better user experience than showing everything at once or limiting the amount of triples in query response. Also, with proper backend support, a true infinite scrolling solution can be done later.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s