Heatmap Details

Overview

Converting the Solr response to a map layer is straigntforward. The heatmap works better if the count data is binned into classifications. At least two independent groups have used Jenks classifications and used computed them with the geostats JavaScript library. From Wikipedia:

The Jenks optimization method, also called the Jenks natural breaks classification method, is a data clustering method designed to determine the best arrangement of values into different classes. This is done by seeking to minimize each class’s average deviation from the class mean, while maximizing each class’s deviation from the means of the other groups. In other words, the method seeks to reduce the variance within classes and maximize the variance between classes.

The number of clusters is may be based on what looks best with your data or, more simply, the number of colors in the color map you want to use.

Once clustered, the counts can be rendered on a map. This library uses OpenLayers 2.x as well as the HeatmapLayer library from Bjoern Hoehrmann. Essentially, each non-zero count is used to create a HeatmapSource and added to the HeatmapLayer. The size of the HeatmapSource object in pixels is based on the grid size of the Solr data and the number of pixels on the map.

Refined Search

When the user clicks on the time slider, selects an item format or enters a keyword, the function web/js/app/views/collections/jda.view.collection.results.js:search is called with an event and a reset flag. The search function calls fetch and web/js/app/collections/jda.collection.item.js:setSearch on this.collection. The code updates an instance of Browser.Items.Collection (jda.view.collection.results.js) but it isn't clear how to me how this affects Browser.Items.MapCollection (jda.collection.item.js).

Finding Nearby Items With Solr

We are using Solr heatmaps to replace the GPU based map tiles. The map tiles provided contained a red icon for each item. Clicking on the map calls up a list of items near the mouse event. The GPU based version of JDA sends a getFeatureInfo request and puts the list of nearby items in a pop-up window. This request includes a SQL query and looks like:

http://geops.cga.harvard.edu/getFeatureInfo&_SQL=select id from jda where dist(point(media_geo_longitude,media_geo_latitude),point(140.45471209375,37.756601563905)) < 2.746582031260301 LIMIT 50&COLUMNS=all

The request goes to the geops (GPU) server. The point is the location of the mouse event. The request returns at most 50 items. The distance (in this case, 2.74...) changes with zoom level. It is computed with:

bounds = jda.app.eventMap.map.getExtent();

latLngBounds = bounds.transform(jda.app.eventMap.map.getProjectionObject(),new OpenLayers.Projection("EPSG:4326"));

dist = 1000*Math.abs(latLngBounds.left-latLngBounds.right)/window.innerWidth;

Here is a sample response (text removed and simplified):

{"n":1,"results": [{"description":"...","display_name":"Hypercities","goog_x":15495339.15782527,"goog_y":4924635.272722726,"id":"391836","layer_type":"Tweet","media_creator_realname":"Unknown","media_creator_username":"skobl_fumiya","media_date_created":1300332945,"media_geo_latitude":40.40270,"media_geo_longitude":139.1970,"media_type":"Tweet","text":"...","thumbnail_url":"http://a0.twimg.com/profile_images/1238183171/skobl_fumiya_normal.jpg","title":"","uri":"48256182158229504","username":"hypercities"}]}

Removing the geops server implies the JDA client must obtain item data in a new way. Since this data is in Solr, we will use it as the source. The required distance calculation can use the new field of type RPT and Solr geofilt spatial filter.