Tuesday 15 January 2013

Playing with NodeJS and MongoDB (Part 2)


In the part 1 we created the data storage for user data, the user crawler and the mini Node server. Today I'll show you the map that fetches the data, lookup and saves the location and finally show them on the map.


First we write the request handler for listing users. In our existing onRequestReceived() callback let's add a new router option:

  switch (parsed_url.pathname) {
    ...
    case '/list':
      doMapDataResponse(response, parsed_url);
      break;
    ...
  }


And the callback:

function doMapDataResponse(response, parsed_url) {
  collection.find(function(err, cursor){
    response.writeHead(200, {'Content-Type': 'script/javascript'});
    var items = [];

    cursor.each(function(err, item){
      if (item) {
        items.push(item);
      }
      else {
        response.write(JSON.stringify(items));
        response.end();
      }
    });
  });
}


It uses the same collection of the open db instance, fetches all the stored records and makes a JSON output. Later that will be called from the map page. I think it's more re-usable than generating it into the page content.

The other server handler we need is when we got the latitude and longitude coordinates for a location and we update the record with it - so next time we won't hammer the geolocation server. It's as simple as receiving a single record from the client and updating in the db. The router item is:

  switch (parsed_url.pathname) {
    ...
    case '/update':
      doUpdate(response, parsed_url);
      break;
    ...
  }


Using the callback:

function doUpdate(response, parsed_url) {
  var item = collection.findOne({'username': parsed_url.query.username}, function(err, item) {
    response.writeHead(200, {'Content-Type': 'text/plain'});

    if (!item) {
      response.end('Item has not been found');
      return;
    }

    item.lng = parsed_url.query.lng;
    item.lat = parsed_url.query.lat;
    collection.save(item);

    response.end('Data has been updated: ' + item._id);
  });
}


There we lookup the item - if it exists we update, or die in trauma otherwise.

Now we can start creating a map page. It's a bit tricky. We are using the NodeJS server for pretty much everything here. Because of the same origin policy (yet again bastard) we need to serve the page from the same domain and protocol. I vamped up a quick file server in Node to do that. It requires a new router item:

  switch (parsed_url.pathname) {
    ...
    case '/map':
      var fs = require('fs');
      var page = fs.readFileSync('./okcmap.map.html');
      response.writeHead(200, {'Content-Type': 'text/html'});
      response.end(page);
      break;
    ...
  }


It reads the file and toss it to the broadband. That's all we need. Let's make the page then.

<!DOCTYPE html>
<html>
  <head>
    <meta name="viewport" content="initial-scale=1.0, user-scalable=no">
    <meta charset="utf-8">
    <script src="https://maps.googleapis.com/maps/api/js?key=YOUR_GOOGLE_MAPS_API_KEY&sensor=false"></script>
    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.1/jquery.min.js"></script>
    <script></script>
    <style>
      html, body, #map_canvas {
        margin: 0;
        padding: 0;
        height: 100%;
      }
    </style>
  </head>
  <body>
    <div id="map_canvas"></div>
  </body>
</html>


Wireframe is almost by the book - we have the Google Maps API and some jQuery support. The style is a little trick we want to apply - it makes possible to stretch the map out to the full screen estate.

In the header script we make two global variables for the map object and the geocoder service object:

      var geocoder;
      var map;


Then we can initialize the map when the DOM is ready:

      jQuery(function(){
        geocoder = new google.maps.Geocoder();

        var latlng = new google.maps.LatLng(30, -30);
        var mapOptions = {
          zoom: 3,
          center: latlng,
          mapTypeId: google.maps.MapTypeId.ROADMAP
        }

        map = new google.maps.Map(document.getElementById('map_canvas'), mapOptions);
      });


Now we have a nice empty map. We can call our JSON resource though Ajax:

      jQuery(function(){

        ...

        jQuery.ajax({
          type: 'GET',
          url: 'http://localhost:8888/list',
          dataType: 'json',
          success: processMapResult,
          error: function(jqXHR, textStatus, errorThrown) {
            // Panic.
          }
        });
      });


Processing the result list splits to two branches - if there is an existing coordinate then we just create a marker and put it out. When it's missing we need to call the geolocation service, save the result on the server and then we can create a marker for the map. First case is the easier:

      function processMapResult(response) {
        for (var idx in response) {
          var user = response[idx];
          if (user.hasOwnProperty('lng')) {
            var marker = new google.maps.Marker({
              map: map,
              position: new google.maps.LatLng(user['lat'], user['lng'])
            });
          }
          else {
            // Missing coords.
          }
        }
      }


As explained the missing coordinate case will do a location lookup. Let's make a handy callback function for that:

      function lookupAddress(address, callback) {
        geocoder.geocode({'address': address}, callback);
      }


This has to be asynchronous, so we will make sure that the iterated user data is saved in a scope. In the else branch let's create this scope and the actions:

            (function(user){
              lookupAddress(user.location, function(results, status) {
                if (status == google.maps.GeocoderStatus.OK) {
                  jQuery.ajax({
                    type: 'GET',
                    url: 'http://localhost:8888/update?username=' + user.username + '&lng=' + results[0].geometry.location.lng() + '&lat=' + results[0].geometry.location.lat()
                  });

                  var marker = new google.maps.Marker({
                    map: map,
                    position: results[0].geometry.location
                  });
                }
              });
            })(user);


It's not the most elegant way to do, but it does the job. And basically that's all. You will see something like this:



The original code has some extras, such as basic error handling. You can find the source in my OkCMap GitHub repo.

If you would like to try it here you are the necessary steps:
  • install NodeJS, MongoDB and the its NodeJS connector
  • get the source
  • run the mongo server: # ./MONGO_BIN_PATH/mongod
  • run the node server file: # node okcmap.server.js
  • add the bookmarklet* to your bookmarks toolbar:
  • go to the okcupid listing page
  • hit the bookmarklet (debug console should show the requests to node)
  • open the map: localhost:8888/map


* the bookmarklet:
javascript:var _='getElementsByClassName';var $='innerHTML';var u=document[_]('match_row');for(var i=0;i<u.length;i++){var l=encodeURIComponent(u[i][_]('location')[0][$]);var m=parseInt(u[i][_]('match')[0][_]('percentage')[0][$]);var n=u[i][_]('username')[0][$];var url='http://localhost:8888/save?location='+l+'&match='+m+'&username='+n;console.log(url);var xmlHttp=new XMLHttpRequest;xmlHttp.open('GET',url,true);xmlHttp.send(null)};void(0);


---

I'm aware that this is not the best way to solve the problem. I might missed modules in NodeJS to solve sub-problems. If you know a better way, please, share.

Peter

No comments:

Post a Comment

Note: only a member of this blog may post a comment.