Building a web app with node js

NodeJS is an event driven javascript framework that makes writing asynchronous code a piece of cake! For small apps and a limited user base, this is almost magical – the code can be churned out fast, it is clean and it is perfectly unit testable. The purpose of this blog is not to sell node js though, we’ll instead look at how an application could be built easily.

The applications I work on are primarily client-server, like web apps, or mobile apps with a backend. So here’s what my toolkit with node js looks like:

  • Express JS: Sinatra like web framework that gives basic structure to the web app
  • Sequelize JS: ORM framework
  • Require JS: for modularization
  • Node DB Migrate module: just like rails db migrations
  • Mocha JS: For unit tests
  • Chai JS: For test assertions
  • Crypto: For encryptions
  • Angular JS: For the front end
  • MySQL: database

Lets keep the front end out of scope for this article and only focus on getting an app that can serve JSON over REST.

Setting up the environment

  1. Install nodejs
  2. Next, we need expressjs
  3. Run the following
    npm install -g express
    # create the app now. we'll call it 'myapp'
    express myapp
    cd myapp
    node app.js
  4. You should now see something like this “Express server listening on port 3000”
  5. Yay! You now have a basic Express JS app running!

Configuring the app

  1. The ExpressJS guide is a pretty good resource for getting started, so I would recommend that you read through it.
  2. Next, add the following in package.json under dependencies:
        "mysql": "*",
        "supervisor": "*",
        "db-migrate": "*",
        "sequelize": "*",
        "requireindex": "*",
        "mocha": "*",
        "crypto": "*",
        "chai": "*"
  3. Save package.json and run “npm install”
  4. Supervisor module is a great module for development environments and it auto reloads the app on change, so you dont have to restart your node server every time. To run the app using supervisor, just use “supervisor app.js”
  5. Requireindex is a nice module that helps get all objects from  a directory into a single object, without adding a “require” for each file
  6. Add the error handler in app.js (as described in expressjs guide)
    app.use(function(err, req, res, next){
      res.send(500, 'Something broke!');
  7. Run the app again and access using http://localhost:3000.

Adding database support

  1. Install mysql
  2. We already have the node module included in package.json (mysql), so the app is now ready to start talking to the database
  3. We’ll use node db migrate module to set up the database.
  4. Create a file database.json under myapp. The contents should look as follows:
      "dev": {
        "driver": "mysql",
        "user": "root",
        "database": "myapp"
      "test": {
        "driver": "mysql",
        "user": "root",
        "database": "myapp_test"
      "production": {
        "driver": "mysql",
        "user": "root",
        "database": "myapp"
  5. Create a db.js in the myapp directory with the following contents:
    var express = require('express'),
        Sequelize = require("sequelize");
    var app = express();
    var env = app.get('env') == 'development' ? 'dev' : app.get('env');
    // db config
    var fs = require('fs');
    var dbConfigFile = __dirname + '/database.json';
    var data = fs.readFileSync(dbConfigFile, 'utf8');
    var dbConfig = JSON.parse(data)[env];
    var password = dbConfig.password ? dbConfig.password : null;
    var sequelize = new Sequelize(dbConfig.database, dbConfig.user, password, { logging: true });
    exports.sequelize = sequelize
  6. The above uses the same file (database.json) as used by db-migrate module so all your configuration stays at one place. It also initializes our ORM framework viz. sequelizejs.
  7. To use this, just add “require(‘db.js’)” as required and get sequelize.

Configure unit tests

  1. Modify your package.json and add the following under “scripts”:
        "pretest": "db-migrate up -m ./migrations --config ./database.json -e test",
        "test": "NODE_ENV=test mocha test test/*/**"
  2. The above will ensure that your db migrations are run automatically when you run the tests, and also that you don’t have to remember longish commands for recursively running tests in sub directories.
  3. Add a basic test case under “test” directory.
    var expect = require('chai').expect,
        should = require('chai').should();
    var db = require("../db.js").sequelize;
    var DataTypes = require("sequelize");
    var assert = require("assert")
    describe('DB', function(){
        it('should check db connection', function(done){
            .query("select count(1) from users")
            }).error(function(error) {
  4. Prepare for test data – create a file _setup.js under tests and put the following:
    var db = require('../db.js').sequelize;
    var testData = [
    "INSERT INTO users (name) VALUES ('test user');",
    "INSERT INTO users (name) VALUES ('test user 2');"
    // now run the test data
        db.query(sql).success(function(){ }).error(function(e){ console.log(e); });
    console.log(">>>> starting tests...");
  5. The SQLs above are obviously just indicative and you’ll have to add your SQLs as needed.
  6. Run “npm test” to execute the tests!

Build a model

  1. Start creating your db migrations (like “db-migrate create users”) ! They will be stored under myapp/migrations directory.
  2. Run the following:
    db-migrate create users
  3. Now open the migration that has been created under migrations directory and add table definition, e.g.
    var dbm = require('db-migrate');
    var type = dbm.dataType;
    exports.up = function(db, callback) {
        db.createTable('users', {
            id: { type: 'int', primaryKey: true, autoIncrement: true },
            name: 'string'
          }, callback);
    exports.down = function(db, callback) {
        db.dropTable('users', callback);
  4. Run the migrations to create users table.
  5. Now create a directory called “models” under myapp. This is where we’ll put our models.
  6. Under models, create users.js with the following contents (or similar)
    var db = require("../db.js").sequelize;
    var crypto = require('crypto');
    var DataTypes = require("sequelize");
    var User = function(name, username, password) { = name,
     this.user_name = username,
     this.password = password
    var users_table = db.define('users', {
          name: DataTypes.STRING,
          user_name: DataTypes.STRING,
          password: DataTypes.STRING
        }, {
          timestamps: false,
          underscored: true
        });, onError) {
        var shasum = crypto.createHash('sha1');
        this.password = shasum.digest('hex');
    User.find=function(username, password, onSuccess, onError) {
        users_table.find({ where: {user_name: username, password: password}, attributes: ['id', 'name', 'user_name'] }).success(onSuccess).error(onError);
    User.lookup=function(name, onSuccess, onError) {
        users_table.findAll( { where : [ "name like ?", '%' + name + '%' ] } ).success(onSuccess).error(onError);
  7. As a reminder, we use sequelizejs for ORM and crypto for encryptions above.
  8. To use this model, all we now need is to create an object of User and call, or directly call User.find or User.lookup as needed. Notice that these take callbacks for success and error, thats because node js is a totally event driven framework and everything is synchronous. These methods don’t return anything🙂
  9. Lets add a route, create user.js under routes directory.
    var User = require('../models/users').get;
    exports.authenticate = function(req, res) {
      User.find(req.body.username, req.body.password, function(o) {
        if (o) {
        } else {
          res.send(401, "Auth failed");
      }, function(error) {
        res.send("Auth failed");
  10. And in app.js, add the route'/authenticate', user.authenticate);
  11. All done! You now have an app that can connect to the database and authenticate users!

In the next blog, we’ll see how we can quickly assemble a front end

Don’t write off Microsoft just yet!

I just bought a new laptop with the cool new Windows 8 installed. I must admit that I was a bit skeptical of how the new OS would be, but its totally taken me by surprise, and in a good way! 

First, I absolutely adore the new Metro layout. Its like a cool dashboard where I have all my tools and information handy for me to get started. From facebook to gmail to google search, everything is a shortcut on there. And its not just a shortcut! The mails tile shows new ones and facebook one shows the highlights – everything that makes you decide if you want to click that icon or not. Similarly, there are news feeds, weather updates and other handy info. Its really much more useful than the mostly empty windows desktop that exists on previous versions. And its much brighter and colorful too!

Secondly, IE 10 is a total pleasure to use. Its super fast, and feels much lighter. I also hear that it finally adheres to global standards too (yay, developers rejoice) !

Thirdly, each new window/ app is a full screen by default. No task bar, no title bar, nothing. It makes the full use of the screen – can it get better ?!

Oh well, some things require 2 clicks (including a right click) that should really just need one. Like closing windows. Yeah. You either drag a window to the bottom to ask it to go to hell (or where ever), or you right click, find yourself a cross button and close the damn thing. I wish there was a close button that would appear when I hover around one of the corners, closing a window should be a one click thing.

And I could not install Google Chrome on Win 8 – its just kept on hanging up on me.

I think the performance of the OS and system in general is pretty good, and the fact that the same system will run on mobile and desktop/ laptop computers is pretty encouraging. May be we should consider how we’ll develop apps for windows along with Android and iOS ? ha ha! I hear Windows 8 is all HTML and JS anyways!

Line Charts with d3 js

Want to do a line chart with d3? There are no ready APIs right? At least none that I could find. What I did find was (very useful!) and I hacked up a line chart taking cue from there.

Here’s an example:

And the code?

<!DOCTYPE html>
<html lang="en">
 <title>Line Charts</title>
 <script src=""></script>
 <script src=""></script>
 <script type="text/javascript">
 function getDate(d) {
 var dt = new Date(;
 return dt;

function showData(obj, d) {
 var coord = d3.mouse(obj);
 var infobox =".infobox");
 // now we just position the infobox roughly where our mouse is"left", (coord[0] + 100) + "px" );"top", (coord[1] - 175) + "px");

function hideData() {

var drawChart = function(data) {
 // define dimensions of graph
 var m = [20, 40, 20, 100]; // margins
 var w = 700 - m[1] - m[3]; // width
 var h = 360 - m[0] - m[2]; // height

data.sort(function(a, b) {
 var d1 = getDate(a);
 var d2 = getDate(b);
 if (d1 == d2) return 0;
 if (d1 > d2) return 1;
 return -1;

// get max and min dates - this assumes data is sorted
 var minDate = getDate(data[0]),
 maxDate = getDate(data[data.length-1]);

 var x = d3.time.scale().domain([minDate, maxDate]).range([0, w]);

// X scale will fit all values from data[] within pixels 0-w
 //var x = d3.scale.linear().domain([0, data.length]).range([0, w]);
 // Y scale will fit values from 0-10 within pixels h-0 (Note the inverted domain for the y-scale: bigger is up!)
 var y = d3.scale.linear().domain([0, d3.max(data, function(d) { return d.trendingValue; } )]).range([h, 0]);

// create a line function that can convert data[] into x and y points
 var line = d3.svg.line()
 // assign the X function to plot our line as we wish
 .x(function(d, i) {
 // return the X coordinate where we want to plot this datapoint
 return x(getDate(d)); //x(i);
 .y(function(d) {
 // return the Y coordinate where we want to plot this datapoint
 return y(d.trendingValue);

 function xx(e) { return x(getDate(e)); };
 function yy(e) { return y(e.trendingValue); };

$("#chart").append("<p><small><em>Please move the mouse over data points to see details.</em></small></p>");

// Add an SVG element with the desired dimensions and margin.
 var graph ="#chart").append("svg:svg")
 .attr("width", w + m[1] + m[3])
 .attr("height", h + m[0] + m[2])
 .attr("transform", "translate(" + m[3] + "," + m[0] + ")");

// create yAxis
 var xAxis = d3.svg.axis().scale(x).ticks(d3.time.months, 1).tickSize(-h).tickSubdivide(true);
 // Add the x-axis.
 .attr("class", "x axis")
 .attr("transform", "translate(0," + h + ")")

// create left yAxis
 var yAxisLeft = d3.svg.axis().scale(y).ticks(10).orient("left"); //.tickFormat(formalLabel);
 // Add the y-axis to the left
 .attr("class", "y axis")
 .attr("transform", "translate(-25,0)")

// Add the line by appending an svg:path element with the data line we created above
 // do this AFTER the axes above so that the line is above the tick-lines
 .attr("fill", "steelblue")
 .attr("r", 5)
 .attr("cx", xx)
 .attr("cy", yy)
 .on("mouseover", function(d) { showData(this, d.trendingValue);})
 .on("mouseout", function(){ hideData();});

 graph.append("svg:path").attr("d", line(data));
 .attr("x", -200)
 .attr("y", -90)
 .attr("dy", ".1em")
 .attr("transform", "rotate(-90)")
 .text("Trending Value");

 $("#chart").append("<div class='infobox' style='display:none;'>Test</div>");

var draw = function() {
 var data = [ {'date': "2012-10-01", 'trendingValue': 1000}, {'date': "2012-09-01", 'trendingValue': 900}, {'date': "2012-08-01", 'trendingValue': 1100}, {'date': "2012-07-01", 'trendingValue': 950}, {'date': "2012-06-01", 'trendingValue': 1050}];
 #chart path {
 stroke: steelblue;
 stroke-width: 2;
 fill: none;
 .axis { shape-rendering: crispEdges; }
 .x.axis line { stroke: lightgrey; }
 .x.axis .minor { stroke-opacity: .5; }
 .x.axis path { display: none; }
 .y.axis line, .y.axis path {
 fill: none;
 stroke: #000;
 .infobox {
 border:2px solid steelblue;
 box-shadow:#333333 0px 0px 10px;
 margin:200px auto;
 padding:5px 10px;
 background:rgba(255, 255, 255, 0.8);
 <body onload="draw();">
 <div id="chart">

Why Clojure scares me

There has been a lot of buzz about this relatively new language. Its a new kid on the block, it is Lisp, functional and runs on the JVM. Nice!

But its very fact that its a Lisp scares me off. When I look at Clojure code, compact as it may be, I just see a lot of brackets. And I mean a lot of them. I was reading a deck yesterday that admitted that long time Java developers may suffer from this ‘problem’, and I do fall in that category🙂

Even if I ignore all those scary brackets, the way the code is written, it looks like prefix notation (it reminds me of Yoda in Star Wars who talked in a very funny and interesting manner, but I won’t use that dialect in practice).

I mean seriously, when I first learned about numbers and Algebra, I wrote expressions as 1 + 2, and not + 1 2. The latter is not really a natural way for me. Oh yes, I can get used to the latter too, but I think for a long time, I would just be doing a translation for myself in my head. And it would surely be slow and painful.

I must admit that this is just a feeling – I have only seen Clojure yet, not practiced it. And there are many things that are good in this language. For me, Scala works wonders and I intend to stick to it for now.

New Features on Factile

Factile, as you know, is a free and open source survey platform. It generated significant interest in the first few days of its launch and I received some feedback that I thought would be good to build into the tool. I have just released a new version at and the following are the main changes:

  1. Better Navigation: Clickable bars showing the steps in survey creation/ editing to enhance user experience.
  2. Offline Survey Capability: Even if the survey participants are in transit or are in areas of unreliable connectivity, they can still ‘work offline’. i.e. They can keep on working through the survey and just submit the results once they are back on the network! Easy-peasy !
  3. Word Clouds: Factile now has the capability to build a word cloud (aka tag cloud) of free text responses. Factile combines all responses for free text questions, removes common words (aka stop words) and generates a word cloud. Just so you can analyse survey takers’ comments much more easily.
  4. Insights: This is a feature that was always planned and was delayed. You can now select a group of questions and define constraints to generate a list of top 5 combinations as picked by the users. So for example, if you wanted to see what demographic group tweets/ blogs the most, you could comine the demographics related questions, add a constraint around tweeting/ blogging and get sorted and aggregated insights.
  5. Custom URI: Would you like your survey to be called instead of the long system generated identifier? You can do so now!

Obviously, Factile is freely downloadable and customizable (and these surveys work on mobile devices, as would be convenient to use on a mobile device and not as small difficult to click HTML radio buttons).

You can let me know if you want a feature/ faced a bug by logging an issue on github, or just leave a comment on this blog! You can also send an email at

I did a quick video on the use of Factile a week or so back and uploaded to YouTube. You can watch it here.


Factile – A free online survey tool

On Friday last week, I launched Factile at It is a free and open source survey platform that I created and aims at making the job of data collection and analysis simpler.

It supports a variety of question types (Text boxes, Radio buttons, Check boxes, Combo boxes (dropdowns), Text areas, Plain texts, matrix of choices/ rating scales). The surveys generated are fully mobile device compatible. So you can create a survey and share it with people to take on iPads/ iPhones/ BackBerry/ Samsung S2s, really, anywhere! It is truly free and unlimited. There is zero usage cost, and you can add as much content to your survey as you like. And, if you wanted to download it and install on your infrastructure, you can!

Whats more? You can define logic in the survey, you can build charts of the captured data, customize the appearance of the survey to match your requirements and the best is, you can request for missing features and I’ll be happy to oblige!

Check out and let me know of what you think.