node.js - Using mongoDB in Express routers

Question

I am just learning how to use node/express and mongodb, and I am a little confused about pooling connections works.

My strategy right now is to have the database connections at the router level, so each route has its own connection. My code for one of my routes looks like this:

var express = require('express');
var router = express.Router();

var config = require('../config.json');

var url = config.db.URI;

var MongoClient = require('mongodb').MongoClient
    , assert = require('assert');

var db;

// Connect to database
router.all('*', function(req, res, next) {
    console.log('Connecting to db');
    MongoClient.connect(url, function(err, database) {
        assert.equal(null, err);
        db = database;
    });
    next();
});

// GET admin list page
router.get('/', function(req, res, next) {
    res.render('lists/index');
    var coll = db.collection('lists');
    coll.find().each(function(err, obj) {
        console.log(obj);
    });
    next();
});

router.get('/new', function(req, res, next) {
    res.render('lists/new');
    next();
});

router.all('*', function(req, res, next) {
    console.log('Closing database');
    //db.close();
});

module.exports = router;

I am testing how this view interacts with my database with the two router.all functions wrapping my http requests. In my mind, any time a page in this router is requested, the server should connect to the database, allow the corresponding http request function to access the database (in this case just printing its contents to console), then close it at then end once the http request is done. However, this is not the case. As you can see I've commented out the db.close function as it throws an error saying "Can't set headers once they are sent" with it. I assume this means that I don't quite understand the Express route "pipeline" and things are not executing in the order I expect them too.

Is there a different router method I should be using? Maybe router.use? Or am I approaching this in the wrong way, should I put my database connection somewhere else?

I am using this as a reference somewhat on what is good practice as far as pooling goes, but they don't seem to close the connection? If I don't close the connection I just see my number of mongodb connections go up and up, which I don't think is a good thing. Is it just that you have to find a balance between the number and length of connections?

Answer 1

You should make your connection once and use it in all handlers.

Right now what you're doing is reconnecting to the database on every single request, even on those requests that don't use the database at all. Besides, you're calling next() before even waiting for the connection to get established.

Database connections are meant to be persistent - not one-time things, so you're likely to get very poor performance with your approach and I'm not even sure why would you want to do that. Did you have any problems with a single connection? I don't think that hammering your database like that would help. If anything, it can only make things worse.

When you connect to Mongo with the native MongoDB Node.js Driver there are some options that you can use, like:

• poolSize - Set the maximum poolSize for each individual server or proxy connection (default is 5)
• autoReconnect - Reconnect on error (default is true)

Some other interesting options are: reconnectTries, reconnectInterval, keepAlive, connectTimeoutMS, socketTimeoutMS.

You can change the values of those options if you're not happy with the defaults.

For more info see:

• Connection Failures and Retries
• MongoClient or how to connect in a new and better way
• Tutorials / Connect to MongoDB
• Reference / Connection Options / Connection Settings
• Question about node.js mongo driver auto-reconnect (mailing list)