node.js - What's going on with Meteor and Fibers/bindEnvironment()?

Question

I am having difficulty using Fibers/Meteor.bindEnvironment(). I tried to have code updating and inserting to a collection if the collection starts empty. This is all supposed to be running server-side on startup.

function insertRecords() {
  console.log("inserting...");
  var client = Knox.createClient({
    key: apikey,
    secret: secret,
    bucket: 'profile-testing'
  });
  console.log("created client");
  client.list({ prefix: 'projects' }, function(err, data) {
    if (err) {
      console.log("Error in insertRecords");
    }

    for (var i = 0; i < data.Contents.length; i++)  {
      console.log(data.Contents[i].Key);
      if (data.Contents[i].Key.split('/').pop() == "") {
        Projects.insert({ name: data.Contents[i].Key, contents: [] });
      } else if (data.Contents[i].Key.split('.').pop() == "jpg") {
        Projects.update( { name: data.Contents[i].Key.substr(0,
                           data.Contents[i].Key.lastIndexOf('.')) },
                         { $push: {contents: data.Contents[i].Key}} );
      } else {
        console.log(data.Contents[i].Key.split('.').pop());
      }
    }      
  });
}

if (Meteor.isServer) {
  Meteor.startup(function () {
    if (Projects.find().count() === 0) {
      boundInsert = Meteor.bindEnvironment(insertRecords, function(err) {
        if (err) {
          console.log("error binding?");
          console.log(err);
        }
      });
      boundInsert();
    }
  });
}

My first time writing this, I got errors that I needed to wrap my callbacks in a Fiber() block, then on discussion on IRC someone recommending trying Meteor.bindEnvironment() instead, since that should be putting it in a Fiber. That didn't work (the only output I saw was inserting..., meaning that bindEnvironment() didn't throw an error, but it also doesn't run any of the code inside of the block). Then I got to this. My error now is: Error: Meteor code must always run within a Fiber. Try wrapping callbacks that you pass to non-Meteor libraries with Meteor.bindEnvironment.

I am new to Node and don't completely understand the concept of Fibers. My understanding is that they're analogous to threads in C/C++/every language with threading, but I don't understand what the implications extending to my server-side code are/why my code is throwing an error when trying to insert to a collection. Can anyone explain this to me?

Thank you.

Answer 1

You're using bindEnvironment slightly incorrectly. Because where its being used is already in a fiber and the callback that comes off the Knox client isn't in a fiber anymore.

There are two use cases of bindEnvironment (that i can think of, there could be more!):

• You have a global variable that has to be altered but you don't want it to affect other user's sessions

• You are managing a callback using a third party api/npm module (which looks to be the case)

Meteor.bindEnvironment creates a new Fiber and copies the current Fiber's variables and environment to the new Fiber. The point you need this is when you use your nom module's method callback.

Luckily there is an alternative that takes care of the callback waiting for you and binds the callback in a fiber called Meteor.wrapAsync.

So you could do this:

Your startup function already has a fiber and no callback so you don't need bindEnvironment here.

Meteor.startup(function () {
   if (Projects.find().count() === 0) {
     insertRecords();
   }
});

And your insert records function (using wrapAsync) so you don't need a callback

function insertRecords() {
  console.log("inserting...");
  var client = Knox.createClient({
    key: apikey,
    secret: secret,
    bucket: 'profile-testing'
  });
      
  client.listSync = Meteor.wrapAsync(client.list.bind(client));

  console.log("created client");
      
  try {
      var data = client.listSync({ prefix: 'projects' });
  }
  catch(e) {
      console.log(e);
  }    

  if(!data) return;


  for (var i = 1; i < data.Contents.length; i++)  {
    console.log(data.Contents[i].Key);
    if (data.Contents[i].Key.split('/').pop() == "") {
      Projects.insert({ name: data.Contents[i].Key, contents: [] });
    } else if (data.Contents[i].Key.split('.').pop() == "jpg") {
      Projects.update( { name: data.Contents[i].Key.substr(0,
                         data.Contents[i].Key.lastIndexOf('.')) },
                       { $push: {contents: data.Contents[i].Key}} );
    } else {
      console.log(data.Contents[i].Key.split('.').pop());
    }
  }      
});

A couple of things to keep in mind. Fibers aren't like threads. There is only a single thread in NodeJS.

Fibers are more like events that can run at the same time but without blocking each other if there is a waiting type scenario (e.g downloading a file from the internet).

So you can have synchronous code and not block the other user's events. They take turns to run but still run in a single thread. So this is how Meteor has synchronous code on the server side, that can wait for stuff, yet other user's won't be blocked by this and can do stuff because their code runs in a different fiber.

Chris Mather has a couple of good articles on this on http://eventedmind.com

What does Meteor.wrapAsync do?

Meteor.wrapAsync takes in the method you give it as the first parameter and runs it in the current fiber.

It also attaches a callback to it (it assumes the method takes a last param that has a callback where the first param is an error and the second the result such as function(err,result).

The callback is bound with Meteor.bindEnvironment and blocks the current Fiber until the callback is fired. As soon as the callback fires it returns the result or throws the err.

So it's very handy for converting asynchronous code into synchronous code since you can use the result of the method on the next line instead of using a callback and nesting deeper functions. It also takes care of the bindEnvironment for you so you don't have to worry about losing your fiber's scope.

Update Meteor._wrapAsync is now Meteor.wrapAsync and documented.