mongodb - Mongos Install/Setup in Elastic Beanstalk

Question

Looking down the road at sharding, we would like to be able to have multiple mongos instances. The recommendation seems to be to put mongos on each application server. I was thinking I'd just load balance them on their own servers, but this article http://craiggwilson.com/2013/10/21/load-balanced-mongos/ indicates that there are issue with this.

So I'm back to having it on the application servers. However, we are using Elastic Beanstalk. I could install Mongo on this as a package install. But, this creates an issue with Mongos. I have not been able to find out how to get a mongos startup going using the mongodb.conf file. For replicated servers, or config servers, additional entries in the conf file can cause it to start up the way I want. But I can't do that with Mongos. If I install Mongo, it actually starts up as mongodb. I need to kill that behaviour, and get it to start as Mongos, pointed at my config servers.

All I can think of is:

Kill the mongodb startup script, that autostarts the database in 'normal' mode. Create a new upstart script that starts up mongos, pointed at the config servers.

Any thoughts on this? Or does anyone know if I'm just being obtuse, and I can copy a new mongodb.conf file into place on beanstalk that will start up the server as mongos?

We are not planning on doing this right off the bat, but we need to prepare somewhat, as if I don't have the pieces in place, I'll need to completely rebuild my beanstalk servers after the fact. I'd rather deploy ready to go, with all the software installed.

Answer 1

I created a folder called ".ebextensions" and a file called "aws.config". The contents of this file is as follows: -

files: 
  "/etc/yum.repos.d/mongodb.repo":
    mode: "000644"
    content: |
      [MongoDB]
      name=MongoDB Repository
      baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64
      gpgcheck=0
      enabled=1
container_commands:
  01_enable_rootaccess:
    command: echo Defaults:root !requiretty >> /etc/sudoers
  02_install_mongo:
    command: yum install -y mongo-10gen-server
    ignoreErrors: true
  03_turn_mongod_off:
    command: sudo chkconfig mongod off
  04_create_mongos_startup_script:
    command: sudo sh -c "echo '/usr/bin/mongos -configdb $MONGO_CONFIG_IPS -fork -logpath /var/log/mongo/mongos.log --logappend' > /etc/init.d/mongos.sh"
  05_update_mongos_startup_permissions:
    command: sudo chmod +x /etc/init.d/mongos.sh
  06_start_mongos:
    command: sudo bash /etc/init.d/mongos.sh

What this file does is: -

• Creates a "mongodb.repo" file (see http://docs.mongodb.org/manual/tutorial/install-mongodb-on-red-hat-centos-or-fedora-linux/).

Runs 4 container commands (these are run after the server is created but before the WAR is deployed. These are: -

1. Enable root access - this is required for "sudo" commands afaik.
2. Install Mongo - install mongo as a service using the yum command. We only need "mongos" but this has not been separated yet from the mongo server. This may change in future.
3. Change config for mongod to "off" - this means if the server restarts the mongod program isn't run if the server restarts.
4. Create script to run mongos. Note the $MONGO_CONFIG_IPS in step 4, you can pass these in using the configuration page in Elastic Beanstalk. This will run on a server reboot.
5. Set permissions to execute. These reason I did 4/5 as opposed to putting into into a files: section is that it did not create the IP addresses from the environment variable.
6. Run script created in step 4.

This works for me. My WAR file simply connects to localhost and all the traffic goes through the router. I stumbled about for a couple of days on this as the documentation is fairly slim in both Amazon AWS and MongoDB.

http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html

---

UPDATE: - If you are having problems with my old answer, please try the following - it works for version 3 of Mongo and is currently being used in our production MongoDB cluster.

This version is more advanced in that it uses internal DNS (via AWS Route53) - note the mongo-cfg1.internal .... This is recommended best practices and well worth setting up your private zone using Route53. This means if there's an issue with one of the MongoDB Config instances you can replace the broken instance and update the private IP address in Route53 - no updates required in each elastic beanstalk which is really cool. However, if you don't want to create a zone you can simply insert the IP addresses in configDB attribute (like my first example).

files: 
  "/etc/yum.repos.d/mongodb.repo":
    mode: "000644"
    content: |
      [mongodb-org-3.0]
      name=MongoDB Repository
      baseurl=http://repo.mongodb.org/yum/amazon/2013.03/mongodb-org/3.0/x86_64/
      gpgcheck=0
      enabled=1
  "/opt/mongos.conf":
    mode: "000755"
    content: |
      net:
        port: 27017
      operationProfiling: {}
      processManagement:
        fork: "true"
      sharding:
        configDB: mongo-cfg1.internal.company.com:27019,mongo-cfg2.internal.company.com:27019,mongo-cfg3.internal.company.com:27019
      systemLog:
        destination: file
        path: /var/log/mongos.log
container_commands:
  01_install_mongo:
    command: yum install -y mongodb-org-mongos-3.0.2
    ignoreErrors: true
  02_start_mongos:
    command: "/usr/bin/mongos -f /opt/mongos.conf > /dev/null 2>&1 &"