How To Install The Nylas Sync Engine On Bare Metal (Ubuntu Server)

After explaining how to build the Nylas N1 email client in my previous post, it’s not time to configure the Sync Engine itself. But before delving into this, a little introduction is in order.

Hosted or self-hosted?

There are two ways to use Nylas:

  • Using their own servers (through a subscription)
  • Hosting the Sync Engine yourself


Using their own servers has the obvious advantage that you don’t have to take care of anything yourself, and you will also be able to download already-built apps for each operating system without the need to build them yourself. At $7 per month for the cheapest plan, this is quite steep for an email service, but I don’t think the price is a concern. If a service is worth the money, I have no issues paying for it. The massive disadvantage, in my opinion, is that it obviously creates a privacy concern: in order for the email client to work, all your contents will also be stored on Nylas servers. Mind you, I am not just talking about your email credentials, I am talking about all your content. From Nylas’s Terms of Use:

The API and Sync Engine Services are offered on both a hosted and downloadable basis. If you choose to access the API and Sync Engine Services on a hosted basis, Nylas will store copies of your end users’ email inboxes, calendar events, and contacts information on Nylas’ servers and keep such copies synced.

While I am sure the guys over at Nylas are taking care of the security of their systems, I just don’t feel comfortable adding another player in the mix who has access to all this information. So for me this option is definitely a no go.

The alternative is to run the Sync Engine yourself. The project is open source and, in this case, you won’t have to give anyone else access to your emails and related information. Taken again from the Terms of Use:

Alternatively, if you choose to download the software implementing the API and Sync Engine Services (“API and Sync Engine Software”) onto a local machine, then your end users’ email inbox data, calendar events, and contacts information will be copied, synced, and stored on such local machine.

And this is something I like. Therefore, off we go configuring the sync engine.

VM or bare metal?

If you look at the official installation instructions for the Sync Engine, you can see that Nylas simply recommends downloading the project and using Vagrant to run the VM with the Sync Engine. While this is definitely the easiest and quickest solution if you are just playing out with the project, it is not mandatory.

The good thing is that, even if it’s maybe not clear from the documentation, running a VM is not necessary. You can definitely install the Sync Engine on bare metal, which is what this guide is going to focus on.

For this, I am going to use Ubuntu Server 16.04-1 LTS.

How to install the Nylas Sync Engine

  1. sudo apt-get update
  2. sudo apt-get upgrade
  3. sudo apt-get install libssl-dev
    1. There is currently a little bug in the install script that doesn’t install the libssl-dev package, so we need to install this manually
  4. git clone https://github.com/nylas/sync-engine.git
  5. cd sync-engine
  6. sudo -H ./setup.sh

    If you get a warning about upgrading to pip version 8.12 from a previous version, run

    sudo -H pip install --upgrade pip

    and then run the setup script again

    1. The -H option is used to avoid pip from complaining that the user root is not the owner of the current cache directory in use
  7. Launch the API service:
    bin/inbox-api
  8. Launch the sync service:
    bin/inbox-start
  9. Add your first email account:
    bin/inbox-auth email@domain.com

And there you go, your Nylas Sync Engine is now up and running and you can connect to it from your N1 client. You simply have to choose the “Hosting your own sync engine?” option on the N1 welcome screen:

Nylas_N1_1.png

Enter your Sync Engine IP and use port 5555 and you are ready to go ;)

screen-shot-2016-10-13-at-17-16-30

39 Comments

  1. Hey I am getting an error about egg_info when I run the ./setup.sh command.

    Command “python setup.py egg_info” failed with error code 1 in /tmp/pip-build-a5VWKs/imapclient/

    I have updated pip, setuptools, reinstalled easy_install, and nothing has worked.
    I’m running this on a digitalocean server with Ubuntu 14.04.

    Any idea what the solution could be?

  2. Maby a stupid question, but when you use nylas services. You use their smtp servers to sent mails.
    What do you use if you setup this self-hosted sync (magic system) by yourself?
    Does it connect with an smtp(s) server, or will it sent emails from the bare metal machine.

    (I want to ask because mail ports are blocked right hire, haha)

    • This is a sync engine, not a mail server, which means that you are going to have to add some email accounts to it so it can do its job, i.e. sync your mail. Consequently, it will use the SMTP server connected to the mail accounts you add to it.

      To give you an example, if you are going to add your Gmail account to the Sync Engine, and you send an email from this account, it will send it using your Gmail account’s SMTP server ;)

      I suppose that if you use their hosted version it’s going to be the same thing, with the only difference that the sync engine will be running on Nylas’s cloud rather than on your own server.

      • Haha, I had the question because the PRO version uses AMAZONE servers to ship email to you

        • Technically, Nylas uses Amazon to store the sync engines if I understood things correctly, then the emails are sent using your specific account SMTP server. Nylas is simply an interface that makes dealing with IMAP/POP/SMTP more straightforward, but all the work is ultimately done by your email accounts ;)

  3. Another question,

    If I run this over an digitalocean server (VPS) etc.
    Is it save, because I don’t need a password to connect with nylas-sync..,

    • Safe? That’s a loaded question, it depends on how you configure your DigitalOcean server :) You do need a password to connect to your server anyway, you can’t just access the sync engine without providing a password, you need to get to the machine that hosts it first.

      Although perhaps I just misunderstood your question.

      • I am running Nylas sync engine on my desktop itself. (Desktop with Ubuntu 15.10, in a few seconds 16.10).
        When I try to connect nylas with the server, I did not have to enter some password :/

        • Oh I see what you mean now! Yes, you are not asked for a password there but you are just connecting to the API endpoint in this case, nothing more. For example, you still have to add your email accounts to the N1 app if you want to do something with it. I think by default the API only exposes the /accounts endpoint, most other requests need to be authenticated.

  4. Hi Daniel
    those instructions are just what i was looking for.
    following them to install on a VM with mint 18.1 using NAT network adapter and also added an account.
    Everything seems to be fine so far. no problems.
    But trying to connect the client to engine using http://127.0.0.1:5555 or http://localhost:5555 or http://10.0.2.15:5555 (VM NAT address), i get an error ‘The request to http://127.0.0.1:5555/accounts failed’.
    Since i am a novice level user for linux what am i doing wrong?
    any ideas?

    thanks

    • Hi Fillip,

      Glad to hear the instructions were helpful :) Is the API service running on your Nylas box? Or in other words, what happens when you run this command on your Sync Engine?

      bin/inbox-api

      • Daniel thanks for replying
        on a terminal ->bin/inbox-api
        some part copied here
        [INFO] * Running on http://0.0.0.0:5556/ (Press CTRL+C to quit)
        [INFO] {“total_time”: 60.14, “pending_avgs”: {“1”: 0.0, “5”: 0.0, “15”: 0.0}, “level”: “info”, “timestamp”: “2017-01-17T13:29:06.757964Z”, “module”: “inbox.instrumentation:144”, “times”: “{None: 3.13, ‘hub’: 56.67}”, “total_switches”: 1669, “greenlet_id”: 140553975130224, “event”: “greenlet stats”}

        and on another terminal ->bin/inbox-start
        [DEBUG] {“account_id”: 1, “level”: “debug”, “timestamp”: “2017-01-17T13:38:02.977111Z”, “provider”: “yahoo”, “module”: “inbox.mailsync.backends.imap.generic:323”, “state”: “poll”, “greenlet_id”: 140100639422256, “folder”: “Inbox”, “event”: “polling”}

        and they keep printing…

  5. Many thanks, I am looking forward to giving this a go. Wish me luck.

  6. Thanks for the tutorial!
    I have an issue with it though. My mysql root user is set with a non-empty password. Is there any way to tweak the setup.sh script to let installer work through?
    Right now I’m getting the following error:

    File “/usr/local/lib/python2.7/dist-packages/MySQLdb/connections.py”, line 204, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
    sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (1045, “Access denied for user ‘root’@’localhost’ (using password: YES)”)

    which must be related to the fact that my database root user has a custom password.

  7. Thanks for the post it was very helpful. The VM installation method warns of security risks e.g. passwords in plaintext in databases.. are there any further things I need to do other than this guide to secure my installation? Thanks

    • I haven’t seen this warning yet (or perhaps I did but I don’t remember now), do you perhaps have a screenshot of it?

      In any case, if Nylas stores your accounts’ passwords in the MySQL database in unencrypted form, I am not sure there is much you could do besides the typical things: use a strong password for your MySQL user, only allow localhost database access etc.

      To make some of this easier, you could also run mysql_secure_installation:

      https://dev.mysql.com/doc/refman/5.7/en/mysql-secure-installation.html

  8. Quick typo fix: pip install –upgrade pip should have two dashes

    • So sorry for the late reply Lucien, these last few days have been super busy! Thank you for pointing this out, I have updated the article ;)

  9. Hi!
    I’ve successfully managed to deploy Sync Engine onto my Ubuntu server (based on LAN). However, I want to know if there is any automatic way of execution of these 2 commands which are necessary for the engine to work:
    1. bin/inbox-start
    2. bin/inbox-api

    Please also do tell, how to make them (the commands) run via a tty console.

    Thanks,
    Abdullah

    • Hi Abdullah!

      If you don’t want to run those two commands manually, I suppose you could always create a cronjob to do that automatically. For example, you could choose to run them every time after a reboot (using cron’s @reboot option, for example).

      Re your question about the console: I am not sure I understand it completely. Do you mean you want to run those commands from Terminal on another machine? I mean, all of the commands in this blog post need to be run on a CLI anyway, so there is something I am not quite getting.

  10. Hi there, thanks for writing this up. I’m definitely thinking about getting this set up now. Someone else asked before but I’m not clear on it yet…

    Once you set up the sync server and have it all running. Do you get asked for a password in the Nylas App when you’re connecting to it to have access to the accounts you’ve added? Or do you add the accounts in the Nylas App? What happens if someone else connects to the server via the Nylas App too?

    Basically, I’m asking if it’s just an open port into my inbox or not – because I’ve not seen anything to suggest otherwise in my (fairly brief) research so far. But I have the same concerns that you have.

    • Hey there! Thanks for reaching out :)

      So, I don’t have access to my Nylas test box right now to confirm this, so I am saying all this from memory: you won’t be asked for a password when you connect to the Nylas API URL (in other words, when you reach the screen in the last screenshot in this post). However, after doing that, you still have to add your email accounts to the Nylas app after this connection, and you will be asked for your email credentials at this point.

      If you find out something more about this before I can get access to my Nylas box again, make sure to post an update here :)

      Cheers!

      • Cool, that does sounds sensible. I’ll see what happens when I try it out. I’m planning on getting it set up this evening (around 4 hours from now I guess my time). I’ll report back with what I find!

        • That’d be awesome mate, thanks! Looking forward to your update :)

          • Well, I’ve actually got the thing running. Started up Nylas Mail (the new one) and I can’t find the option for hosting your own server anymore! I may have to revert to N1 to see if I can access it in there. I know they’ve already fixed some bugs I found in N1 in the newer Mail app though…

            In any case, I’ve not got as far as any security stuff yet.

          • Okay, got past there with N1, it does look like you log into your accounts in the sync-engine, and the client just connects. I’ve seen somewhere that you can setup a proxy using HTTP basic auth to protect the sync engine. It’s not really an ideal solution IMO – and getting this up and running and maintained seems like it might be hit and miss at best as it’s not run like the hosted version… which is a real shame, because it is hard to trust something that has access to all your emails these days!

          • Thanks for testing this! So there is an option to self-host even in the latest apps, yes?

            My memory probably failed me, as I seemed to remember that, even after adding the email accounts on the sync engine, you still had to add them inside Nylas Mail, under Preferences. I will have to play with this a little bit more again as soon as I can then, there have been quite a few updates to the service that I probably missed something along the road, even if I am subscribed to their email newsletters.

            And yes, I agree with your concerns on email security, that’s what made the self-hosted version of Nylas so appealing to my eyes. You could have access to an awesome-looking email client without having to bring another third party into the picture (there are so many already anyway!) And of course you always have additional methods to protect your server from unauthorized clients connecting to it. The self-hosted version is for techies, so I wouldn’t be surprised if we had to take care of that on our own.

          • Hi Daniel,

            There does NOT appear to be an option to connect to the sync server in “Nylas Mail” which was recently replaced to supercede Nylas N1 (and does include several bug fixes…).

            Aye, I do feel with the way it’s not documented (and some of the documentation seems to have disappeared), and the way the client is going that they don’t really want you to do this – which makes it more of a hassle. Doubly so if it means running an older, buggier client. It’s a real shame, because the client is excellent.

            I’m starting to think I should just build my own super opinionated, ultra-basic email client that just does the things I want, and nothing more…

          • Oh man, that’s a bummer, I didn’t know about that! I’ll see if I can find out something more about the latest apps and the future of the self-hosted sync engine. I have always felt that Nylas had pretty bad (if not non-existent) documentation, and that there is kind of a “don’t give a shit” attitude about the docs in general and the bug fixes, which is a pity seeing that the project has a lot of potential.

            I am sure they are going to feel the backslash soon enough if that’s how they want to continue doing business though, I just hope that they can become more transparent on everything they do.

            I have also thought many times about developing a very basic email client myself, but it would really be a PITA. Never say never though, if I keep switching email clients every other month I might end up doing it for real one day :D

Leave a Reply

© 2017 Daniel's TechBlog

Theme by Anders NorénUp ↑

%d bloggers like this: