Which hardware should I use as a Remote Machine Learning Engineer?

Which hardware should I use as a Remote Machine Learning Engineer?

Advice · May 30, 2018

The premise of Digital Nomadism is great: As a coder, you can take your slim MacBook to a beach in Bali and just work from there. But what if you are working in Machine Learning and your work requires expensive and powerful GPUs? A few options.


There are a variety of options on how to handle the hardware when working remote - or even travelling around. Each of them has their own pros and cons, so I want to go through them.

Tower

The most obvious choice - just like in non-remote workplaces, have a beefy and powerful tower next to your desk. Sure, it’ll heat up your office - but you are in charge of your own hardware. You can upgrade your machine at any time. Plus, it might be one of the cheaper options if you do a lot of training.

By the way, working from the beach and/or cafés is not an option anymore - you’d be bound to the office.

Pro

  • In charge of your own hardware
  • Price

Con

  • Bound to office
  • Power and Heat consumption

Gamer Notebook

If you want to spend flexible but still want to work offline/on your own hardware, you might want to look into Gaming Notebooks. While Gaming Notebooks used to be nearly as wide as a Tower and had a battery runtime of under 2 hours when used - they are now much better when it comes to that! A modern gaming notebook is only a little bit wider than conventional laptops and their battery runtime is often anywhere between 7-9 hours!

It has it’s downsides though: Gaming Notebooks often have heating problems and break down easier. They are also very expensive and not easy to repair. Meaning: if you get dirt in there or short-circuit something, you’ll likely have to buy a new $2000+ notebook.

Pro

  • More flexible, still own hardware

Con

  • Price
  • Repairability
  • Heating problems

Notebook + Tower

This is a great option if you are a startup with several Machine Learning Engineers. While working, you are sitting on any notebook, such as a Macbook. As soon as you are ready to train, you are SSH-ing into a powerful ML machine with several GPUs and so on. The tower can be in an office, in a server room or anywhere else.

While you still maintain control over your hardware and can upgrade it at any time, you don’t have to worry about flexibility, battery runtimes or things breaking. If you let your MacBook fall into the sea - get a new one. The training still runs somewhere else.

Pro

  • Flexible
  • Control over hardware
  • Training is decoupled from working machine

Con

  • Internet connection needed
  • Two machines -> higher total price

Notebook + eGPU

If you are a solo-engineer and not working in a team, it might not be worth for you to get a tower and a notebook. An easy but still flexible solution would be an external GPU! You’d still be able to take your notebook to the beach or wherever, but you can let your precious GPU stay in the hotel room. As soon as you are ready for training, you can plug the eGPU in and start the training - all that while still maintaining control over your hardware.

Pro

  • Control over hardware
  • Training is decoupled from working machine
  • Cheaper than tower
  • Offline

Con

  • Not completely flexible

Notebook + Cloud [AWS/GCP]

The last solution is also preferred by single engineers or early-stage startups.

Many cloud providers, such as Amazon, Google or Paperspace allow you to train your models on their machines. This is the only solution where you don’t have to worry about your own hardware. This has good and bad sides: On one side, you don’t have to worry about broken GPUs, power usage, cooling and so on. On the other side, you won’t be able to upgrade your hardware whenever you want - The provider decided when to upgrade, and what to upgrade.

Also, if you are working in a team, this might not be cheap. You’ll get billed for every minute you are using the cloud instance - and letting it run overnight can become expensive.

Pro

  • Not in control of hardware
  • Price when alone
  • Very flexible

Con

  • Not in control of hardware
  • Price in teams

Conclusion

I have worked with many of these variations:

  • When I was doing my internship at NVIDIA, I had powerful, local machines that I could access to.
  • When doing my own work, I usually worked on AWS
  • At my current position, I work on GCP

My personal recommendation for Remote companies is to work with lightweight notebooks and a cloud provider in the beginning. It gives your employees a lot of flexibility and no one has to maintain hardware.

When the team is growing, I’d start utilizing a local machine as well. At some point it’ll start saving money, but you’ll have to have someone to maintain it (and probably a centralized office space too), so this option is not for everyone.

About the Author

Hey everybody! My name is Dominic and I am the founder of RemoteML. Additionally to working as a (Remote) Deep Learning Engineer and working on my own sideprojects, I'm also a vivid writer (I have written for Towards Data Science and O'Reilly before).

When I started working remotely, I had a lot of questions. Now, I hope to help you with yours!