close

Jobspire is a pretty image heavy site. We accept a large number of image uploads to Heroku. Here are some lessons I learned the hard way:

heroku-logo-1

 

  • The first lesson you need to learn is don’t upload to Heroku. At least not directly. The upload will lock down our entire web worker for it’s entire duration. That’s up to a minute. Unacceptable.

  • Use a javascript uploader (like jquery-file-upload) to upload directly to s3. The simplest method is to use s3_file_field. Rolling your own solution can be a little complicated at first. But once you get it working it’s like magic. You can use the s3_direct_upload gem. Or you can just read their source to build your own solution from scratch. That gem was based on a railscasts pro episode, that you can look up though you’d have to pay.
  • Wait for the upload to finish. Then make an ajax request to your application passing the new s3 url as a remote url. Then you can process the image on s3 like it was uploaded to Heroku. Except in only a couple seconds instead of up to a minute.
  • Use jquery-file-upload’s client-side image resizing. People will put up ginormous images and complain that the upload takes forever. This will make all uploads as fast as possible.
  • Don’t use thin. Use unicorn. A couple seconds is too long to be processing a request on thin. Unicorn is much more forgiving. Heroku changed their official recommendation from thin to unicorn. The problem is much less pronounced with three or four workers.
  • Don’t use rmagick. It’s a better API for complex image manipulation but uses amazing amounts of memory which can bog down the Heroku web worker. Use mini_magick instead.
  • You’ll note that I’m not using a background worker for any of this. The controller that receives the remote url can pass it to a background worker. I recommend sidekiq. But this wasn’t necessary for my application. I’d rather spend my money on another web dyno than a worker dyno on Heroku.
  • The entire process can be handled through AJAX and the user won’t be bothered.

I’m looking into handling the basic image manipulations we need through client side technologies. Then we can simply save the remote url and save on costly image manipulations. I’ll be looking into using the File API and HTML5 canvas to crop the image and then upload it to S3 as a blob. I haven’t seen any end to end guides to doing this. But I plan to do some research(browser support may vary) and try it out. I’ll put down my findings in my next article. Till then, ciao.

Kartik Luke Singh

The author Kartik Luke Singh

Full Stack Developer, UI/UX Enthusiast. Mostly harmless. You can check me out at kartikluke.com Made in India, 1993.