Playing around with Python on the Google App Engine
I needed to run some CRON tasks for Jobspire. We’re hosted on Heroku. For those of you who don’t know what CRON is: it’s a time based scheduler for running specific tasks. Case in point, we need to send weekly mails to our customers as well as more fine grained tasks like making calls to third party APIs. We could have used Heroku Scheduler and that’s a good idea if you need to schedule tasks in intervals of ten minutes, daily or monthly.
However, for certain tasks we needed to define a much more specific execution interval. Three times a day, every two hours or even every 5 minutes.
To do so in Heroku, it is necessary to put in place a custom clock process to schedule jobs. This runs the risk of using up too many dyno-hours (units of work on Heroku), leading to unwanted expenses.
Recently I’d worked on a simple project to inform me on Yo whenever certain channels put out new videos. I wanted to try out Python to make simple apps. I’ve built this in the Python framework webapp2. Building it was fun. I implemented Google App Engines built in cron functionality. You can schedule unto 20 tasks on a free app.
And the thought came to me to use this functionality to build a CRON scheduler for our Rails app.
Disclaimer: This is probably not the best way to go about building a CRON scheduler for your app. But it’s a nice sample app to start off on Python and Google App Engine
What you’ll need:
- Google Developer Account
- Some spare time
- Basic coding experience or the ability to Google (the two are interchangeable)
We’ll be building a small Flask app. Flask is a micro framework built in Python. It links up specific urls to run specific bits of code. Take a look at the project’s page here.
So sign into you Google App Engine console and create a new project.
Then go to Source Code > Browse. Here you’ll have instructions to connect a repository. Now you need to clone my example repository to the app. The easiest way to go about this is to create a fork of my repo and clone the repository to your local machine. Then follow the instructions to push from an existing repository.
Make the appropriate changes to your app. The main parts you’ll need to edit are the cron.yml:
cron: - description: daily url: /daily schedule: every day 10:00 - description: bimonthly url: /bimonthly schedule: second,fourth mon of month 10:00 - description: monthly url: /monthly schedule: 1 of month 10:00
As you can see the cron.yml defines three main things. The url of your app that you want to call, the schedule and a description. Look at the documentation to set the schedule according to your needs. Next edit the main.py:
import requests from flask import Flask app = Flask(__name__) app.config['DEBUG'] = True @app.route('/daily') def daily(): return requests.get('http://yourcompany.com/cron/?password=longstringshouldprobablyberandom&period=daily', verify=True).content @app.route('/bimonthly') def bimonthly(): return requests.get('http://yourcompany.com/cron/?password=longstringshouldprobablyberandom&period=bimonthly', verify=True).content @app.route('/monthly') def monthly(): return requests.get('http://yourcompany.com/cron/?password=longstringshouldprobablyberandom&period=monthly', verify=True).content @app.errorhandler(404) def page_not_found(e): """Return a custom 404 error.""" return 'Sorry, nothing at this URL.', 404
Here you have to define what happens when certain urls are called by the cron. I’ve made a GET request to my own app with the period set appropriately as well as a unique string as a password to prevent any security issues.
Depending on the framework you’re using you can handle this GET request to run specific tasks. In my Ruby on Rails app I’ve handled it like so.
class HomeController < ApplicationController ... def cron if params[:password] == 'groupsiteetispuorg' if params[:period] == 'daily' # Daily code elsif params[:period] == 'bimonthly' # Bimonthly code elsif params[:period] == 'monthly' # Monthly code end end render layout: false end end
Next all you have to do is run
appcfg.py update . in the root folder of your app and you have a free CRON scheduler up and running.