Doing things without cron jobs

As you start to build applications that are bigger than your average site, you can no longer rely on cron jobs to do tasks for you. That’s because the data grows and you hit a point where your cron job takes too long to run.

Take Sitizens for instance. We’ve got over 42 million rows of data in the system, and not a single cron job for that site. Yet everything works great.

Today I’ve been working on bringing this same concept over to We’ve been growing tremendously but the system still depends on cron jobs. I’ve been coding for a few hours now on removing that dependency.

It’s actually a really fun problem to have, but it’s better practice to think big and code it so you don’t have to use cron jobs in the first place.

Technically speaking doing it without cron jobs uses more CPU time on the web servers but the bottleneck these days is the database servers. We could add in another web server anytime without having to do much. But a database server no matter how you do it, will require more effort to scale.

For that reason I’ve been trying to change my style of coding to rely less on the DB server and more on the web server. I’ve been so used to coding on single machines that one way or the other didn’t matter because it was all on the machine so CPU on the DB or CPU on the web was still the same CPU.

So anyways, that’s my techy thoughts for the day. I was originally gonna make a blog post about Bill’s blog post or Patrick’s blog post but by the time I started typing I forgot what I was typing and wrote this!

6 thoughts on “Doing things without cron jobs

  1. Ok I will be the first to admit I have no idea what Tim is talking about here…but the meaning I take from this is as follows:
    “Hi everyone. This is a quick update to let you all know that I am doing things to make all the stuff we have at TimTech work better.”

  2. LMAO Patrick ,,well shorten down lol
    But Tim, how long will this remove of cron-jobs take and we yet again can have a stable site/sites? and how much impact will it have on maybe changing reset-times for servers?
    Great idea to remove the cron-jobs btw :)

  3. Tim, if you have a script that’s normally scheduled to run as a cronjob because you want to have it run, for example, once every night at 2am… how will you trigger it to run reliably if it’s not a cronjob? I’m assuming you’re using PHP here? One thing I’ve done is tag a script onto another page, so when that page is loaded (by someone visiting your site) the script checks current date and time and compares it to maybe a text file on the server where the previous run’s date and time was recorded. If a day has passed, then you run the script, otherwise you don’t. That’s how I’ve done it in a few instances; wondering if you have a better trigger, since mine is based on visitors hitting a particular page frequently enough to trigger the script when I want it to.

Leave a Reply