As you start to build applications that are bigger than your average site, you can no longer rely on cron jobs to do tasks for you. That’s because the data grows and you hit a point where your cron job takes too long to run.
Take Sitizens for instance. We’ve got over 42 million rows of data in the system, and not a single cron job for that site. Yet everything works great.
Today I’ve been working on bringing this same concept over to Trck.me. We’ve been growing tremendously but the system still depends on cron jobs. I’ve been coding for a few hours now on removing that dependency.
It’s actually a really fun problem to have, but it’s better practice to think big and code it so you don’t have to use cron jobs in the first place.
Technically speaking doing it without cron jobs uses more CPU time on the web servers but the bottleneck these days is the database servers. We could add in another web server anytime without having to do much. But a database server no matter how you do it, will require more effort to scale.
For that reason I’ve been trying to change my style of coding to rely less on the DB server and more on the web server. I’ve been so used to coding on single machines that one way or the other didn’t matter because it was all on the machine so CPU on the DB or CPU on the web was still the same CPU.
So anyways, that’s my techy thoughts for the day. I was originally gonna make a blog post about Bill’s blog post or Patrick’s blog post but by the time I started typing I forgot what I was typing and wrote this!