Python Automation #6: Scheduling — Making Scripts Run While You Sleep

If you’ve followed along through part 5, you now have a script that collects data, organizes it into Excel, and delivers the results by email or messenger. But one thing is still missing: the person pressing the run button every day is still you. Automation isn’t really automation until the running is automated too. In this post we make the script fire on its own at a fixed time.

• #1 First steps with scripts
• #2 Excel automation
• #3 Web scraping: static pages
• #4 Web scraping: dynamic pages
• #5 Email and notifications
• #6 Scheduling ← this post
• #7 Packaging as a CLI tool

There are two broad approaches. One is letting the OS keep time and wake your script up — cron or Task Scheduler. The other is keeping a Python process alive that keeps time itself — APScheduler. Let’s start with the OS schedulers, which are enough for most run-and-exit automation.

cron: the default on macOS and Linux #

cron is the scheduler that has shipped with Unix-like systems for decades. Run crontab -e in a terminal to open an editor, then register one job per line as “five time fields + the command to run.”

* means “every,” */5 means “every 5,” and 1-5 is a range. A line that runs a daily report at 7 AM on weekdays looks like this:

0 7 * * 1-5 /home/me/scripts/.venv/bin/python /home/me/scripts/daily_report.py >> /home/me/scripts/cron.log 2>&1

The trailing >> appends standard output to a file, and 2>&1 sends error output to the same file. Capturing output like this from the very first registration makes debugging far easier. And here’s the trap people hit most often: environment variables. cron does not read your terminal’s .zshrc or .bashrc — it runs commands with an almost empty PATH. That’s why python daily_report.py, which worked fine in your terminal, dies with “command not found” under cron. The fix is one principle: write everything as absolute paths.

• The Python executable and the script: not python daily_report.py, but the virtual environment’s /home/me/scripts/.venv/bin/python /home/me/scripts/daily_report.py
• Files the script reads and writes: cron’s working directory is not your project folder, so build paths from pathlib.Path(__file__).parent instead of using relative paths

launchd on macOS, Task Scheduler on Windows #

crontab works on macOS too, but Apple’s officially preferred mechanism is launchd. The practical difference is sleep handling. If your Mac is asleep at the scheduled time, cron simply skips that run, while launchd’s StartCalendarInterval runs the missed job once after the machine wakes up. Configuration means writing a plist file, which is wordier than cron — so just file this away and look it up when you need that behavior on a laptop.

Windows has Task Scheduler instead of cron. In the GUI, click “Create Basic Task,” set the trigger (daily at 7 AM), the action (program: C:\Users\me\scripts\.venv\Scripts\python.exe, arguments: the script path), and the start-in directory (the script folder), and you’re done. The absolute-path principle applies on Windows just the same.

Solving it inside Python: APScheduler #

If OS schedulers are “the OS wakes the script up,” APScheduler — installed with pip install apscheduler — is a Python process keeping time itself.

from apscheduler.schedulers.blocking import BlockingScheduler

scheduler = BlockingScheduler()

@scheduler.scheduled_job("cron", day_of_week="mon-fri", hour=7, minute=0)
def daily_report():
    ...  # call the job function built through part 5 here

scheduler.start()

Run it and the process stays alive, calling the function at each registered time. Besides the cron trigger, which uses cron syntax as is, there’s an interval trigger for things like “every 30 minutes,” and since it’s all the same Python code, there are no virtual environment or path issues. There is one decisive difference, though: if this process dies, every schedule stops with it. Close the terminal or reboot, and it’s over. cron keeps working as long as the OS is alive; with APScheduler, keeping the process alive is your responsibility.

Which one to use #

Did it actually run? logging instead of print #

The moment a script goes onto a scheduler, it no longer runs in front of your eyes. print has nowhere to go without a terminal, so switch to logging, which writes the run record to a file.

import logging
from pathlib import Path

logging.basicConfig(
    filename=Path(__file__).parent / "automation.log",
    level=logging.INFO,
    format="%(asctime)s %(levelname)s %(message)s")

Note that the log file path is also built from Path(__file__).parent — an absolute path. Sprinkle logging.info("processed 32 rows") through the job, and in the morning you can open automation.log and immediately see whether it ran last night and how much it processed.

Preparing for failure: get an alert when it dies #

The scary moment in automation isn’t when something fails — it’s when something fails and you don’t know. The classic case: a site changes its structure, the scraper dies, and you find out a month later. Reuse the notification function from part 5 and wrap the main logic.

import logging
import traceback
from notify import send_message  # notification function from part 5

if __name__ == "__main__":
    try:
        main()
        logging.info("job succeeded")
    except Exception:
        logging.exception("job failed")
        send_message(f"[Automation failure] daily_report\n{traceback.format_exc()[-500:]}")
        raise

On success, one log line. On failure, the full traceback goes to the log and the last 500 characters go to your messenger. The final raise re-throws the exception so the exit code is nonzero, recording the run as a failure on the scheduler’s side too.

Free and serverless: GitHub Actions #

Everything so far requires your own machine to be on. If you want to shut the computer down and go to bed, the schedule trigger in GitHub Actions is the simplest free option. Push the script to a repository and add a single file at .github/workflows/daily.yml.

name: daily-report
on:
  schedule:
    - cron: "0 22 * * *"   # UTC — this is 7 AM in UTC+9
jobs:
  run:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.13"
      - run: pip install -r requirements.txt
      - run: python daily_report.py

It’s the same cron syntax, with three things to remember. The times are in UTC, so you have to convert from your local timezone. Exact start times aren’t guaranteed — runs can start minutes or even tens of minutes late. And if the repository has no activity for 60 days, the schedule is automatically disabled. Secrets like tokens and webhook URLs don’t go into the code; put them in the repository’s Settings → Secrets and read them as environment variables.

Wrap-up #

The flow this post built:

• Run-and-exit scripts go on an OS scheduler (cron / Task Scheduler); the environment-variable and path traps are solved by using absolute paths everywhere
• Periodic work inside an always-on app goes to APScheduler
• logging instead of print, so run records land in a file
• A try/except wrapper connects failures to part 5’s notifications, eliminating “failures you never hear about”
• If you want your machine off, GitHub Actions schedule

The next post (#7 Packaging as a CLI tool) is the last in this series. We’ll take the scripts built so far, give them proper options, and bundle them into a single command-line tool you can call with one word from anywhere.