How to scrape job postings with Apify and create automated alerts

Looking for a job is a full-time job. You can waste hours refreshing listings, or you can automate the grunt work. This guide is for anyone who wants to scrape job postings from almost any site using Apify and get notified automatically when something new pops up. No, you don’t need to be a coding genius, but some basic scripting (or at least patience for copy-pasting) helps.

If you’re tired of wrestling with clunky job boards, missing out on fresh postings, or just want to build something useful, you’re in the right place. We’ll cut through the hype and show you what actually works, what’s a pain, and where you might hit a wall.

Step 1: Understand What You’re Scraping (and What’s Legal)

Before you fire up any scraping tool, a quick reality check:

  • Not all job boards want to be scraped. Some have legal terms against it, others actively block bots. Check the site’s robots.txt and terms of service.
  • If you’re scraping public company career pages, you’re usually fine, but don’t take my word for it—read the fine print.
  • Don't hammer a site with requests. Be respectful, go slow, and don’t ruin it for everyone else.

Pro Tip: Start with your target list—specific job boards, company career sites, or aggregators. The more focused you are, the easier this will be.

Step 2: Set Up Your Apify Account

Apify is a cloud-based scraping platform. It’s not the only option, but it’s one of the easiest if you don’t want to build everything from scratch.

  • Go to Apify and sign up for a free account. The free tier is enough for most experiments.
  • Get familiar with the dashboard. You’ll see “Actors”—these are ready-made (or custom) scrapers for different sites.
  • If you want to scrape a well-known site (like LinkedIn or Indeed), search for an existing Actor. If you want to scrape a custom site, you might need to build your own.

What Works: Apify’s templates and Actors can save you hours.
What Doesn’t: Don’t expect a one-click solution for every job board. Some need custom scripts.

Step 3: Find or Build a Scraper (“Actor”) For Your Target Site

Option A: Use an Existing Actor

  • In Apify’s store, type the name of the job board or site. For example, “LinkedIn Jobs” or “Indeed.”
  • Click into the Actor and read the docs. Check if it supports your target filters—location, keywords, remote, etc.
  • Run a test scrape with a small sample (e.g., 10 results). Make sure you get what you need: job title, company, URL, date posted, etc.
  • Save the Actor to your account.

Option B: Build a Custom Scraper

If your target isn’t covered, you’ll need to roll up your sleeves:

  • Use Apify’s “Web Scraper” Actor. You can point and click to define elements, but sometimes you’ll need CSS selectors or XPath.
  • Open the job site in your browser. Inspect the HTML (right-click → Inspect Element) to find the data you want.
  • In Apify’s scraper UI, tell it how to extract each field (job title, company, etc.).
  • Set up pagination if the site has multiple pages of results.
  • Test your scraper. Don’t be shocked if it breaks—the web is messy, and sites change all the time.

What Works: Point-and-click works for simple sites.
What Doesn’t: Dynamic sites (lots of JavaScript, infinite scroll, popups) are a pain. Sometimes, you’ll need to write a bit of JavaScript or use Apify’s Puppeteer integration.

Step 4: Schedule Your Scraper

You don’t want to do this manually every day. Automate it:

  • In your Apify dashboard, go to your Actor. Click “Schedule.”
  • Set how often you want the scraper to run (hourly, daily, weekdays, etc.).
  • Don’t overschedule—once or twice a day is usually enough.

Pro Tip: If you’re scraping a site with rate limits, don’t push your luck. Slower is safer.

Step 5: Store or Export the Results

You’ve got data. Now what? Apify lets you:

  • Export to CSV, JSON, or Excel. Easy for quick checks or importing into spreadsheets.
  • Save to Google Sheets using Apify’s integration (handy for sharing or connecting to other tools).
  • Pipe data to a database or webhook if you want to get fancy.

Honest Take: For most people, Google Sheets is enough. Don’t overengineer this unless you need to.

Step 6: Set Up Automated Alerts

You want to know when something new pops up, not just collect data. Here’s how:

Option A: Email Alerts via Google Sheets

  1. Connect Apify to Google Sheets so new rows get added each time the scraper runs.
  2. Use Google Sheets’ built-in notification rules (Tools → Notification Rules) to email you when changes happen.
  3. Or, write a simple Google Apps Script to send a custom email when a new row matches your criteria (e.g., specific keywords or locations).

Option B: Use Zapier (or Make/Integromat)

  • Connect Apify’s webhook to Zapier.
  • Set up a Zap: when new data arrives, filter for what you care about (e.g., “software engineer” in New York), then send an email, Slack message, or push notification.
  • This is more flexible, but you’ll need to fiddle with Zapier’s logic.

Option C: Custom Script (for the Brave)

  • Download the results as JSON or CSV.
  • Write a Python script that checks for new postings and emails you if there’s a match.
  • Schedule your script with cron (Linux/Mac) or Task Scheduler (Windows).

What Works: Google Sheets + notification rules is dead simple for personal use.
What Doesn’t: Zapier gets expensive if you go over free plan limits.

Step 7: Filter Out the Noise

If you scrape too broadly, you’ll drown in alerts. Set up filters:

  • Only track jobs with certain keywords, locations, or companies.
  • Use Google Sheets formulas (like =FILTER) or Zapier filters.
  • If building a custom scraper, add logic to skip irrelevant postings at the scraping stage.

Pro Tip: Start broad, then tighten your filters as you see what comes in.

Step 8: Monitor and Maintain

Scraping isn’t “set it and forget it.” Things will break:

  • Sites redesign their pages—your selectors might stop working.
  • Actors get updates or deprecation notices.
  • You might hit rate limits or get blocked.

Check your scraper every week or two. If you stop getting alerts, don’t assume there just aren’t any jobs—it probably broke.

What to Ignore (and Why)

  • “AI-powered job scrapers”: Most are just wrappers around basic scraping. Don’t pay extra for buzzwords.
  • Overly complex pipelines: Unless you’re running a recruiting firm, you don’t need Kafka, Docker, and a data lake for job alerts.
  • Scraping LinkedIn at scale: LinkedIn really doesn’t want you to scrape them. Expect to get blocked, and don’t build anything mission-critical on it.

Summary: Keep It Simple, Iterate Fast

You don’t need a PhD in data science to automate job search alerts. Start with one site, use Apify’s cloud platform to build or borrow a scraper, and connect it to Google Sheets or Zapier for notifications. Don’t obsess over making it perfect—the web changes constantly, and you’ll need to tweak things anyway.

Build something that works, see what you get, then improve as you go. The best automation is the one you actually use.