URL Watch

urlwatch is an open-source command-line tool for monitoring web pages and notifying users when their content changes. Unlike browser-based monitoring services, urlwatch runs locally or on a server, giving investigators and security professionals complete control over monitoring logic, notification methods, and data handling — with no third-party service having access to monitored URLs or results.

The tool is written in Python and supports flexible configuration for monitoring any number of URLs at user-defined intervals. Each monitored URL can have custom filters applied — stripping HTML tags to monitor only text content, applying regular expressions to focus on specific content sections, or diffing only specific JSON fields in API responses. This granular filtering prevents alert fatigue from irrelevant changes (like ad content or timestamps) while ensuring meaningful content changes trigger notifications.

Notification channels supported by urlwatch include email, Slack, Telegram, XMPP, and many others through plugin extensions. For security teams maintaining operational security (OPSEC), running urlwatch on an isolated server with custom notification routing is preferable to using commercial monitoring services that might log monitored URLs.

For OSINT investigators, urlwatch enables monitoring of dark web pages (via Tor proxy integration), private network resources accessible from a monitoring server, and sites that block known monitoring service IP ranges. Running urlwatch from a customized server allows monitoring targets that block commercial services.

Advanced users can write custom Python scripts triggered by changes — enabling automated responses like capturing a full page archive, running a downstream analysis script, or updating an investigation database when a monitored page changes. This programmability makes urlwatch a foundational building block for automated web intelligence workflows.

Installation via pip: run "pip install urlwatch". Configuration uses YAML files for URL definitions and notification settings. Full documentation is available on GitHub.

Use urlwatch in a dedicated monitoring environment and document all monitored URLs with investigation context for audit purposes.