ReferrerCop Changelog

Version 1.1.0 (02/26/2006)

The format of blacklists and whitelists has changed; regular expressions must now be enclosed in forward slashes (i.e., /regexp/). All other list entries will be treated as plain strings.
The standard blacklist is now the default, rather than the regexp-heavy optimized blacklist. We were getting a little too close to Ruby’s regexp memory limit with the optimized list.
Performance with large lists has been greatly improved; ReferrerCop is now capable of handling blacklists and whitelists of 100,000 entries or more without problems.
The blacklist update feature has been revised to use SHA1 checksums rather than timestamps to determine whether the remote blacklist and the local blacklist are out of sync.
Many common options can now be specified in a configuration file.

Added -U mode to check for an updated default blacklist and download it if available.
Fixed a bug that caused an extra newline to be inserted into AWStats data files when no referrer URLs were present.

Added -i mode for in-place filtering of files.
Minor performance optimizations.
Improved parsing of AWStats data files.
Fixed a bug that would occasionally result in an extra newline being inserted into filtered AWStats data files.

Over 300% faster than previous versions due to a new heavily optimized blacklist and various other improvements.
Added -n and -s modes to extract ham and spam URLs, respectively.
Removed -B and -W modes (the optimized blacklist makes them useless).
Renamed -v option to -V.
Renamed -d option to -v (for “verbose”).
Improved error handling.
Refactored and greatly improved module API (for developers who want to use ReferrerCop’s functionality in their own Ruby applications).

Filter all files passed on the command line (wildcards supported), or standard input if no files are specified.
If input isn’t an Apache combined log or AWStats data file, assume that it’s a list of URLs and attempt to filter them.
Added -d option to display debugging output and statistics.
Added -u mode to display the ham/spam status of a single URL passed on the command line.
Added -B and -W modes to provide respective blacklist and whitelist pattern counts.
Don’t panic on malformed/corrupted lines in Apache logs.
Display less cryptic error messages when invalid command-line options are entered.