VANTAGE
Deep DiveFreeTelecommunications & Media

Rate Limiting Public Feeds Without Losing Partners

A pragmatic 429 policy for open tiers that keeps legitimate aggregators online.

Tomas Ferreira ·

Principle\n\nOpen tiers exist to be scraped. The goal is shaping, not blocking.\n\n## Policy\n\n- Soft cap per consumer per minute\n- 429 with Retry-After, never a silent drop\n- Never rate limit a conditional request that returns 304\n\nPartners who see a clean Retry-After header almost always back off correctly.