A rate limit caps how often you can ask a system for something: requests per minute, tokens per minute, or daily usage. It can feel like the system is broken. Usually it’s protecting itself, since APIs, AI models, and web services can only handle so much at once. An agent retrying a failed command too quickly can hit the cap, and the service returns an error or tells it to wait.
A deli counter with a ticket machine makes this concrete. You take a number and wait your turn. If two hundred people rush a counter that serves sixty an hour, the deli doesn’t magically get faster. It makes people wait, or stops taking new orders for a bit.
Why you care
Real systems have traffic rules, and ignoring them makes your workflow flaky right when you need it dependable. A queue helps: line up the work and process it at the pace the API allows. Batch where it makes sense, retry politely, and spread work out. A token limit counts how much text a model can process; a rate limit counts how fast you send work. Good agent work knows when to wait its turn.