Appeak logo

How to Add Rate Limiting and Cost Caps to LLM-Powered Features in Laravel

Talk To an Expert

Table of Contents

1. Introduction

Adding an AI chat box or a summarizer to a Laravel app takes an afternoon. Keeping it affordable once real people use it takes more planning. Large language model APIs bill by the token, so a single long prompt, a retry loop, or one over-enthusiastic user can burn through a monthly budget in a few hours.

Traditional API protection only counts requests. With LLMs, the size of each request matters just as much as the number of requests. This guide walks through a practical, layered setup for rate limiting in Laravel and cost caps for LLM features, using tools the framework already ships with. At Appeak Technology, we apply the same layered approach when we build AI features for client products.

2. Rate Limits and Cost Caps Solve Different Problems

It helps to separate the two ideas before writing any code.

A rate limit controls speed. It answers the question: how many requests can this user or IP send in a minute or a day? It protects you from bursts, bots, and accidental loops.

A cost cap controls spend. It answers the question: how much money are we willing to let this user, or the whole app, consume today? It protects your budget even when every request is technically within the rate limit.

You need both. A user sending only five requests a minute can still cost you a lot if each request carries a huge document. Think of the rate limiter as a turnstile and the cost cap as a spending account.

3. Step 1: Add Named Rate Limiters

				
					Laravel's built-in rate limiter works well for this. Define a named limiter in your AppServiceProvider and give different tiers different limits:
use Illuminate\Cache\RateLimiting\Limit;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\RateLimiter;

public function boot(): void
{
    RateLimiter::for('ai', function (Request $request) {
        $user = $request->user();

        if (! $user) {
            return Limit::perMinute(3)->by($request->ip());
        }

        return [
            Limit::perMinute($user->isPaid() ? 20 : 5)->by('ai-min:'.$user->id),
            Limit::perDay($user->isPaid() ? 500 : 30)->by('ai-day:'.$user->id),
        ];
    });
}
Then attach it to the route that talks to the model:
Route::post('/assistant', AssistantController::class)
    ->middleware(['auth', 'throttle:ai']);

Returning an array lets you stack a short window and a long window together. When a limit is hit, Laravel responds with a 429 status and a Retry-After header, so your front end can show a friendly countdown instead of a vague error.

One production detail matters here: use a shared cache driver such as Redis. If each server keeps its own file-based counters, a user can multiply their allowance by the number of servers behind your load balancer.

4. Step 2: Track Spend in Money, Not Just Requests

Request counts are blunt. To cap cost, record what each call actually used. Most LLM providers return input and output token counts with every response, which gives you everything you need.

A small service class can convert those tokens into spend and keep daily totals. To avoid floating-point rounding problems, store money as whole micro-dollars (one dollar equals 1,000,000):

				
					<?php

namespace App\Services;

use Illuminate\Support\Facades\Cache;

class AiBudget
{
    // Example prices in micro-dollars per 1K tokens.
    // Replace with your provider's current rates.
    private const INPUT_PER_1K = 3000;
    private const OUTPUT_PER_1K = 15000;

    public function __construct(
        private int $userDailyLimit = 500_000,      // $0.50 per user
        private int $globalDailyLimit = 20_000_000, // $20 for the whole app
    ) {}

    public function allows(int $userId): bool
    {
        return $this->spent($this->userKey($userId)) < $this->userDailyLimit
            && $this->spent($this->globalKey()) < $this->globalDailyLimit;
    }

    public function record(int $userId, int $inputTokens, int $outputTokens): void
    {
        $cost = intdiv($inputTokens * self::INPUT_PER_1K, 1000)
              + intdiv($outputTokens * self::OUTPUT_PER_1K, 1000);

        $this->add($this->userKey($userId), $cost);
        $this->add($this->globalKey(), $cost);
    }

    private function add(string $key, int $amount): void
    {
        Cache::add($key, 0, now()->endOfDay());
        Cache::increment($key, $amount);
    }

    private function spent(string $key): int
    {
        return (int) Cache::get($key, 0);
    }

    private function userKey(int $id): string
    {
        return 'ai-spend:user:'.$id.':'.now()->toDateString();
    }

    private function globalKey(): string
    {
        return 'ai-spend:global:'.now()->toDateString();
    }
}

The Cache::add call creates the counter only if it does not exist yet and sets it to expire at midnight, so budgets reset daily without a scheduled job. Cache::increment is atomic on Redis, which keeps totals accurate when many requests arrive at once.

5. Step 3: Check the Budget Before You Call the Model

Now wire the budget into your controller. Check first, call second, record third:

				
					public function __invoke(AssistantRequest $request, AiBudget $budget)
{
    $userId = $request->user()->id;

    if (! $budget->allows($userId)) {
        return response()->json([
            'message' => 'Daily AI usage limit reached. Please try again tomorrow.',
        ], 429);
    }

    // $this->llm is your own thin wrapper around the provider SDK.
    $result = $this->llm->chat(
        $request->validated()['message'],
        maxTokens: 500
    );

    $budget->record($userId, $result->inputTokens, $result->outputTokens);

    return response()->json(['reply' => $result->text]);
}
The same global counter doubles as a kill switch. When the app-wide ceiling is reached, every user is blocked until the next day, and your invoice stays predictable.

6. Step 4: Cap the Size of Each Request

A cost cap works best when individual requests stay small. Three habits keep them that way:
  • Validate input length. In your form request, use a rule such as 'message' => ['required', 'string', 'max:2000'] so nobody can paste an entire book into the prompt.
  • Set a maximum output size. Always pass the provider’s maximum-token parameter. Without it, the model decides how long to talk, and you pay for every word.
  • Trim conversation history. Sending the full chat on every turn makes costs grow with each message. Keep the last few turns, or summarize older ones.
These limits also make your cost estimates more reliable, because the worst case for any single call becomes a number you know.

7. Step 5: Protect Queued AI Jobs

Many teams move slow LLM calls into queues, which is a good idea, but queue workers can drain a budget faster than web requests because they run continuously. Laravel’s job rate limiting middleware handles this: 
				
					use Illuminate\Queue\Middleware\RateLimited;
use Illuminate\Support\Facades\RateLimiter;

// In AppServiceProvider::boot()
RateLimiter::for('ai-jobs', fn (object $job) => Limit::perMinute(30));

// In the job class
public function middleware(): array
{
    return [new RateLimited('ai-jobs')];
}
Jobs that exceed the limit are released back onto the queue and retried later. Remember to set a sensible $tries or retryUntil value so a throttled job does not retry forever. Run the same budget check inside the job before it calls the model, because a job may sit in the queue long enough for the daily cap to be reached in the meantime.

8. Step 6: Fail Gracefully and Keep an Eye on Spend

Limits only help if users understand them and you can see them working.

  • Show a clear message. Tell users what happened and when they can try
    again. The Retry-After header makes this easy.
  • Add a feature flag. A simple config('ai.enabled') check
    lets you switch AI features off instantly during an incident.
  • Alert before the ceiling. Run a scheduled command that compares the
    global counter with the limit and sends a Slack or email alert at
    around 80 percent.
  • Log every call. Store user ID, feature name, token counts, and
    estimated cost. This data tells you which feature is expensive and
    which users need a higher tier.

If you want to understand why prototypes often fail once real traffic arrives, our article on why Laravel AI demos break in production covers the wider picture.

9. Common Mistakes to Avoid

  • Counting requests only and ignoring token size.
  • Using a local cache driver on multi-server setups.
  • Forgetting to cap output length.
  • Letting queued jobs bypass the budget check.
  • Hard-coding prices instead of keeping them in config, which makes updates painful when providers change their rates.
  • Having no global ceiling, so one bug can spend the whole month in a night.

10. Conclusion

Safe LLM features in Laravel come down to layers. Rate limiters control how fast people can call the model, token-based budgets control how much they can spend, request size limits keep each call predictable, and queue middleware plus monitoring protect the background work. None of it needs a new package; Laravel’s limiter, cache, and middleware cover the essentials.

If you are planning an AI feature and want it designed with cost control from day one, the team at Appeak Technology can help you build it so it scales without surprises.

11. Frequently Asked Questions

Rate limiting restricts how many requests can be made in a time window. A cost cap restricts how much money those requests may consume. Rate limits stop bursts and abuse, while cost caps keep your overall bill under control.
Yes. Define a named limiter with RateLimiter::for and apply it with throttle:name. It handles per-user, per-IP, and multi-window limits well, but it only counts requests, so pair it with token-based budget tracking.
Most LLM providers return input and output token counts in each API response. Read those values and record them after every call instead of estimating from text length.
Use Redis in production. It is shared across servers and supports atomic increments, which keeps counters accurate under heavy concurrency.

Return a 429 response with a clear message and a retry time. Optionally offer an upgrade path, and never let the request silently fail or reach the model anyway.

Recent Posts

Categories