Token and cost estimator script for prompts and batch jobs

Before running a prompt over 50,000 records, or before approving a feature that sends a long document with every request, you want a rough number: how many tokens, and roughly what it costs. This script gives you that in one command, and states clearly that it is an estimate.

**How it estimates**

- Default: characters divided by a ratio (4.0 unless you pass `--chars-per-token`). This is a common rule of thumb for English prose. It is less accurate for code, non-English text and heavy whitespace. - Optional: `--tiktoken o200k_base` (or `cl100k_base`) counts with OpenAI's tokenizer library if you have it installed. For models from other vendors this is still an approximation, because each model family has its own tokenizer.

**Cost projection**

No prices are built in, because they change and differ by model and plan. You pass your current input and output prices in USD per million tokens, the expected output length, and the number of requests:

``` $ python3 scripts/estimate_tokens.py prompts/summarize.md sample-doc.txt \ --output-tokens 400 --requests 50000 --input-price <input $/MTok> --output-price <output $/MTok>

file chars ~tokens prompts/summarize.md 1840 460 sample-doc.txt 22410 5603

method: chars/4 (approximate) per request: ~6063 input + 400 output tokens for 50000 request(s): ~303150000 input + 20000000 output tokens estimated cost: ~$... ```

Each file counts as part of one request's input, so pass the prompt template plus a representative input. `--json` prints a machine-readable summary for spreadsheets or CI.

**What it will not do**

- Give billing-accurate numbers. For exact counts, use your provider's token counting endpoint or the `usage` field returned with each response. - Account for prompt caching discounts, batch discounts, images, tool definitions or system overhead tokens. Adjust the prices you pass in if those apply.

Exit codes: 0 success, 2 usage error or `--tiktoken` requested without the package installed.