Choose what goes into each model call, under a token budget.

How it decides

1. Every item gets a token count: yours, or an estimate (the larger of its word count and its character count divided by four, rounded up). Plug in your model's tokenizer through the TokenCounter trait for exact counts.
2. Room for the answer and for tool calls is reserved first (reserved_completion, reserved_tools, safety_margin).
3. Pinned items (priority 0) are always kept.
4. The rest are taken in policy order, each one kept if it still fits and skipped if it does not: by priority, newest first, or in input order.
5. You get back the kept items, the dropped items, tokens used and tokens left.

Install

[dependencies]
harpe = "=0.1.0-alpha.4"

pip install "harpe==0.1.0a4"

Version 0.1.0-alpha.4 is on crates.io and PyPI.

use harpe::{ContextBudgetEngine, ContextItem, ContextSynthesizer, PrunePolicy, TokenBudget};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut items = vec![
        ContextItem::new("policy", "Never deploy on Fridays.", 0), // priority 0: always kept
        ContextItem::new("decisions", "We chose blue-green deploys in March.", 1),
        ContextItem::new("meeting", "Long notes from the weekly sync, mostly unrelated.", 5),
    ];
    items[2].token_count = 900; // or leave 0 and Harpe estimates it
    let mut budget = TokenBudget::new(1_000);
    budget.reserved_completion = 200; // room for the model's answer
    let report = ContextBudgetEngine::default().synthesize(&items, &budget, PrunePolicy::LowestPriority)?;
    for it in &report.included_items { println!("keep  {}", it.id); }
    for it in &report.pruned_items { println!("drop  {}", it.id); }
    println!("{} tokens used, {} left", report.consumed_tokens, report.budget_remaining);
    Ok(())
}

from harpe import ContextBudgetEngine, ContextItem, PrunePolicy, TokenBudget

items = [
    ContextItem(id="policy", content="Never deploy on Fridays.", priority=0, pinned=True),
    ContextItem(id="decisions", content="We chose blue-green deploys in March.", priority=1),
    ContextItem(id="meeting", content="Long notes from the weekly sync.", priority=5, token_count=900),
]
budget = TokenBudget(total_budget=1000, reserved_completion=200)  # room for the answer
report = ContextBudgetEngine().synthesize(items, budget, PrunePolicy.LOWEST_PRIORITY)
print("keep:", [i.id for i in report.included_items], "drop:", [i.id for i in report.pruned_items])
print(report.consumed_tokens, "tokens used")

Measurements

How these numbers were measured. Each figure is the median of 5 runs of the telarch-benchmarks programs at release tag v0.1.0-alpha.4, with the range of all runs beside it. 13th Gen Intel Core i5-13500, 20 logical CPUs, Linux 6.18, rustc 1.97.1, pinned to the performance-core threads (CPUs 0-11). Shared server, not isolated; each run started only after the 1-minute load average was below 4 on two checks 20 s apart. Synthetic data; real workloads will differ. Raw output of every run.

Limits, stated plainly

• Greedy, not optimal. Harpe takes items in order and keeps each one that fits. A different combination could sometimes fill the budget more fully.
• Pinned items can exceed the budget. If the pinned items alone are larger than the budget, Harpe keeps them anyway and reports zero tokens left. Check consumed_tokens if you need a hard cap.
• Estimates are estimates. Without your own counter, token counts approximate the model's tokenizer.
• No relevance scoring yet. In 0.1.0-alpha.4, the LeastRelevant policy behaves exactly like TruncateTail (input order), and TruncateTail drops whole items; it does not shorten text.
• Deterministic. No randomness, no clock, no network: the same items and budget always give the same report.