--- name: context-prompt-management description: "Manage token limits and optimize prompts for LLM workflows. Use when you need to: (1) Stay within model token limits automatically, (2) Accumulate context from multiple sources, (3) Generate prompts with intelligent summarization, or (4) Template prompts with dynamic content. NOT for: real-time chat, streaming contexts, or cases requiring perfect context preservation." --- # Context & Prompt Management Automatically manage token limits and optimize prompts for LLM workflows. Accumulate context from multiple sources, intelligently summarize when limits are exceeded, and generate prompts that fit within model constraints. ## Quick Start ```python from context_prompt_optimizer import ContextPromptOptimizer optimizer = ContextPromptOptimizer(token_limit=4096) optimizer.add_to_context("Research notes...") optimizer.add_to_context("Additional context...") prompt = optimizer.get_optimized_prompt("Write about: {context}") ``` ## Core Rules **Token Management:** Always set `token_limit` to match your target model's context window. The tool counts tokens and trims automatically. **Summarization:** When context exceeds `summary_threshold`, uses GitHub Models API for intelligent compression. Set `GITHUB_TOKEN` environment variable. **Templates:** Use `{context}` placeholder in prompt templates. Gets replaced with optimized context that fits token limits. ```python # Configuration options optimizer = ContextPromptOptimizer( token_limit=4096, # Max tokens in final prompt summary_threshold=3000, # When to trigger AI summarization trimming_strategy="summarize", # "summarize" or "truncate" verbose=True # Debug logging ) ``` ## Summarization vs Truncation **Summarize** (recommended): Uses AI to compress context while preserving key information. Requires GitHub Models API access. **Truncate**: Simple tail-trimming, keeps most recent additions. No API calls, faster but less intelligent. ```python # AI summarization (smart but slower) optimizer = ContextPromptOptimizer(trimming_strategy="summarize") # Simple truncation (fast but crude) optimizer = ContextPromptOptimizer(trimming_strategy="truncate") ``` ## Progressive Context Building Add context incrementally as you gather information: ```python optimizer = ContextPromptOptimizer(token_limit=6144) # Add sources progressively optimizer.add_to_context("API documentation excerpts...") optimizer.add_to_context("Community discussion points...") optimizer.add_to_context("User feedback themes...") # Generate final prompt result = optimizer.get_optimized_prompt(""" Analyze the following information: {context} Focus on: - Key technical insights - Community sentiment - Actionable recommendations """) ``` ## CLI Usage ```bash export GITHUB_TOKEN=$(gh auth token) ./main.py --add "Context to analyze..." --verbose ./main.py --show ``` **Note:** CLI context doesn't persist between calls. Use Python API for accumulating context over time. ## Error Handling ```python try: optimizer = ContextPromptOptimizer() result = optimizer.get_optimized_prompt(template) except EnvironmentError: # GITHUB_TOKEN not set, falls back to truncation optimizer = ContextPromptOptimizer(trimming_strategy="truncate") result = optimizer.get_optimized_prompt(template) ``` The tool gracefully falls back to truncation if GitHub Models API is unavailable. ## Common Patterns **Research Synthesis:** ```python optimizer.add_to_context(github_releases) optimizer.add_to_context(hn_discussions) optimizer.add_to_context(stackoverflow_trends) analysis = optimizer.get_optimized_prompt("Summarize key developments: {context}") ``` **Content Generation:** ```python optimizer.add_to_context(source_material) optimizer.add_to_context(style_guidelines) content = optimizer.get_optimized_prompt("Write article: {context}") ``` **Multi-source Analysis:** ```python for source in data_sources: optimizer.add_to_context(source.content) insight = optimizer.get_optimized_prompt("Extract insights from: {context}") ``` ## Limitations - Uses whitespace-based token counting (approximation, not exact tokenizer) - Summarization adds API latency vs simple truncation - Large contexts held in memory - not suitable for massive datasets - Requires GitHub Models API access for intelligent summarization ## Dependencies - Python 3.7+ - GitHub token with Models API access - No external packages (uses stdlib only) Set `GITHUB_TOKEN` environment variable or summarization falls back to truncation. ## Complete Implementation ```python #!/usr/bin/env python3 """ Context & Prompt Optimizer Manages and optimizes prompt context windows with trimming and summarization. Uses GitHub Models API for optional summarization. """ import os import sys import time import json import urllib.request from typing import List, Optional # --- CONFIGURATION --- TOKEN_LIMIT = 2048 # max tokens allowed in prompt SUMMARY_THRESHOLD = 1500 # tokens count to trigger summarization SUMMARY_MODEL = "gpt-4o-mini" # model for summarization (GitHub Models API) PROMPT_TEMPLATE = "Answer the following:\n{context}" TRIMMING_STRATEGY = "summarize" # options: "truncate", "summarize" VERBOSE = False # GitHub Models API endpoint and token GITHUB_API_URL = "https://models.inference.ai.azure.com/chat/completions" GITHUB_TOKEN = os.getenv("GITHUB_TOKEN") # must be set in env def log(*args, **kwargs): if VERBOSE: print("[ContextPromptOptimizer]", *args, **kwargs) def count_tokens(text: str) -> int: # Simple heuristic: count whitespace-separated tokens return len(text.split()) class ContextPromptOptimizer: def __init__(self, token_limit: int = TOKEN_LIMIT, summary_threshold: int = SUMMARY_THRESHOLD, summary_model: str = SUMMARY_MODEL, prompt_template: str = PROMPT_TEMPLATE, trimming_strategy: str = TRIMMING_STRATEGY, verbose: bool = VERBOSE): self.token_limit = token_limit self.summary_threshold = summary_threshold self.summary_model = summary_model self.prompt_template = prompt_template self.trimming_strategy = trimming_strategy self.verbose = verbose self.context_segments: List[str] = [] self.current_context = "" if not GITHUB_TOKEN: raise EnvironmentError("GITHUB_TOKEN environment variable not set") log(f"Initialized with token_limit={token_limit}, trimming_strategy={trimming_strategy}") def add_to_context(self, text: str): if not text or not text.strip(): log("Empty or whitespace-only text ignored") return self.context_segments.append(text.strip()) log(f"Added to context, segments count: {len(self.context_segments)}") def _build_context(self) -> str: return "\n".join(self.context_segments) def _call_github_model(self, prompt: str, max_tokens: int = 256) -> Optional[str]: """ Calls GitHub Models API to generate a completion (used for summarization). """ url = GITHUB_API_URL headers = { "Authorization": f"Bearer {GITHUB_TOKEN}", "Content-Type": "application/json", "User-Agent": "ContextPromptOptimizer/1.0" } data = { "model": self.summary_model, "messages": [ {"role": "user", "content": prompt} ], "max_tokens": max_tokens, "temperature": 0.3, "top_p": 1, } try: req = urllib.request.Request(url, data=json.dumps(data).encode(), headers=headers) with urllib.request.urlopen(req, timeout=15) as resp: if resp.status != 200: log(f"GitHub API error: HTTP {resp.status}") return None resp_json = json.load(resp) choices = resp_json.get("choices") if choices and len(choices) > 0: content = choices[0].get("message", {}).get("content", "").strip() log(f"GitHub model response received, length: {len(content)}") return content log("GitHub API response missing choices") return None except Exception as e: log(f"Exception during GitHub API call: {e}") return None def _summarize_context(self, context: str) -> str: prompt = f"Summarize the following text concisely:\n\n{context}\n\nSummary:" summary = self._call_github_model(prompt, max_tokens=256) if summary: log("Context summarized successfully") return summary else: log("Summarization failed, falling back to truncation") return self._truncate_context(context) def _truncate_context(self, context: str) -> str: tokens = context.split() truncated_tokens = tokens[-self.token_limit:] # keep last tokens within limit truncated = " ".join(truncated_tokens) log(f"Context truncated to last {len(truncated_tokens)} tokens") return truncated def get_optimized_prompt(self, template: Optional[str] = None) -> str: """ Returns a prompt string that fits within token_limit using the configured strategy. """ template = template or self.prompt_template context = self._build_context() token_count = count_tokens(context) log(f"Current context tokens: {token_count}") if token_count > self.token_limit: log(f"Context exceeds token limit ({self.token_limit})") if self.trimming_strategy == "summarize" and token_count > self.summary_threshold: context = self._summarize_context(context) else: context = self._truncate_context(context) final_prompt = template.replace("{context}", context) final_tokens = count_tokens(final_prompt) if final_tokens > self.token_limit: # As a last resort, truncate prompt forcibly log(f"Final prompt tokens {final_tokens} exceed limit, truncating forcibly") prompt_tokens = final_prompt.split() final_prompt = " ".join(prompt_tokens[:self.token_limit]) log(f"Final prompt tokens count: {count_tokens(final_prompt)}") return final_prompt def main(): import argparse parser = argparse.ArgumentParser(description="Context & Prompt Optimizer CLI") parser.add_argument("--add", type=str, help="Add text to context") parser.add_argument("--show", action="store_true", help="Show optimized prompt") parser.add_argument("--verbose", action="store_true", help="Enable verbose logging") args = parser.parse_args() global VERBOSE VERBOSE = args.verbose try: optimizer = ContextPromptOptimizer(verbose=VERBOSE) except EnvironmentError as e: print(f"Error: {e}", file=sys.stderr) sys.exit(1) if args.add: optimizer.add_to_context(args.add) print("Added text to context.") if args.show: prompt = optimizer.get_optimized_prompt() print("=== Optimized Prompt ===") print(prompt) if __name__ == "__main__": main() ``` ## Usage Save as `context_optimizer.py` and run: ```bash export GITHUB_TOKEN=$(gh auth token) python context_optimizer.py --add "Your context here" --show --verbose ```