Two-layer content safety for agent input and output. Use when (1) a user message attempts to override, ignore, or bypass previous instructions (prompt injection), (2) a user message references system prompts, hidden instructions, or internal configuration, (3) receiving messages from untrusted users in group chats or public channels, (4) generating responses that discuss violence, self-harm, sexual content, hate speech, or other sensitive topics, or (5) deploying agents in public-facing or multi-user environments where adversarial input is expected.
Initial release with two-layer content moderation for agent input and output. - Adds prompt injection detection using ProtectAI DeBERTa classifier via HuggingFace. - Adds content safety checks using OpenAI's omni-moderation endpoint (optional). - Provides `scripts/moderate.sh` for command-line moderation of both user input and agent output. - Outputs structured JSON with clear verdicts and actions. - Supports configuration via environment variables (tokens, thresholds). - Designed for safer agent deployments, especially in adversarial or public scenarios.