Skip to main content

​
Use case

Regression probe for a bug where outbound PII redaction corrupted JSON request bodies. When Presidio classified a name like “Aarav Sharma” as PERSON, the span could grab a neighbouring JSON quote character. Substituting the redaction token into the raw body broke JSON syntax, causing upstream APIs (e.g. OpenAI) to return 400 “could not parse the JSON body”. After the fix, each JSON string value is scanned in isolation so entity spans cannot cross structural characters. The re-serialised body is guaranteed to stay valid JSON regardless of replacement length.

​
What you’ll learn

  • How outbound PII scanning interacts with JSON request bodies
  • Why per-value scanning is necessary (vs. scanning the whole body as one string)
  • Testing the fix end-to-end with a real OpenAI API call

​
Prerequisites

Also: 
export OPENAI_API_KEY="sk-..."

​
Code walkthrough

The security policy enables PII redaction with rehydration on, so the sandbox code receives a normal-looking response from the LLM: 
from declaw import (
    ALL_TRAFFIC,
    AuditConfig,
    NetworkPolicy,
    PIIConfig,
    Sandbox,
    SecurityPolicy,
)

POLICY = SecurityPolicy(
    pii=PIIConfig(
        enabled=True,
        types=["ssn", "credit_card", "email", "phone", "person_name"],
        action="redact",
        rehydrate_response=True,
    ),
    network=NetworkPolicy(
        allow_out=["api.openai.com", "pypi.org", "*.pythonhosted.org"],
        deny_out=[ALL_TRAFFIC],
    ),
    audit=AuditConfig(enabled=True),
)
The probe sends a chat completion request whose user message contains a name (Aarav Sharma) — the minimum trigger for the original bug. It uses stdlib urllib so it runs on the minimal sandbox template without extra dependencies: 
PROBE = """
import json, os, urllib.request, urllib.error

msg = ("Echo bot. Repeat verbatim between markers. "
       "<<<ssn=123-45-6789 name=Aarav Sharma>>>")
body = json.dumps({
    "model": "gpt-4.1",
    "messages": [{"role": "user", "content": msg}],
    "max_completion_tokens": 80,
}).encode()
req = urllib.request.Request(
    "https://api.openai.com/v1/chat/completions",
    data=body,
    method="POST",
    headers={
        "Authorization": "Bearer " + os.environ["OPENAI_API_KEY"],
        "Content-Type": "application/json",
    },
)
try:
    with urllib.request.urlopen(req, timeout=60) as resp:
        data = json.loads(resp.read())
        print("STATUS: 200")
        print("AGENT_READ_BACK:", data["choices"][0]["message"]["content"])
except urllib.error.HTTPError as e:
    print("STATUS: ERROR")
    print("ERROR_MSG:", e.read().decode("utf-8", "replace")[:500])
"""
Create the sandbox, inject the OpenAI key, and run: 
sbx = Sandbox.create(
    template="ai-agent",
    timeout=120,
    security=POLICY,
    envs={"OPENAI_API_KEY": os.environ["OPENAI_API_KEY"]},
)
try:
    sbx.files.write("/tmp/script.py", PROBE)
    r = sbx.commands.run("python3 /tmp/script.py", timeout=90)
    print(r.stdout)
finally:
    sbx.kill()

​
Expected output

STATUS: 200
AGENT_READ_BACK: <<<ssn=[REDACTED_SSN] name=[REDACTED_PERSON]>>>
VERDICT  : PASS
A STATUS: 200 means the JSON body arrived intact at OpenAI after redaction. If the body were corrupted, OpenAI would return a 400 and the probe would report FAIL.

​
Full source

See cookbook/examples/pii-json-body-safety/main.py in the repo.