AI Pentesting, Part 5: LLM-Assisted Filter Testing: Find Parser Disagreements Safely

1. Map the decoding chain

Trace CDN, reverse proxy, framework, middleware, and application. Identify URL decoding, Unicode normalisation, case folding, duplicate-key handling, content-type parsing, path normalisation, and limits at each stage. A filter may inspect raw bytes while the application interprets decoded text. A second decode can change meaning after the check.

Reframe LLM-assisted filter bypass as differential testing. Ask which harmless inputs adjacent components interpret differently. Use neutral LAB MARKER values rather than evasive exploit strings. Compare parsed values, not just status codes. Disagreements become security defects when they cross an authorisation or interpretation boundary.

2. Write a local differential harness

This standard-library example exposes two differences: form parsing treats plus as space while a percent decoder does not, and a second decode changes a literal percent sequence. There are no network requests or exploit payloads.

from urllib.parse import parse_qs, unquote

for raw in ["LAB+MARKER", "LAB%20MARKER", "LAB%2520MARKER"]:
    generic = unquote(raw)
    form = parse_qs("label=" + raw, strict_parsing=True)["label"][0]
    twice = unquote(form)
    print({"raw": raw, "generic": generic, "form": form, "twice": twice})
assert unquote("LAB+MARKER") == "LAB+MARKER"
assert parse_qs("label=LAB+MARKER")["label"] == ["LAB MARKER"]
assert unquote(unquote("LAB%2520MARKER")) == "LAB MARKER"

Expected: plus differs between generic and form decoding; double encoding changes after a second decode. Neither alone proves a vulnerability. Connect the difference to a policy, such as a field that must equal one permitted operation name.

Run the corpus through the real lab stack. Capture edge request bytes and application parsed values. Do not log secrets or production bodies while investigating normalisation.

3. Reject ambiguity in policy-bearing fields

One layer may select the first duplicate parameter, another the last, and another an array. A validator approving read while business logic selects write has no coherent policy. Reject multiple values for scalar fields before authorisation.

from urllib.parse import parse_qs

def parse_mode(query):
    parsed = parse_qs(query, keep_blank_values=True, max_num_fields=20)
    if set(parsed) != {"mode"}:
        raise ValueError("unexpected fields")
    modes = parsed.get("mode", [])
    if len(modes) != 1 or modes[0] not in {"read", "preview"}:
        raise ValueError("ambiguous or unsupported mode")
    return modes[0]

assert parse_mode("mode=read") == "read"
for invalid in ["mode=read&mode=preview", "mode=", "mode=write"]:
    try:
        parse_mode(invalid)
    except ValueError:
        continue
    raise AssertionError(invalid)

This fixture still needs production byte limits, invalid-percent handling, content-type policy, and consistent errors. Pass the parsed mode forward; do not reparse unchecked query bytes after validation.

4. Treat Unicode and paths as domain-specific

NFC makes canonically equivalent labels comparable but does not solve visual confusables. NFKC changes compatibility characters and may be inappropriate for display names or passwords. Define policy per field, preserve stable internal IDs, and review display confusion separately from authentication.

Filesystem paths, URLs, Windows paths, and object-store keys have different semantics. Use appropriate platform APIs instead of string replacement. Enforce containment after resolution and consider symlinks or reparse points. Prefix comparisons are insufficient; checking before a race-prone operation can create a time-of-check/time-of-use defect.

For outbound URLs, a hostname allowlist is only part of SSRF protection. Constrain schemes and ports, block redirects or validate each hop, bind policy to the actual connection destination, and restrict egress. DNS can change between validation and use. Test local stubs, not cloud metadata endpoints or unrelated services.

5. Generate a test plan, not a bypass list

Compare the supplied edge-parser and application-parser contracts.
Propose benign differential test cases only.
For each: raw form, edge value, app value, expected decision,
          invariant under test, and unknown behaviour.
Use neutral markers; no executable markup or commands.
Do not send requests or modify a target.

Keep cases in a reviewed corpus. Models hallucinate framework parsing behaviour, and libraries may silently rewrite malformed requests. Preserve transmitted bytes. Where a client cannot express a case, mark it untested rather than claiming coverage.

Useful cases include empty versus missing fields, JSON strings versus booleans, scalar versus array values, unknown enum members, and boundary-length labels. Use small explicit thresholds in disposable environments rather than massive bodies or uncontrolled nesting.

6. Fix the contract and retain regressions

A WAF rule can provide temporary containment, but the root fix belongs at the interpretation boundary. Prefer strict schemas, typed values, one canonical parse, parameterised queries, context-aware output handling, and authorisation after parsing. More string substitutions can create a representation the next stage interprets differently.

Retest direct application access and the complete deployment path. Include positive controls for legitimate international text and valid encoded spaces so the patch does not simply reject all non-ASCII input. Record exact proxy/framework versions, configuration hashes, raw requests, parsed values, and assertions. A passing model explanation is not a regression test.

Completion criterion: each policy-bearing value has one representation, ambiguous input is rejected consistently, and downstream code cannot reinterpret unchecked bytes. Part 6 uses similarly strict evidence boundaries for CVE patch decisions.

7. Step-by-Step Lab: Compare Two Parsers Using Neutral Inputs

Prerequisites: An offline JavaScript runtime and the documented parser contracts for an owned lab stack. No WAF, public endpoint, or bypass payload is required. Use only neutral label and mode values; the experiment compares representations rather than attempting exploitation.

1. Create a three-column case sheet: raw representation, edge interpretation, and application interpretation. State which layer owns URL/form decoding and which fields must be scalar.
2. Run the example below. URLSearchParams applies form-style plus handling; decodeURIComponent alone does not. Record those differences instead of declaring either API universally wrong.
3. Add one duplicate mode field. Decide that the application contract rejects duplicates before authorisation. Compare getAll with a single-value accessor so the ambiguity is visible.
4. Add a double-encoded neutral marker. Check whether any downstream component performs a second decode; do not assume it does because a model recognises the pattern.
5. If you have the actual owned proxy/framework lab, replay the same small corpus through it and capture raw bytes plus parsed values. Use the framework's test client where possible rather than a hand-written HTTP parser.
6. Fix interpretation at the owning boundary: one parse, schema validation, typed values, and no unchecked downstream reparsing. Keep legitimate international text and valid encoded spaces as positive controls.
7. Ask AI to explain only the observed differences and generate a regression table. A proposed but unexecuted case must be marked not tested, not passed.

const raw = "LAB+MARKER";
const generic = decodeURIComponent(raw);
const form = new URLSearchParams("label=" + raw).get("label");
const duplicate = new URLSearchParams("mode=read&mode=preview").getAll("mode");
const once = decodeURIComponent("LAB%2520MARKER");
const twice = decodeURIComponent(once);
if (generic !== "LAB+MARKER" || form !== "LAB MARKER" || duplicate.length !== 2) throw Error("parser mismatch");
console.log(JSON.stringify({ generic, form, duplicate, once, twice }));

Expected result: The generic value retains plus, the form value contains a space, duplicate has two values, once contains a literal percent sequence, and twice contains a space. These observations establish representation differences only. Security impact requires a policy-bearing value to be validated and consumed inconsistently.

8. Worked Examples and Expected Outcomes

Example A: scalar versus array

A validator expects one mode while the framework supplies two. Reject the ambiguity rather than selecting whichever value seems safest. Test that the business layer consumes the exact validated scalar and cannot re-read the raw query.

Example B: Unicode labels

Two canonically equivalent display labels compare differently before the chosen normalisation. Define the field's policy, preserve stable internal IDs, and check positive controls. Do not apply a global compatibility transformation to passwords or names without analysing its consequences.

Example C: content-type confusion

The edge inspects a request as form data while the application accepts JSON under the same media type. Use a harmless lab record to inspect parser selection. Reject unsupported media types consistently; do not add another string filter and assume the contracts now agree.

{"case":"PARSE-021","invariant":"one mode value after one canonical parse","ambiguous_input":"reject","legitimate_text":"retain","downstream_reparse":"forbidden"}

9. Troubleshooting, Retesting, and Completion Evidence

Retest and completion: Keep the parser versions, edge configuration, corpus, actual values, expected decisions, and regression results. Include a clean control proving valid data still works. A WAF-only workaround may be temporary containment; record the remaining root-cause work explicitly.

10. Real Tool Workflow: Compare Real Parser Behaviour with Python and curl

Prerequisites: Python 3 and an owned local parser-echo application if HTTP replay is used. The optional server must be deliberately configured to echo harmless parsed fields on loopback; no such persistent service is installed by this article. The Python-only comparison runs entirely offline.

Use reviewed releases and record the versions actually installed. Command references for this workflow were reviewed on 29 October 2026; product support and flags can change. Instructions and output shapes are not claims of a live assessment.

Step-by-Step Tool Operation

1. Create a case file with the exact edge and application parser contract. Specify whether the label field uses form encoding and whether mode is a single enum. Use only LAB markers.
2. Put the standard-library Python example below in a local script and run it without network access. Retain the output for plus, percent-space, double-encoded space, and duplicate fields.
3. Ask AI to explain the observed values and identify the owning parse step. Reject any suggested exploit payload or unsupported assertion about the real framework.
4. If an owned parser-echo service is available at the stated loopback origin, use the curl request below once. It submits one neutral encoded label and mode with no redirect following.
5. Compare a capture at the edge with the application's parsed value. A local curl trace contains whatever you send, so never use live session cookies or secret form fields in this exercise.
6. Change the lab contract to reject ambiguous scalar inputs and remove downstream reparsing. Retest valid spaces, duplicate modes, missing fields, and legitimate international text. Keep raw and canonical values in separate evidence fields.

from urllib.parse import parse_qs, unquote

for raw in ["LAB+MARKER", "LAB%20MARKER", "LAB%2520MARKER"]:
    parsed = parse_qs("label=" + raw, strict_parsing=True)["label"][0]
    print({"raw": raw, "generic": unquote(raw), "form": parsed,
           "second_decode": unquote(parsed)})
values = parse_qs("mode=read&mode=preview")["mode"]
assert len(values) == 2
print({"mode_values": values, "scalar_policy": "reject duplicate"})

python --version
python .\parser-lab.py
curl.exe --version
curl.exe --max-time 5 --max-filesize 8192 --trace-ascii .\parser-request.trace --data-raw 'label=LAB%2520MARKER&mode=read' -H 'Content-Type: application/x-www-form-urlencoded' http://127.0.0.1:8080/parse

Expected Evidence and Interpretation

The offline output should distinguish generic decoding from form-style plus handling and expose two mode values. The optional HTTP result depends on the owner-supplied echo application; its absence is a failed prerequisite, not a negative test. The doubled encoding only demonstrates a changed representation if a second decoder is actually observed.

The trace file supports a byte-level comparison for this harmless request. It does not prove every transport preserved identical bytes, and curl's output is not the application's parsed state. Instrument both boundaries before making a security claim.

Troubleshooting and Cleanup

On systems where python is only a store alias, install an approved interpreter or use the known environment's documented launcher; do not treat a failed command as a passing case. If the framework rejects the media type, record that decision instead of silently switching parsers. Remove trace files after approved retention, particularly if the method is later reused with authenticated requests.