Elastic high stable kql

GCP Vertex AI Repeated Assistant Refusals

Detects three or more Vertex AI GenerateContent replies in 60 minutes whose assistant text indicates a refusal (cannot fulfill/assist/help/answer, unable to assist, decline the request, against my guidelines, and similar), aggregated by project and model. Prompt-response logs lack a stable caller identity, so this is a project-wide refusal burst signal — pivot to auditlogs for principal/IP attribution.

View Source

Detection Logic

from logs-gcp_vertexai.prompt_response_logs-*
| where
    data_stream.dataset == "gcp_vertexai.prompt_response_logs" and
    MV_CONTAINS(gcp.vertexai.prompt_response_logs.full_response.candidates.finish_reason, "STOP")
| eval response_text = to_lower(coalesce(mv_concat(gcp.vertexai.prompt_response_logs.full_response.candidates.content.parts.text, " "), ""))
| where
    response_text like "*cannot fulfill*" or
    response_text like "*cannot assist*" or
    response_text like "*can't assist*" or
    response_text like "*can't help*" or
    response_text like "*cannot help*" or
    response_text like "*i cannot provide*" or
    response_text like "*i can't provide*" or
    response_text like "*i will not provide*" or
    response_text like "*i cannot answer*" or
    response_text like "*i can't answer*" or
    response_text like "*unable to provide*" or
    response_text like "*unable to assist*" or
    response_text like "*unable to fulfill*" or
    response_text like "*not able to help*" or
    response_text like "*not able to provide*" or
    response_text like "*must decline*" or
    response_text like "*respectfully decline*" or
    response_text like "*decline this request*" or
    response_text like "*decline that request*" or
    response_text like "*cannot disclose*" or
    response_text like "*can't disclose*" or
    response_text like "*against my guidelines*" or
    response_text like "*that violates my guidelines*"
| stats
    Esql.event_count = count(*),
    Esql.timestamp_first_seen = min(@timestamp),
    Esql.timestamp_last_seen = max(@timestamp),
    Esql.prompt_values = values(gcp.vertexai.prompt_response_logs.full_request.contents.parts.text),
    Esql.response_values = values(gcp.vertexai.prompt_response_logs.full_response.candidates.content.parts.text)
  by
    gcp.vertexai.prompt_response_logs.model,
    cloud.project.id
| where Esql.event_count >= 3
| keep
    gcp.vertexai.prompt_response_logs.model,
    cloud.project.id,
    Esql.event_count,
    Esql.timestamp_first_seen,
    Esql.timestamp_last_seen,
    Esql.prompt_values,
    Esql.response_values

False Positives

  • ⚠ Approved red-team suites that intentionally elicit refusals. Exclude those models or raise the threshold.

Field Validations

Loading…

Comments (0)

Loading comments...