Elastic high stable kql
OpenAI ChatGPT Enterprise Repeated Assistant Refusals
Detects three or more ChatGPT Enterprise assistant messages for the same user whose text matches common policy-refusal phrases (cannot assist, cannot help, cannot fulfill, unable to assist, against my guidelines, and similar) within the lookback window. A refusal burst can indicate jailbreak probing, policy-bypass attempts, or automated unsafe prompting intended to evade model safety controls.
Detection Logic
from logs-openai_chatgpt_enterprise.conversation_message-*
| where
data_stream.dataset == "openai_chatgpt_enterprise.conversation_message" and
openai_chatgpt_enterprise.conversation_message.message.author.type == "assistant" and
message is not null and
user.email is not null
| eval Esql.message_text = to_lower(message)
| where
Esql.message_text like "*i can't assist*" or
Esql.message_text like "*i can't help*" or
Esql.message_text like "*i cannot fulfill*" or
Esql.message_text like "*i cannot help*" or
Esql.message_text like "*i cannot provide*" or
Esql.message_text like "*i can't provide*" or
Esql.message_text like "*i will not provide*" or
Esql.message_text like "*i won't provide*" or
Esql.message_text like "*i cannot answer*" or
Esql.message_text like "*i can't answer*" or
Esql.message_text like "*unable to assist*" or
Esql.message_text like "*against my guidelines*"
| stats
Esql.event_count = count(*),
Esql.conversation_id_count_distinct = count_distinct(openai_chatgpt_enterprise.conversation_message.conversation.id),
Esql.message_values = values(message),
Esql.timestamp_first_seen = min(@timestamp),
Esql.timestamp_last_seen = max(@timestamp)
by
user.email,
organization.id
| where Esql.event_count >= 3
| keep
user.email,
organization.id,
Esql.event_count,
Esql.conversation_id_count_distinct,
Esql.message_values,
Esql.timestamp_first_seen,
Esql.timestamp_last_seen False Positives
- ⚠ Approved red-team or prompt-security evaluations that intentionally elicit refusals. Exclude those users or raise the count threshold.
- ⚠ Long helpful answers that incidentally include phrases such as "I can't help with X, but I can help with Y." Read the sampled messages before treating the alert as a hard refusal campaign.
Field Validations
Loading…
Comments (0)
Loading comments...