Elastic high stable kql
Microsoft Foundry Repeated Assistant Refusals
Detects three or more completed assistant refusals from the same client, API Management subscription, and model. Matching replies finished on their own (finish_reason stop) and declined the prompt. A burst means that caller kept sending prompts the model will not answer.
Detection Logic
from logs-azure_ai_foundry.logs-*
| eval Esql.reply_text = mv_concat(azure.ai_foundry.properties.backend_response_body.choices.message.content, " ")
| where
data_stream.dataset == "azure_ai_foundry.logs" and
azure.ai_foundry.category == "GatewayLogs" and
azure.ai_foundry.properties.backend_response_body.choices.finish_reason in ("stop", "content_filter") and
(
Esql.reply_text like "*I can't assist*" or
Esql.reply_text like "*I can't help*" or
Esql.reply_text like "*I cannot fulfill*" or
Esql.reply_text like "*I cannot help*" or
Esql.reply_text like "*I cannot provide*" or
Esql.reply_text like "*I can't provide*" or
Esql.reply_text like "*I will not provide*" or
Esql.reply_text like "*I cannot answer*" or
Esql.reply_text like "*I can't answer*"
)
| stats
Esql.event_count = count(*),
Esql.timestamp_first_seen = min(@timestamp),
Esql.timestamp_last_seen = max(@timestamp),
Esql.azure_ai_foundry_properties_backend_request_body_messages_content_values = values(azure.ai_foundry.properties.backend_request_body.messages.content),
Esql.azure_ai_foundry_properties_backend_response_body_choices_message_content_values = values(azure.ai_foundry.properties.backend_response_body.choices.message.content)
by
source.ip,
azure.ai_foundry.properties.user_agent,
azure.ai_foundry.properties.apim_subscription_id,
azure.ai_foundry.properties.api_id,
azure.ai_foundry.properties.operation_id,
azure.ai_foundry.properties.backend_response_body.model,
azure.ai_foundry.service_name,
azure.resource.group,
url.domain,
url.path,
source.geo.country_iso_code,
source.geo.city_name,
source.as.organization.name
| where Esql.event_count >= 3
| keep
source.ip,
azure.ai_foundry.properties.user_agent,
azure.ai_foundry.properties.apim_subscription_id,
azure.ai_foundry.properties.api_id,
azure.ai_foundry.properties.operation_id,
azure.ai_foundry.properties.backend_response_body.model,
azure.ai_foundry.service_name,
azure.resource.group,
url.domain,
url.path,
source.geo.country_iso_code,
source.geo.city_name,
source.as.organization.name,
Esql.* False Positives
- ⚠ Red-team or application tests that intentionally send prompts the model refuses. Exclude that source IP or API Management subscription.
- ⚠ A shared subscription key behind one NAT can combine refusals from unrelated users. Raise the threshold or split keys per application if that subscription is noisy.
Field Validations
Loading…
Comments (0)
Loading comments...