AWS Bedrock LLM Sensitive Content Refusals

from logs-aws_bedrock.invocation-*
| WHERE @timestamp > NOW() - 1 DAY
   
AND (
     gen_ai.completion LIKE "*I cannot provide any information about*"
     
AND gen_ai.completion LIKE "*end_turn*"
   )
| STATS user_request_count = count() BY gen_ai.user.id
| WHERE user_request_count >= 3

False Positives

• ⚠ Examine flagged interactions for patterns or anomalies in user requests that may indicate malicious intent or probing of model boundaries.
• ⚠ Regularly review and update the phrases that trigger refusals to adapt to new ethical guidelines and compliance requirements.
• ⚠ Ensure that data logs contain enough detail to provide context around the refusal, which will aid in subsequent investigations by security teams.