GCP Vertex AI Caller Activity From High Number of Countries
Description
Detects Vertex AI authenticated caller activity for the same client.user.email from at least three countries and three source IPs within a four-hour interval. That pattern can indicate stolen user or service-account credentials reused from multiple regions, including VPN or cloud egress and return trips where first and last locations are close.
Query · esql
from logs-gcp_vertexai.auditlogs-*
| where
data_stream.dataset == "gcp_vertexai.auditlogs" and
client.user.email is not null and
source.ip is not null and
source.geo.location is not null and
source.geo.country_name is not null and
(
event.action like "*PredictionService.GenerateContent*" or
event.action like "*PredictionService.StreamGenerateContent*" or
event.action like "*PredictionService.CountTokens*"
)
| stats
Esql.country_count = count_distinct(source.geo.country_name),
Esql.source_ip_count = count_distinct(source.ip),
Esql.user_agent_count = count_distinct(user_agent.original),
Esql.event_count = count(*),
Esql.event_action_values = values(event.action),
Esql.source_ip_values = values(source.ip),
Esql.source_geo_country_name_values = values(source.geo.country_name),
Esql.source_geo_region_name_values = values(source.geo.region_name),
Esql.source_geo_city_name_values = values(source.geo.city_name),
Esql.source_as_organization_name_values = values(source.as.organization.name),
Esql.user_agent_original_values = values(user_agent.original),
Esql.timestamp_first_seen = min(@timestamp),
Esql.timestamp_last_seen = max(@timestamp)
by
client.user.email
| where Esql.country_count >= 3 and Esql.source_ip_count >= 3
| eval source.ip = MV_FIRST(Esql.source_ip_values)
| keep
client.user.email,
source.ip,
Esql.country_count,
Esql.source_ip_count,
Esql.user_agent_count,
Esql.event_count,
Esql.event_action_values,
Esql.source_ip_values,
Esql.source_geo_country_name_values,
Esql.source_geo_region_name_values,
Esql.source_geo_city_name_values,
Esql.source_as_organization_name_values,
Esql.user_agent_original_values,
Esql.timestamp_first_seen,
Esql.timestamp_last_seen
Investigation fields
Pivot points the source recommends for triage.
client.user.emailsource.ipEsql.country_countEsql.source_ip_countEsql.user_agent_countEsql.event_countEsql.event_action_valuesEsql.source_ip_valuesEsql.source_geo_country_name_valuesEsql.source_geo_region_name_valuesEsql.source_geo_city_name_valuesEsql.source_as_organization_name_valuesEsql.user_agent_original_valuesEsql.timestamp_first_seenEsql.timestamp_last_seen
Implementation guide
Requires GCP Vertex AI auditlogs with source.ip, source.geo.location, source.geo.country_name,
client.user.email, and preferably user_agent.original.
Known false positives
- Shared service accounts used by multi-region CI, build agents, or dual-homed VPN/cloud egress can span several countries. Prefer per-region identities, or exclude known automation principals.
- VPN or proxy egress that geo-resolves through hubs far from the caller's physical location. Corroborate with ASN and user-agent diversity before rotating credentials.
Analyst notes
Investigating GCP Vertex AI Caller Activity From High Number of Countries
Successful Vertex AI GenerateContent / CountTokens audit activity for the same
client.user.email spans at least three countries and three source IPs inside four hours.
That pattern can indicate leaked or stolen credentials reused from VPN or cloud egress in
multiple regions, including return trips where the first and last locations are close.
Escalate when countries differ, user agents diverge (SDK vs curl/browser), privileged methods appear on the distant IPs, or IAM key activity precedes the burst. Close as a false positive for documented multi-region automation with consistent corporate tooling.
Possible investigation steps
- Use
Esql.source_geo_country_name_values,Esql.source_geo_region_name_values,Esql.source_ip_values, andEsql.user_agent_original_valuesto map the geographic and client spread. - Use
Esql.event_action_valuesandEsql.event_countto see what the caller did. - Contact the owner of
client.user.email(or the owning team for a service account) to confirm whether VPN or multi-region automation is expected. - Pivot to
logs-gcp_vertexai.prompt_response_logs-*in the same window for prompt abuse, refusals, or token volume after the distant events. - Check GCP IAM / audit logs for key creation, key download, or role grants near the first distant country.
False positive analysis
- Multi-region CI and dual-homed VPN/cloud egress are the main FPs — require ASN/UA corroboration and known-principal allowlists before rotating credentials.
- Shared org-wide service accounts used from many regions by design. Split those identities per region or exclude them.
Response and remediation
- On suspected compromise: disable or rotate the caller's keys/tokens, revoke active sessions, hunt concurrent sessions from the listed IPs, and review IAM bindings after the distant events.
- Prefer region-scoped service accounts for automation so legitimate multi-region traffic does not share one email principal.