Coordinating Traffic at Group Granularity for Tail Latency Control in Service Meshes

초록

Microservice architectures execute user requests through chains of replica-to-replica remote procedure calls, where end-to-end tail latency is shaped not only by cumulative processing and network delays, but also by how routing decisions are coordinated across replicas at short timescales. Modern service meshes provide a proxy-based data plane for traffic control, yet their default mechanisms expose a fundamental tension: replica-level routing preserves locality and isolation but fragments decisions across many independent proxies, while workload-level proxies improve coordination at the cost of additional inter-node detours and expanded failure domains. Existing heterogeneity-aware schedulers improve long-term traffic distribution by incorporating compute capacity or network latency, but they continue to enforce routing independently at replicas, leaving short-timescale traffic concentration largely unaddressed. As a result, transient queueing and tail-latency spikes persist even under well-provisioned steady-state allocations. We present ARCANE, a traffic control framework that introduces group-level coordination as a new control primitive in service meshes. ARCANE elevates routing decisions from individual replicas to groups of co-located replicas, enabling group-level coordination that mitigates short-timescale traffic concentration while preserving locality and service isolation. ARCANE combines a parameter-free adaptive proportioning algorithm that reacts to runtime load imbalance with a costaware sparse routing construction that reduces routing fragmentation and communication overhead. We implement ARCANE as a Kubernetes operator integrated with Istio and evaluate it on heterogeneous edge-cloud deployments. Experiments on multi-hop service chains demonstrate that ARCANE significantly reduces P99 tail latency and inter-node traffic compared to Least Request, HEART, and Proxy-Service, while remaining compatible with existing service-mesh architectures. © 2026 IEEE.

키워드