11/12/2022 0 Comments Netfits utility![]() The attributed flow data drives various use cases within Netflix like network monitoring and network usage forecasting available via Lumen dashboards and machine learning based network segmentation. The attributed flows are pushed to Keystone that routes them to the Hive and Druid datastores. It performs real time attribution of flow data with application metadata from Sonar. Sonar is an IPv6 and IPv4 address identity tracking service.įlow Collector consumes two data streams, the IP address change events from Sonar via Kafka and eBPF flow log data from the Flow Exporter sidecars. We use Sonar to attribute an IP address to a specific application at a particular time. IP addresses within the cloud can move from one EC2 instance or Titus container to another over time. So how do we ingest and enrich these flows at scale ?įlow Collector is a regional service that ingests and enriches flows. These metrics are visualized using Lumen, a self-service dashboarding infrastructure. The Flow Exporter also publishes various operational metrics to Atlas. The runtime behavior of the Flow Exporter can be dynamically managed by configuration changes via Fast Properties. The choice of transport protocols like GRPC, HTTPS & UDP is runtime dependent on characteristics of the instance placement. The sidecar has been implemented by leveraging the highly performant eBPF along with carefully chosen transport protocols to consume less than 1% of CPU and memory on any instance in our fleet. In some ways, eBPF does to the kernel what JavaScript does to websites: it allows all sorts of new applications to be created.Īn eBPF flow log record represents one or more network flows that contain TCP/IP statistics that occur within a variable aggregation interval. What is BPF?īerkeley Packet Filter (BPF) is an in-kernel execution engine that processes a virtual instruction set, and has been extended as eBPF for providing a safe way to extend kernel functionality. The Flow Exporter is a sidecar that uses eBPF tracepoints to capture TCP flows at near real time on instances that power the Netflix microservices architecture. By collecting, accessing and analyzing network data from a variety of sources like VPC Flow Logs, ELB Access Logs, eBPF flow logs on the instances, etc, we can provide network insight to users and central teams through multiple data visualization techniques like Lumen, Atlas, etc. Network Availability: The expected continued growth of our ecosystem makes it difficult to understand our network bottlenecks and potential limits we may be reaching.Ĭloud Network Insight is a suite of solutions that provides both operational and analytical insight into the cloud network infrastructure to address the identified problems.Without having network visibility, it’s difficult to improve our reliability, security and capacity posture. Service Segmentation: The ease of the cloud deployments has led to the organic growth of multiple AWS accounts, deployment practices, interconnection practices, etc.Pathway Validation: Netflix velocity of change within the production streaming and studio environment can result in the inability of services to communicate with other resources.App Dependencies and Data Flow Mappings: With the number of micro services growing by the day without understanding and having visibility into an application’s dependencies and data flows, it is difficult for both service owners and centralized teams to identify systemic issues.While we strive to keep the ecosystem simple, the inherent nature of leveraging a variety of technologies will lead us to challenges such as: Netfits utility software#Netflix software infrastructure is a large distributed ecosystem that consists of specialized functional tiers that are operated on the AWS and Netflix owned services. The cloud network infrastructure that Netflix utilizes today consists of AWS services such as VPC, DirectConnect, VPC Peering, Transit Gateways, NAT Gateways, etc and Netflix owned devices. At much less than 1% of CPU and memory on the instance, this highly performant sidecar provides flow data at scale for network insight. Netflix has developed a network observability sidecar called Flow Exporter that uses eBPF tracepoints to capture TCP flows at near real time. How Netflix uses eBPF flow logs at scale for network insightīy Alok Tiagi, Hariharan Ananthakrishnan, Ivan Porto Carrero and Keerti Lakshminarayan ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |