ExPECA: An Experimental Platform for Trustworthy: Supported Experimentatation

Table of Links

• Abstract & Introduction
• Testbed Design and Architecture
• Experimental Testbed Validation
• Supported Experimentation
• Acknowledgements & References

IV. SUPPORTED EXPERIMENTATION

This section delves into the features of the testbed that users can customize to meet their unique experimental goals. These features can be broadly categorized into two types: (1) configurations related to workload and (2) configurations related to the network.

In terms of workload-related configurations, ExPECA offers researchers a range of options to tailor to their experiments. Users can select from various geographical locations for computational workloads using the cloud interface and by reserving public IPs. It is possible to integrate GPUs for tasks requiring accelerated processing, or choose a CPU with a certain number of cores, and allocate a specific amount of RAM to benchmark workload performance. The platform supports experiments involving both static (fixed-location) and mobile (varying-location, e.g., drone with a wireless connectivity dongle) wireless end nodes.

For network-related configurations, ExPECA provides a diverse array of network types, such as SDR 5G, SDR LTE, SDR WiFi, and COTS 5G, among others. Researchers can also customize network topologies and control interference levels, thanks to the testbed’s isolated location. The platform supports a wide range of wireless protocols compatible with USRP e320 and allows users to configure channel conditions. Additionally, the use of Software Defined Radios (SDRs) offers further flexibility, including control over transmit power and the ability to implement a variety of wireless protocols.

ExPECA offers the opportunity to explore various experimentation scenarios involving edge computing, wireless communication, and machine learning research. Here we explain various scenarios, each shedding light on some of the interesting aspects of wireless communication and/or edge computing offering unique insights into the dynamic landscape of edge computing. In Figure 6, we provide an illustration that helps a reader to visualize these scenarios.

Below, we provide a few of such use cases in various research areas. While some of these aim at direct measurements for data-driven modeling and prediction, others target initial observations for forthcoming theoretical work or the verification and validation of existing theoretical models.

A. Optimization of Closed-Loop Applications

With the booming interest and applications surrounding cyber-physical systems, it is imperative to study Networked Control Systems (NCSs) [8]. NCSs form crucial control systems where the control loops are closed through a communication network. Due to the safety-critical nature of the applications these NCSs encompass, such as robots, UAVs etc., there is a need to benchmark and understand the bottlenecks/boundaries associated with these applications [9]. Namely, the latency and reliability boundaries of robotic control are crucial to understand before they can be deployed in real-world scenarios, where their potential failure might prove expensive [10].

Another such feedback system where latency is of importance are human-in-the-loop applications such as augmented reality, wearable cognitive assistants (WCA) [9], [11], and ambient safety. In the context of criticality, we have systems of automated fault detection, in which case the acoustic [12] or motion amplified visual [13], [14] data is processed for vibration analysis to potentially trigger some maintenance, safety or emergency procedures [12]. The ExPECA testbed provides developers with the opportunities to experiment with workloads corresponding to these types of applications in a containerized manner with various real network architectures. These experiments could either be for data collection or for verification of existing theoretical models.

B. Time-sensitive Wireless Networking

Supporting the above-mentioned CPS and HITL applications requires the network to offer a certain level of endto-end network latency, often combined with extremely high reliability, in order to operate correctly and safely. This has spurred various efforts to integrate wireless networking technologies (e.g., 5G URLLC) with wired technologies (e.g., TSN) as shown in Figure 7. However, an efficient wiredwireless integration realization to support time-critical applications necessitates accurate system characterization. In particular, latency characterization of wireless segments is crucial as they are subject to various stochastic and temporal influences, unlike wired links where latencies are immune to external influences. Furthermore, accurate predictions of wireless latency are also envisioned to be leveraged in 6G networks to support deterministic communications [15].

Lately, several data-driven methods for latency prediction have been proposed [16], [17]. Training and validating these approaches require a vast amount of datasets comprising latency measurements alongside network parameters (e.g., SINR and MCS). Precise time-synchronization of all nodes of the testbed ensures accurate latency measurements can be collected between any two nodes of the testbed. These experiments can employ diverse wireless communication technologies, including COTS 5G, software-defined 5G and IEEE 802.11g Wi-Fi integrated in ExPECA. Furthermore, the containerized workflow within ExPECA enables users to perform automated, reproducible and repetitive latency measurement experiments.

C. Hierarchical Learning and Inference

Another set of experiments researchers can use the ExPECA testbed for is the verification of theoretical results corresponding to Hierarchical Learning (HI) [18], [19]. In these works, authors develop the idea of Hierarchical Inference, where a compute constrained edge device chooses to perform local inference or classification of an image sample, or offloading via a wireless channel to get help of a high power edge server for classification. A small-sized ML model with limited accuracy is assumed to have been deployed at the edge device and offloading to the server incurs cost. In essence, these policies aim to make intelligent decisions regarding task offloading, prioritizing complex tasks or tasks where accuracy is paramount. Conversely, they process simpler tasks or tasks where reduced latency is of utmost importance locally. It is to be noted that in this specific work on Hierarchical Inference this decision is made in an online manner. Using the ExPECA testbed, researchers can plan to implement the various algorithms proposed and verify if the real-world results match simulations and models.

D. Federated Learning Validation

Federated learning (FL) is emerging as one of the key approaches for training machine learning models [20]. FL aims to improve over centralized learning in terms of privacy, total training time, computation overhead, etc. However, FL can cause large communication overhead because the machine learning model should be repeatedly exchanged between the central server and the participating nodes during the training process. In order to analyze and optimize communication overhead, many works have proposed models for FL systems operating in wireless environments [21], [22]. Validating these theoretical models in real systems is required to reveal otherwise unavailable insights. The ExPECA testbed supports experimentation involving both centralized learning (CL) and federated learning to evaluate them in terms of communication/computation overhead and accuracy with various wireless conditions. For example in case of communication overhead, CL and FL will consume communication resources when exchanging data (i.e., using uplink to send input data in CL and uplink/downlink to send machine learning model in FL). The ExPECA testbed is able to measure the communication resource usage, and training time (communication/computation time), and capture wireless parameters at that time. Thanks to improved reproducibility and repeatability, the ExPECA testbed supplies stable validation for machine learning architectures.