Saturday, January 13, 2024

Monday, January 30, 2023

Communication Slice Modeling and Optimization with SARSA Reinforcement Learning

Communication Slice Modeling and Optimization with SARSA Reinforcement Learning: In this document, we present a conceptual model of network slicing, we then formulate analytically some aspects of the model and the optimization problem to address. Next, we propose to use a reinforcement learning SARSA agent to solve the optimization problem and implement a proof of concept prototype highlighting its results.

Sunday, February 07, 2021

Reconfiguração de Redes de Distribuição de Energia Elétrica Utilizando Aprendizado de Máquina

Reconfiguração de Redes de Distribuição de Energia Elétrica Utilizando Aprendizado de Máquina: Electrical networks are composed of stages of generation, transmission, and distribution of energy. Distribution networks (RD) are an important element of the electricity grid because it provides the effective delivery of energy to end users. The RD’s are subject to failure and their optimization is of fundamental importance in the context of Smart Grids, where it is sought a greater efficiency of the processes involved between the production and distribution of energy. The distribution networks (RD) have topologies and loads of various type. This dissertation proposes a method and algorithm for the reconfiguration of electrical network using machine learning with linear regression and branch exchange algorithm aiming the optimization of RD operation. The method and algorithm proposed do maneuvers in the RD as transfer and load balancing and aiming to increase its level of reliability. The proposal is validated in a test network of IEEE (IEEEbus14) using simulation and testing environment implemented in “R” language and using the Newton Raphson method to calculate the power flow. The solution developed show satisfactory in supporting the decision-making for three reconfigurations of distribution networks in the context of the Smart Grid.

Thursday, January 14, 2021

Illustrated Technical Paper - Enhanced Pub/Sub Network Communication

Illustrated Technical Paper - Enhanced Pub/Sub Network Communication: This illustrated technical paper presents the slides describing the contents of the paper 'Enhanced Pub/Sub Communications for Massive IoT Traffic with SARSA Reinforcement Learning'. The talk was presented at the 3rd International Conference on Machine Learning for Networking (MLN'2020), 24 - 26 November 2020 at Paris, France - http://www.adda-association.org/mln-2020. The illustrated technical paper format is intended to complement, enrich and subsidize the technical paper content and contains slides, complementary text and additional and/or focused bibliographic references.

Sunday, January 03, 2021

Enhanced Pub/Sub Communications for Massive IoT Traffic with SARSA Reinforcement Learning

Enhanced Pub/Sub Communications for Massive IoT Traffic with SARSA Reinforcement Learning: Sensors are being extensively deployed and are expected to expand at significant rates in the coming years. They typically generate a large volume of data on the internet of things (IoT) application areas like smart cities, intelligent traffic systems, smart grid, and e-health. Cloud, edge and fog computing are potential and competitive strategies for collecting, processing, and distributing IoT data. However, cloud, edge, and fog-based solutions need to tackle the distribution of a high volume of IoT data efficiently through constrained and limited resource network infrastructures. This paper addresses the issue of conveying a massive volume of IoT data through a network with limited communications resources (bandwidth) using a cognitive communications resource allocation based on Reinforcement Learning (RL) with SARSA algorithm. The proposed network infrastructure (PSIoTRL) uses a Publish/ Subscribe architecture to access massive and highly distributed IoT data. It is demonstrated that the PSIoTRL bandwidth allocation for buffer flushing based on SARSA enhances the IoT aggregator buffer occupation and network link utilization. The PSIoTRL dynamically adapts the IoT aggregator traffic flushing according to the Pub/Sub topic's priority and network constraint requirements.