A Self-Contained Reinforcement Learning Code of FEC for ns-3

Forward Erasure Correction (FEC)

Packet-level forward erasure correction is effective for achieving low-latency transmissions in non-terrestrial networks (NTNs), which often contain lossy links with long propagation delay[1]. In [1] the authors showed that the problem can be formulated using reinforcement learning (RL) by tracking history feedback and system state and carefully devised state space and reward signal, the RL scheme is shown
to be effective for achieving high and smooth in-order goodput with low delay.

Reinforcement Learning based FEC (RL-FEC)

In [2] the authors of [1] provided the code of the project that implements a reinforcement learning (RL) based forward-erasure correction (FEC) scheme in ns-3. FEC is effective for achieving low-latency transmissions over lossy links with long propagation delay. Such links will be commonly seen in non-terrestrial networks (NTNs) in the forthcoming 6G, including satellite links, UAV/HAP links, etc[2].

Requirements

The authors tested the code in ns-3.35 so we will need a working installation of ns-3.35.
ns-3 needs to be configured with –disable-werror to avoid warning of unused variables that would result in errors or failure of the compile process.

Since the authors didn’t use any big AI/ML framework to implement the Reinforcement Learning based FEC, we will not need any third party AI/ML framework such as TensorFlow, Keras etc.,

In this procedure, we tried to install RL-FEC under a chroot based ns-3 installation.

Installation

A) Installation of ns-3.35

You may follow any procedure such as [3] to install ns-3.35 on your computer. For our demonstrations, we are going to use a chroot Jail based installation. In the following link, I provided chroot Jail based installation procedure for ns-3.35.

Installing ns3.35 in Debian 10 chroot Jail Under Debian 11 Host OS or any Version of Linux Host

The following terminal shows the successful compile of ns-3.35 under chroot jail

B) Installing RL-FEC Code

In our chroot based ns-3 install, we did the installation under /home folder of the chroot environment. So, we are doing this installation according to that path.

Step 1: Clone RL-FEC from GitHub repository

Conclusion

Unlike ns3-AI and ns3-Gym, the RL-FEC provides a simple way of implementing reinforcement learning under ns-3. The same idea can be used to implement some other elementary ML/AI algoritym under ns-3.

References

F. Zhang, Y. Li, J. Wang, T. Q. Quek, “Learning Based FEC for Non-Terrestrial Networks with Delayed Feedback”, IEEE Communications Letters, Accepted. (Early Access: https://ieeexplore.ieee.org/document/9628079)
https://github.com/yeliqseu/rl-fec
https://www.nsnam.org/wiki/Installation