NT549: Reinforcement Learning for Networked Systems

Introduction

Welcome to NT549

This course introduces the fundamental principles and techniques of Reinforcement Learning (RL) with a focus on applications in computer networks and communications. Students will learn how RL agents make decisions through interaction with dynamic environments, and how to implement RL algorithms such as Q-learning, Deep Q-Networks (DQN), policy gradient methods, and actor–critic techniques. Emphasising practical implementation and conceptual understanding, the course prepares students to design and evaluate RL-based solutions for networked systems.

Learning Outcomes

CLO1: Theory

Formulate network problems as Markov Decision Processes (MDPs).

CLO2: Algorithms

Implement RL algorithms (DQN, PPO) using Python/PyTorch.

CLO3: Application

Design custom Gymnasium environments for networks.

CLO4: Synthesis

Analyse the behaviour and performance of RL agents.

Textbooks

BOOK

Reinforcement Learning: An Introduction (2nd Ed)

Sutton & Barto
BOOK

Mathematical Foundations of RL

Shiyu Zhao (2025)
BOOK

Deep Reinforcement Learning for Wireless Communications and Networking

Hoang et al. (IEEE Press, 2023)

Weekly Timeline

Scroll to view all weeks

▼ Scroll

Select a Week

Tap a week from the list above to view details here.

Project Lifecycle Guidelines

General steps for the Capstone Phase (Weeks 11-15)

Phase 1: Proposal & Definition

Teams must present their problem statement. Define the State Space, Action Space, and Reward Function clearly.

Deliverable: 1-page PDF

Phase 2: Environment Design

Showcase a runnable gymnasium environment. A random agent must be able to interact without crashing.

Activity: Code Walkthrough

Phase 3: Baselines

Compare your RL agent against standard algorithms (e.g., Round Robin, OSPF, TCP Vegas). Show preliminary reward plots.

Phase 4: Tuning & Optimization

Address convergence issues. Discuss hyperparameter tuning (learning rate, discount factor) and reward shaping.

Phase 5: Final Defense

Final group presentation and live demo. Evaluation based on Complexity, Methodology, and Performance.

Assessment

Topic Distribution

Component	Weight
Final Exam	50%
Capstone Project	20%
Lab Exercises	20%
Attendance	10%

NT549 | Reinforcement Learning for Networked Systems