Paper 2022/1385

Deep Reinforcement Learning-based Rebalancing Policies for Profit Maximization of Relay Nodes in Payment Channel Networks

Nikolaos Papadis, Yale University
Leandros Tassiulas, Yale University
Abstract

Payment channel networks (PCNs) are a layer-2 blockchain scalability solution, with its main entity, the payment channel, enabling transactions between pairs of nodes "off-chain," thus reducing the burden on the layer-1 network. Nodes with multiple channels can serve as relays for multihop payments by providing their liquidity and withholding part of the payment amount as a fee. Relay nodes might after a while end up with one or more unbalanced channels, and thus need to trigger a rebalancing operation. In this paper, we study how a relay node can maximize its profits from fees by using the rebalancing method of submarine swaps. We introduce a stochastic model to capture the dynamics of a relay node observing random transaction arrivals and performing occasional rebalancing operations, and express the system evolution as a Markov Decision Process. We formulate the problem of the maximization of the node's fortune over time over all rebalancing policies, and approximate the optimal solution by designing a Deep Reinforcement Learning (DRL)-based rebalancing policy. We build a discrete event simulator of the system and use it to demonstrate the DRL policy's superior performance under most conditions by conducting a comparative study of different policies and parameterizations. Our work is the first to introduce DRL for liquidity management in the complex world of PCNs.

Metadata
Available format(s)
PDF
Category
Applications
Publication info
Published elsewhere. Minor revision. Best Paper Award at the 4th International Conference on Mathematical Research for the Blockchain Economy (MARBLE 2023)
Keywords
Payment Channel NetworkLightning Networkrebalancingsubmarine swapDeep Reinforcement Learningoptimizationsimulation
Contact author(s)
nikolaos papadis @ yale edu
leandros tassiulas @ yale edu
History
2023-10-08: revised
2022-10-13: received
See all versions
Short URL
https://ia.cr/2022/1385
License
Creative Commons Attribution
CC BY

BibTeX

@misc{cryptoeprint:2022/1385,
      author = {Nikolaos Papadis and Leandros Tassiulas},
      title = {Deep Reinforcement Learning-based Rebalancing Policies for Profit Maximization of Relay Nodes in Payment Channel Networks},
      howpublished = {Cryptology ePrint Archive, Paper 2022/1385},
      year = {2022},
      note = {\url{https://eprint.iacr.org/2022/1385}},
      url = {https://eprint.iacr.org/2022/1385}
}
Note: In order to protect the privacy of readers, eprint.iacr.org does not use cookies or embedded third party content.