A modular framework for reinforcement learning optimal execution

Posth, Jan-Alexander; De Meer Pardo, Fernando

Publication type:	Conference other
Type of review:	No review
Title:	A modular framework for reinforcement learning optimal execution
Authors:	Posth, Jan-Alexander De Meer Pardo, Fernando
et. al:	No
Conference details:	7th European COST Conference on Artificial Intelligence in Finance, Bern, Switzerland, 30 September 2022
Issue Date:	2022
Language:	English
Subject (DDC):	332: Financial economics
Abstract:	One of the most crucial challenges of Reinforcement Learning applied to market applications is that the true environment (the market) is inaccessible and thus we have to decide on how to simulate its behaviour, be it via historical data or data generation approaches. In this article, we develop a modular framework for the application of Reinforcement Learning to the problem of Optimal Trade Execution. The framework is designed with flexibility in mind, in order to ease the implementation of different simulation setups. Rather than focusing on agents and optimization methods, we focus on the environment and break down the necessary requirements to simulate an Optimal Trade Execution under a Reinforcement Learning framework (data pre-processing, construction of observations, action processing, child order execution, simulation of benchmarks, reward calculations etc.), give examples of each component, explore the difficulties their individual implementations & the interactions between them entail, and discuss the different phenomena that each component induces in the simulation, highlighting the divergences between the simulation and the behavior of a real market. We showcase our modular implementation through a setup that, following a Time-Weighted Average Price (TWAP) order submission schedule, allows the agent to place limit orders, simulates their execution via iterating over snapshots of the Limit Order Book (LOB), and calculates rewards as the $ improvement over the price achieved by a TWAP benchmark algorithm following the same schedule. We also develop evaluation procedures that incorporate iterative re-training and evaluation of a given agent over intervals of a training horizon, mimicking how an agent may behave when being continuously retrained as new market data becomes available, and emulating the monitoring practices that algorithm providers are bound to perform under current regulatory frameworks.
URI:	https://digitalcollection.zhaw.ch/handle/11475/26871
Fulltext version:	Published version
License (according to publishing contract):	Licence according to publishing contract
Departement:	School of Engineering School of Management and Law
Organisational Unit:	Institute of Data Analysis and Process Design (IDP) Institute of Wealth & Asset Management (IWA)
Appears in collections:	Publikationen School of Management and Law

Files in This Item:

There are no files associated with this item.

Show full item record

Posth, J.-A., & De Meer Pardo, F. (2022). A modular framework for reinforcement learning optimal execution. 7th European COST Conference on Artificial Intelligence in Finance, Bern, Switzerland, 30 September 2022.

Posth, J.-A. and De Meer Pardo, F. (2022) ‘A modular framework for reinforcement learning optimal execution’, in 7th European COST Conference on Artificial Intelligence in Finance, Bern, Switzerland, 30 September 2022.

J.-A. Posth and F. De Meer Pardo, “A modular framework for reinforcement learning optimal execution,” in 7th European COST Conference on Artificial Intelligence in Finance, Bern, Switzerland, 30 September 2022, 2022.

POSTH, Jan-Alexander und Fernando DE MEER PARDO, 2022. A modular framework for reinforcement learning optimal execution. In: 7th European COST Conference on Artificial Intelligence in Finance, Bern, Switzerland, 30 September 2022. Conference presentation. 2022

Posth, Jan-Alexander, and Fernando De Meer Pardo. 2022. “A Modular Framework for Reinforcement Learning Optimal Execution.” Conference presentation. In 7th European COST Conference on Artificial Intelligence in Finance, Bern, Switzerland, 30 September 2022.

Posth, Jan-Alexander, and Fernando De Meer Pardo. “A Modular Framework for Reinforcement Learning Optimal Execution.” 7th European COST Conference on Artificial Intelligence in Finance, Bern, Switzerland, 30 September 2022, 2022.