On November 17, the first Bright Minds, Cold Days Winter Workshop will take place at the Puerta de Toledo Campus of UC3M. We are thrilled to welcome two outstanding speakers: Finale Doshi-Velez, Professor at Harvard University, and Fernando Pérez-Cruz, Professor at ETH Zurich.
Don’t miss this unique opportunity to learn from leading experts in the field. You can register for free here.
Title: Thinking about the Discount Factor in Reinforcement Learning
Abstract:
In RL, we often take the discount factor for granted as a nuisance parameter, spending most of our time learning transitions and rewards (as well as representing states and actions). However, the discount factor is a key ingredient: it is a subjective quality that tells the planner how long we are willing to wait to get the rewards.
In the forward RL setting, works have explored setting the discount factor lower than desired to avoid planner overfitting — taking advantage of an inaccurate transition function by looking too far ahead. I’ll start by sharing work in which we show that this logic also applies in the inverse RL case. Here, the discount factor is unknown — just as we don’t know the agent’s rewards, we don’t know how long the agent was willing to wait to get them. I’ll first share work in which we show that learning the discount factor alongside the rewards, often a smaller one than the true one, improves our inverse RL. Next I’ll describe how if we observe two agents with different behaviors, but we know they are optimizing the same reward function, then we can get greater identifiability in the reward function by jointly learning those agent’s different discount factors. That is, the fact that agents with different levels of patience make different decisions tells us something about the structure of the rewards.
Finally, I’ll return to the forward RL setting to note that there are cases in which this lovely logic doesn’t always go as intended, and provide an alternative. I hope that through these explorations, we’ll have a chance to have fun thinking about this often overlooked RL parameter in a new way.
Title: A Foundational Model for Conditional Forecasting Macroeconomic Variables
Abstract:
Timely and accurate assessments of inflation, unemployment, and GDP are critical for central banks and policymakers to design effective macroeconomic policies. However, traditional econometric tools, such as ARIMA and Bayesian VARs, struggle to handle the challenges posed by lagged, non-stationary, and nonlinear macroeconomic data, necessitating frequent ad hoc adjustments. Recent advancements in transformer-based foundation models offer a promising solution by leveraging multi-headed attention and large-scale pre-training to capture complex cross-variable dependencies without the need for hand-crafted features. This paper introduces M-TSFM, a scalable architecture for macroeconomic time-series analysis, derived from the MOIRAI framework. M-TSFM achieves state-of-the-art performance in nowcasting, forecasting, and scenario analysis while providing probabilistic uncertainty measures. Trained on a novel dataset comprising monthly and quarterly series from 63 economies, the open-weight design of M-TSFM enables researchers to fine-tune the model for specific tasks at low computational cost. By releasing the model publicly, we aim to lower barriers for analysts with limited machine-learning resources and foster more agile, data-driven macroeconomic decision-making. This work builds on the strengths of MOIRAI and extends its capabilities to macroeconomic contexts, offering a universal and adaptive tool for multivariate time-series forecasting.
You can also attend the workshop online here: