Better Decision-Making Through Data

From what to watch on TV to how to diversify a stock portfolio, our decisions are increasingly based on algorithmic recommendation systems, which in turn are based on data informed by our decisions, in an endless feedback loop.

So how can we be sure that feedback loop is used to refine the system, rather than just trap users in a content bubble?

“Going beyond learning to make decisions using data, my research asks what kind of decisions will generate better data?” says Shipra Agrawal, an assistant professor of industrial engineering and operations research (IEOR).

To build such continuously improving decision-making mechanisms, Agrawal utilizes reinforcement learning (RL), a branch of machine learning where agents interacting with their environment learn through trial and error to improve in future scenarios. This tool is especially promising for situations involving complex, dynamic, or uncertain environments— say, when a driverless car needs to determine the speediest route to a specific destination—but as Agrawal has found, it can also be leveraged in surprising directions.

“Reinforcement learning has generated significant recent interest in the artificial intelligence community, due to unprecedented successes demonstrated in game playing applications,” she says. “We propose to use reinforcement learning in a starkly different context: for problem solving and algorithm discovery.”

Take portfolio optimization, where decisions must adapt in response to real-time market data. There, an RL agent could learn to interact with the market, effectively devising its own algorithm to solve the challenge. Building on that idea, Agrawal’s collaboration with fellow IEOR faculty member Yuri Faenza combines their expertise in deep RL and discrete optimization to create RL agents that can outperform human-designed algorithms for solving general optimization tasks such as scheduling and production planning.

Agrawal, who has collaborated with Google AI on auctions and Amazon on content layout and sponsored products, also sees the bigger social impact for her research. She hopes, in time, to empower end users with a better understanding of the policies behind data-driven decisions and whether they are fair and unbiased.

“It’s important to analyze learning and decision-making algorithms,” she says, “so we can quantify not just the performance but also the long-term implications of these algorithmic decisions.”