PageSourceSearch

https://mengmenglain.com/research/

html mengmenglain.com collected 2026-10-05 06:28:45 UTC 20,229 bytes, 1 lines download raw bytes

1<!DOCTYPE html> <html lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <title>Research | Mengmeng Lain</title> <meta name="author" content="Mengmeng Lain"> <meta name="description" content=""> <meta name="robots" content="noindex"> <link rel="stylesheet" href="/assets/css/bootstrap.min.css?a4b3f509e79c54a512b890d73235ef04"> <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/css/mdb.min.css" integrity="sha256-jpjYvU3G3N6nrrBwXJoVEYI/0zw8htfFnhT9ljN3JJw=" crossorigin="anonymous"> <link defer rel="stylesheet" href="https://unpkg.com/[email protected]/dist/bootstrap-table.min.css"> <link rel="stylesheet" href="/assets/css/academicons.min.css?f0b7046b84e425c55f3463ac249818f5"> <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Roboto:300,400,500,700|Roboto+Slab:100,300,400,500,700|Material+Icons"> <link rel="stylesheet" href="/assets/css/jekyll-pygments-themes-github.css?19f3075a2d19613090fe9e16b564e1fe" media="" id="highlight_theme_light"> <link rel="shortcut icon" href="/assets/img/logo.png?29f3fb3dc80b29007631561817f9cfeb"> <link rel="stylesheet" href="/assets/css/main.css?d41d8cd98f00b204e9800998ecf8427e"> <link rel="canonical" href="https://thatmengmengli.github.io/research/"> <link rel="stylesheet" href="/assets/css/jekyll-pygments-themes-native.css?e74e74bf055e5729d44a7d031a5ca6a5" media="none" id="highlight_theme_dark"> 
1<script src="/assets/js/theme.js?6185d15ea1982787ad7f435576553d64"></script>
vendor: 64 bytes, line 1
1 <script async src="https://www.googletagmanager.com/gtag/js?id=
1G-G76RE8D9X5
vendor: 12 bytes, line 1
1"></script> 
1<script>function gtag(){dataLayer.push(arguments)}window.dataLayer=window.dataLayer||[],gtag("js",new Date),gtag("config","G-G76RE8D9X5");</script>
1 </head> <body class="fixed-top-nav "> <header> <nav id="navbar" class="navbar navbar-light navbar-expand-sm fixed-top"> <div class="container"> <a class="navbar-brand title font-weight-lighter" href="/"> Mengmeng <del>Li</del> Lain </a> <button class="navbar-toggler collapsed ml-auto" type="button" data-toggle="collapse" data-target="#navbarNav" aria-controls="navbarNav" aria-expanded="false" aria-label="Toggle navigation"> <span class="sr-only">Toggle navigation</span> <span class="icon-bar top-bar"></span> <span class="icon-bar middle-bar"></span> <span class="icon-bar bottom-bar"></span> </button> <div class="collapse navbar-collapse text-right" id="navbarNav"> <ul class="navbar-nav ml-auto flex-nowrap"> <li class="nav-item "> <a class="nav-link" href="/">Home</a> </li> <li class="nav-item active"> <a class="nav-link" href="/research/">Research<span class="sr-only">(current)</span></a> </li> <li class="nav-item "> <a class="nav-link" href="/teaching/">Teaching</a> </li> <li class="nav-item "> <a class="nav-link" href="/service/">Service</a> </li> <li class="toggle-container"> <button id="light-toggle" title="Change theme"> <i class="fa-solid fa-moon"></i> <i class="fa-solid fa-sun"></i> </button> </li> </ul> </div> </div> </nav> <progress id="progress" value="0"> <div class="progress-container"> <span class="progress-bar"></span> </div> </progress> </header> <div class="container mt-5"> <div class="post"> <header class="post-header"> <h1 class="post-title">Research</h1> <p class="post-description"></p> </header> <article> <div class="publications"> <ol class="bibliography"> <li> <div class="row"> <div id="li2026robust" class="col-sm-12"> <div class="title">Robust Markov Decision Processes on Continuous State Spaces</div> <div class="author"> <em>ML</em>, Yifan Hu, Daniel Kuhn, Yan Li</div> <div class="periodical"> <em>Working paper</em>, 2026 </div> <div class="links" style="margin-top: 0;"> <a class="abstract btn btn-sm z-depth-0" role="button">Abs</a> <a href="http://arxiv.org/pdf/2605.28706" class="btn btn-sm z-depth-0" role="button" rel="external nofollow noopener" target="_blank">arXiv</a> </div> <div class="abstract hidden"> <p>We study infinite-horizon robust Markov decision processes (MDPs) on continuous state spaces with structured rectangular ambiguity set. The proposed ambiguity set falls within the convex hull of unknown generating kernels. We utilize the dynamic formulation of the corresponding robust MDPs, and subsequently introduce a stochastic first-order method for robust policy evaluation. We establish its high probability convergence to the robust value function, which in turn leads to an O(1/ϵ^2) sample complexity. This high probability accuracy certificate is then used in an approximate policy iteration method that finds an ϵ-optimal policy with O(1/ϵ^2) samples. The obtained sample complexities for both robust policy evaluation and optimization appear to be new for robust MDPs with continuous state spaces. Of independent interest, the proposed method is also directly applicable to zero-sum Markov games, which seems to strictly improve the existing sample complexities for continuous state spaces.</p> </div> </div> </div> </li> <li> <div class="row"> <div id="li2025efficient" class="col-sm-12"> <div class="title">Efficient Best-of-Both-Worlds Algorithms for Contextual Combinatorial Semi-Bandits</div> <div class="author"> <em>ML</em>, Philipp Schneider, Jelisaveta Aleksić, Daniel Kuhn</div> <div class="periodical"> <em>In International Conference on Learning Representations (ICLR)</em>, 2026 </div> <div class="links" style="margin-top: 0;"> <a class="abstract btn btn-sm z-depth-0" role="button">Abs</a> <a href="http://arxiv.org/pdf/2508.18768" class="btn btn-sm z-depth-0" role="button" rel="external nofollow noopener" target="_blank">arXiv</a> <a href="https://proceedings.iclr.cc/paper_files/paper/2026/hash/49cf35ff2298c10452db99d08036805b-Abstract-Conference.html" class="btn btn-sm z-depth-0" role="button" rel="external nofollow noopener" target="_blank">HTML</a> </div> <div class="abstract hidden"> <p>We introduce the first best-of-both-worlds algorithm for contextual combinatorial semi-bandits that simultaneously guarantees O(√T) regret in the adversarial regime and O(ln T) regret in the corrupted stochastic regime. Our approach builds on the Follow-the-Regularized-Leader (FTRL) framework equipped with a Shannon entropy regularizer, yielding a flexible method that admits efficie
1nt implementations. Beyond regret bounds, we tackle the practical bottleneck in FTRL (or, equivalently, Online Stochastic Mirror Descent) arising from the high-dimensional projection step encountered in each round of interaction. By leveraging the Karush-Kuhn-Tucker conditions, we transform the K-dimensional convex projection problem into a single-variable root-finding problem, dramatically accelerating each round. Empirical evaluations demonstrate that this combined strategy not only attains the attractive regret bounds of best-of-both-worlds algorithms but also delivers substantial per-round speed-ups, making it well-suited for large-scale, real-time applications.</p> </div> </div> </div> </li> <li> <div class="row"> <div id="li2025towards" class="col-sm-12"> <div class="title">Towards Optimal Offline Reinforcement Learning</div> <div class="author"> <em>ML</em>, Daniel Kuhn, Tobias Sutter</div> <div class="periodical"> <em></em>R&amp;R in<em> Journal of Machine Learning Research</em>, 2025 </div> <div class="award-line" style="margin: 0; padding: 0;"> <span style="display: inline-block; width: 1.5ch; text-align: center;">☆</span> Second Place, 2025 Dupačová-Prékopa Best Student Paper in Stochastic Programming </div> <div class="links" style="margin-top: 0;"> <a class="abstract btn btn-sm z-depth-0" role="button">Abs</a> <a href="http://arxiv.org/pdf/2503.12283" class="btn btn-sm z-depth-0" role="button" rel="external nofollow noopener" target="_blank">arXiv</a> </div> <div class="abstract hidden"> <p>We study offline reinforcement learning problems with a long-run average reward objective. The state-action pairs generated by any fixed behavioral policy thus follow a Markov chain, and the empirical state-action-next-state distribution satisfies a large deviations principle. We use the rate function of this large deviations principle to construct an uncertainty set for the unknown true state-action-next-state distribution. We also construct a distribution shift transformation that maps any distribution in this uncertainty set to a state-action-next-state distribution of the Markov chain generated by a fixed evaluation policy, which may differ from the unknown behavioral policy. We prove that the worst-case average reward of the evaluation policy with respect to all distributions in the shifted uncertainty set provides, in a rigorous statistical sense, the least conservative estimator for the average reward under the unknown true distribution. This guarantee is available even if one has only access to one single trajectory of serially correlated state-action pairs. The emerging robust optimization problem can be viewed as a robust Markov decision process with a non-rectangular uncertainty set. We adapt an efficient policy gradient algorithm to solve this problem. Numerical experiments show that our methods compare favorably against state-of-the-art methods. </p> </div> </div> </div> </li> <li> <div class="row"> <div id="li2024optimism" class="col-sm-12"> <div class="title">Optimism in the Face of Ambiguity Principle for Multi-Armed Bandits</div> <div class="author"> <em>ML</em>, Daniel Kuhn, Bahar Taşkesen</div> <div class="periodical"> <em></em>Major revision in<em> Operations Research</em>, 2025 </div> <div class="note-line" style="margin: 0; padding: 0;"> <span style="display: inline-block; font-size: 1em;">•</span> Extended abstract appeared in WINE 2024 </div> <div class="links" style="margin-top: 0;"> <a class="abstract btn btn-sm z-depth-0" role="button">Abs</a> <a href="http://arxiv.org/pdf/2409.20440" class="btn btn-sm z-depth-0" role="button" rel="external nofollow noopener" target="_blank">arXiv</a> </div> <div class="abstract hidden"> <p>Follow-The-Regularized-Leader (FTRL) algorithms often enjoy optimal regret for adversarial as well as stochastic bandit problems and allow for a streamlined analysis. Nonetheless, FTRL algorithms require the solution of an optimization problem in every iteration and are thus computationally challenging. In contrast, Follow-The-Perturbed-Leader (FTPL) algorithms achieve computational efficiency by perturbing the estimates of the rewards of the arms, but their regret analysis is cumbersome. We propose a new FTPL algorithm that generates optimal policies for both adversarial and stochastic multi-armed bandits. Like FTRL, our algorithm admits a unified regret analysis, and similar to FTPL, it offers low computational costs. Unlike existing FTPL algorithms that rely on independent additive disturbances governed by a known distribution, we allow for disturbances governed by an ambiguous distribution that is only known to belong to a given set and propose a principle of optimism in the face of ambiguity. Consequently, our framework generalizes existing FTPL algorithms. It also encapsulates a broad range of FTRL methods as special cases, including several optimal ones, which appears to be impossible with current FTPL methods. Finally, we use techniques from discrete choice theory to devise an efficie
1nt bisection algorithm for computing the optimistic arm sampling probabilities. This algorithm is up to 10^4 times faster than standard FTRL algorithms that solve an optimization problem in every iteration. Our results not only settle existing conjectures but also provide new insights into the impact of perturbations by mapping FTRL to FTPL.</p> </div> </div> </div> </li> <li> <div class="row"> <div id="jongeneel2023large" class="col-sm-12"> <div class="title">A Large Deviations Perspective on Policy Gradient Algorithms</div> <div class="author"> (α-β) Wouter Jongeneel, Daniel Kuhn, <em>ML</em> </div> <div class="periodical"> <em>In Learning for Dynamics and Control Conference (L4DC)</em>, 2024 </div> <div class="links" style="margin-top: 0;"> <a class="abstract btn btn-sm z-depth-0" role="button">Abs</a> <a href="http://arxiv.org/pdf/2311.07411" class="btn btn-sm z-depth-0" role="button" rel="external nofollow noopener" target="_blank">arXiv</a> <a href="https://proceedings.mlr.press/v242/jongeneel24a.html" class="btn btn-sm z-depth-0" role="button" rel="external nofollow noopener" target="_blank">HTML</a> </div> <div class="abstract hidden"> <p>Motivated by policy gradient methods in the context of reinforcement learning, we identify a large deviation rate function for the iterates generated by stochastic gradient descent for possibly non-convex objectives satisfying a Polyak-Łojasiewicz condition. Leveraging the contraction principle from large deviations theory, we illustrate the potential of this result by showing how convergence properties of policy gradient with a softmax parametrization and an entropy regularized objective can be naturally extended to a wide spectrum of other policy parametrizations.</p> </div> </div> </div> </li> <li> <div class="row"> <div id="li2023policy" class="col-sm-12"> <div class="title">Policy Gradient Algorithms for Robust MDPs with Non-Rectangular Uncertainty Sets</div> <div class="author"> <em>ML</em>, Daniel Kuhn, Tobias Sutter</div> <div class="periodical"> <em>SIAM Journal on Optimization</em>, 2026 </div> <div class="links" style="margin-top: 0;"> <a class="abstract btn btn-sm z-depth-0" role="button">Abs</a> <a href="http://arxiv.org/pdf/2305.19004" class="btn btn-sm z-depth-0" role="button" rel="external nofollow noopener" target="_blank">arXiv</a> <a href="https://epubs.siam.org/doi/abs/10.1137/24M1631250" class="btn btn-sm z-depth-0" role="button" rel="external nofollow noopener" target="_blank">HTML</a> </div> <div class="abstract hidden"> <p>We propose policy gradient algorithms for robust infinite-horizon Markov decision processes (MDPs) with non-rectangular uncertainty sets, thereby addressing an open challenge in the robust MDP literature. Indeed, uncertainty sets that display statistical optimality properties and make optimal use of limited data often fail to be rectangular. Unfortunately, the corresponding robust MDPs cannot be solved with dynamic programming techniques and are in fact provably intractable. We first present a randomized projected Langevin dynamics algorithm that solves the robust policy evaluation problem to global optimality but is inefficient. We also propose a deterministic policy gradient method that is efficient but solves the robust policy evaluation problem only approximately, and we prove that the approximation error scales with a new measure of non-rectangularity of the uncertainty set. Finally, we describe an actor-critic algorithm that finds an ϵ-optimal solution for the robust policy improvement problem in O(1/ϵ^4) iterations. We thus present the first complete solution scheme for robust MDPs with non-rectangular uncertainty sets offering global optimality guarantees. Numerical experiments show that our algorithms compare favorably against state-of-the-art methods.</p> </div> </div> </div> </li> <li> <div class="row"> <div id="li2021distributionally" class="col-sm-12"> <div class="title">Distributionally Robust Optimization with Markovian Data</div> <div class="author"> <em>ML</em>, Tobias Sutter, Daniel Kuhn</div> <div class="periodical"> <em>In International Conference on Machine Learning (ICML)</em>, 2021 </div> <div class="links" style="margin-top: 0;"> <a class="abstract btn btn-sm z-depth-0" role="button">Abs</a> <a href="http://arxiv.org/pdf/2106.06741" class="btn btn-sm z-depth-0" role="button" rel="external nofollow noopener" target="_blank">arXiv</a> <a href="https://proceedings.mlr.press/v139/li21t.html" class="btn btn-sm z-depth-0" role="button" rel="external nofollow noopener" target="_blank">HTML</a> </div> <div class="abstract hidden"> <p>
1We study a stochastic program where the probability distribution of the uncertain problem parameters is unknown and only indirectly observed via finitely many correlated samples generated by an unknown Markov chain with d states. We propose a data-driven distributionally robust optimization model to estimate the problem’s objective function and optimal solution. By leveraging results from large deviations theory, we derive statistical guarantees on the quality of these estimators. The underlying worst-case expectation problem is nonconvex and involves O(d^2) decision variables. Thus, it cannot be solved efficiently for large d. By exploiting the structure of this problem, we devise a customized Frank-Wolfe algorithm with convex direction-finding subproblems of size O(d). We prove that this algorithm finds a stationary point efficiently under mild conditions. The efficiency of the method is predicated on a dimensionality reduction enabled by a dual reformulation. Numerical experiments indicate that our approach has better computational and statistical properties than the state-of-the-art methods.</p> </div> </div> </div> </li> </ol> </div> </article> </div> </div> <footer class="fixed-bottom"> <div class="container mt-0"> © Copyright 2026 Mengmeng Lain. </div> </footer> 
1<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/jquery.min.js" integrity="sha256-/xUj+3OJU5yExlq6GSYGSHk7tPXikynS7ogEvDej/m4=" crossorigin="anonymous"></script>
1 
1<script src="/assets/js/bootstrap.bundle.min.js"></script>
1 
1<script src="https://cdn.jsdelivr.net/npm/[email protected]/js/mdb.min.js" integrity="sha256-NdbiivsvWt7VYCt6hYNT3h/th9vSTL4EDWeGs5SN3DA=" crossorigin="anonymous"></script>
1 
1<script defer src="https://cdn.jsdelivr.net/npm/[email protected]/dist/masonry.pkgd.min.js" integrity="sha256-Nn1q/fx0H7SNLZMQ5Hw5JLaTRZp0yILA/FRexe19VdI=" crossorigin="anonymous"></script>
1 
1<script defer src="https://cdn.jsdelivr.net/npm/imagesloaded@4/imagesloaded.pkgd.min.js"></script>
1 
1<script defer src="/assets/js/masonry.js" type="text/javascript"></script>
1 
1<script defer src="https://cdn.jsdelivr.net/npm/[email protected]/dist/medium-zoom.min.js" integrity="sha256-7PhEpEWEW0XXQ0k6kQrPKwuoIomz8R8IYyuU1Qew4P8=" crossorigin="anonymous"></script>
1 
1<script defer src="/assets/js/zoom.js?7b30caa5023af4af8408a472dc4e1ebb"></script>
1 
1<script defer src="https://unpkg.com/[email protected]/dist/bootstrap-table.min.js"></script>
1 
1<script src="/assets/js/no_defer.js?d633890033921b33e0ceb13d22340a9c"></script>
1 
1<script defer src="/assets/js/common.js?acdb9690d7641b2f8d40529018c71a01"></script>
1 
1<script defer src="/assets/js/copy_code.js?9b43d6e67ddc7c0855b1478ee4c48c2d" type="text/javascript"></script>
1 
1<script async src="https://d1bxh8uas1mnw7.cloudfront.net/assets/embed.js"></script>
1 
1<script async src="https://badge.dimensions.ai/badge.js"></script>
1 
1<script type="text/javascript">window.MathJax={tex:{tags:"ams"}};</script>
1 
1<script defer type="text/javascript" id="MathJax-script" src="https://cdn.jsdelivr.net/npm/[email protected]/es5/tex-mml-chtml.js"></script>
1 
1<script type="text/javascript">function progressBarSetup(){"max"in document.createElement("progress")?(initializeProgressElement(),$(document).on("scroll",function(){progressBar.attr({value:getCurrentScrollPosition()})}),$(window).on("resize",initializeProgressElement)):(resizeProgressBar(),$(document).on("scroll",resizeProgressBar),$(window).on("resize",resizeProgressBar))}function getCurrentScrollPosition(){return $(window).scrollTop()}function initializeProgressElement(){let e=$("#navbar").outerHeight(!0);$("body").css({"padding-top":e}),$("progress-container").css({"padding-top":e}),progressBar.css({top:e}),progressBar.attr({max:getDistanceToScroll(),value:getCurrentScrollPosition()})}function getDistanceToScroll(){return $(document).height()-$(window).height()}function resizeProgressBar(){progressBar.css({width:getWidthPercentage()+"%"})}function getWidthPercentage(){return getCurrentScrollPosition()/getDistanceToScroll()*100}const progressBar=$("#progress");window.onload=function(){setTimeout(progressBarSetup,50)};</script>
1 </body> </html>

Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.