publications

2026

1. Workshop

   

   BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Tasks

   Ruth Wan Theng Chew, Zhiliang Chen, Apivich Hemachandra, and 1 more author

   In DEMO Workshop @ ICML 2026, 2026

   Abs PDF Code Website

   Optimization of LLM training and inference configurations, such as hyperparameters, data mixtures, and prompts, is critical to performance, but it is often approached heuristically in practice, leading to potentially suboptimal outcomes. By framing them as noisy, expensive, and derivative-free optimization problems, Bayesian optimization (BO) and other black-box optimization (BBO) methods offer a promising yet underexplored direction for principled, sample-efficient methods. However, LLM training and inference costs are prohibitively high for most of the BBO research community, and new methods are often only evaluated on synthetic test functions and small-scale datasets that fail to capture the challenges of modern LLM optimization problems. This impedes the development of BBO methods and makes it difficult to assess their effectiveness on modern LLM tasks. We introduce BOLT, the first LLM-centric benchmark that democratizes LLM research for the BBO community. BOLT is released at https://github.com/chewwt/bolt. BOLT covers broad and well-motivated LLM optimization problems, involving multi-fidelity, multi-objective, heteroscedastic noise, and high-dimensional search spaces. Each problem in BOLT is grounded in real experimental data and madefully reproducible and accessible through lightweight surrogate models fitted to the results of thousands of real LLM experiments. We benchmark BOLT againstan extensive range of BO and BBO methods, showing that selected BO methods consistently outperform others across tasks and highlighting gaps in existing BBO methods on LLM tasks, underscoring the need to modernize benchmarks for the BBO community.

2. Workshop

   

   Balance Human Agency & AI Assistance in the Tussle for the “Right” to *

   Zi-Yu Khoo, Yuriel Ryan, Nicole Heng Yim Oo, and 8 more authors

   In AI4GOOD Workshop @ ICML 2026, 2026

   Abs PDF

   NLP has been documenting, in its own literature, what its systems do at scale. Language models homogenize text and ideation, causing their users to sound like them. Tutors built on them now mediate learning at scale, often without teacher oversight. They produce fluent text whose con- fidence is uncorrelated with correctness. They collapse when trained recursively on their own outputs. These systems are moving from research artifacts to infrastructure mediating everyday life in how people learn, work, own their creations, and choose what they consume. This paper argues that the missions of NLP research must now include taking responsibility for these effects. We organize this responsibility around four capacities that NLP systems, or AI systems in general, put at risk. Namely, the “rights” to learn, to work, to own, and to choose. For each, we translate findings the NLP community has already made into concrete research questions that admit recognizable AI contributions. The questions are broad enough to admit many lines of inquiry, but specific enough that a successful answer is identifiable as an AI paper, and grounded in interdisciplinary collaboration where the work exceeds what AI or NLP researchers alone can settle. Our aim is to articulate these as a set of new missions for NLP research.

2025

1. ICML

   

   BILBO: BILevel Bayesian Optimization

   Ruth Wan Theng Chew, Quoc Phong Nguyen, and Bryan Kian Hsiang Low

   In International Conference on Machine Learning, 2025

   Abs PDF Code

   Bilevel optimization is characterized by a two-level optimization structure, where the upper-level problem is constrained by optimal lower-level solutions, and such structures are prevalent in real-world problems. The constraint by optimal lower-level solutions poses significant challenges, especially in noisy, constrained, and derivative-free settings, as repeating lower-level optimizations is sample inefficient and predicted lower-level solutions may be suboptimal. We present BILevel Bayesian Optimization (BILBO), a novel Bayesian optimization algorithm for general bilevel problems with blackbox functions, which optimizes both upper- and lower-level problems simultaneously, without the repeated lower-level optimization required by existing methods. BILBO samples from confidence-bounds based trusted sets, which bounds the suboptimality on the lower level. Moreover, BILBO selects only one function query per iteration, where the function query selection strategy incorporates the uncertainty of estimated lower-level solutions and includes a conditional reassignment of the query to encourage exploration of the lower-level objective. The performance of BILBO is theoretically guaranteed with a sublinear regret bound for commonly used kernels and is empirically evaluated on several synthetic and real-world problems.

2. Workshop

   

   README: Rapid Equation Discovery with Multimodal Encoders

   Gregory Kang Ruey Lau, Yue Ran Kang, Zi-Yu Khoo, and 3 more authors

   In 2nd AI for Math Workshop @ ICML 2025, 2025

   Abs PDF

   Discovering scientific laws or interpretable symbolic equations from data rapidly is important in many setting, such as decision-making in time-sensitive high-stake scenarios or applications involving interactive or iterative experimentation such as in scientific or machine learning workflows. However, existing methods, generally known as symbolic regression (SR), typically require long computational time to achieve good performance and have to run from scratch for each dataset. Recent methods that use pre-training SR foundation models for faster inference also suffer from performance limitations and require large training datasets. In this work, we propose README, a framework for rapid equation discovery that can generate performant, interpretable equations from limited, noisy data in just a few seconds, and requires significantly less training data compared to past SR foundation model approaches. We achieve this by being the first to (1) work with image representations of datasets to efficiently capture their key properties, (2) combine the capabilities of open-sourced pre-trained text and image encoders to produce an informative SR embedding space, and (3) develop a novel Grey Wolf Optimizer with Bayesian Optimization (GWOBO) algorithm to rapidly optimize for the best symbolic expression within seconds. We empirically show that README outperforms benchmarks on a wide range of realistic datasets, including real experimental data from various domains and noisy video-extracted dynamics.

2024

1. ICLR

   

   Optimistic Bayesian optimization with unknown constraints

   Quoc Phong Nguyen, Wan Theng Ruth Chew, Le Song, and 2 more authors

   In The Twelfth International Conference on Learning Representations, 2024

   Abs PDF

   Though some research efforts have been dedicated to constrained Bayesian optimization (BO), there remains a notable absence of a principled approach with a theoretical performance guarantee in the decoupled setting. Such a setting involves independent evaluations of the objective function and constraints at different inputs, and is hence a relaxation of the commonly-studied coupled setting where functions must be evaluated together. As a result, the decoupled setting requires an adaptive selection between evaluating either the objective function or a constraint, in addition to selecting an input (in the coupled setting). This paper presents a novel constrained BO algorithm with a provable performance guarantee that can address the above relaxed setting. Specifically, it considers the fundamental trade-off between exploration and exploitation in constrained BO, and, interestingly, affords a noteworthy connection to active learning. The performance of our proposed algorithms is also empirically evaluated using several synthetic and real-world optimization problems.

2021

1. 

   Method and system for object localization, and controlling movement of an autonomous underwater vehicle for object intervention

   Eng Wei Goh, Wan Theng Ruth Chew, and Yaadhav Raaj

   WO Patent WO2021045679A1, 2021

   Abs HTML PDF

   An underwater object localisation method performed by an autonomous underwater vehicle (AUV) (100) having a sensing device coupled to its body is disclosed. In a described embodiment, the method includes generating sensor data (312) corresponding to an underwater object (314) detected by the sensing device, generating a distribution of N particles (203) based on the generated sensor data (312) to represent a posterior distribution of the object’s location, with each particle (203) associated with an initial position (326) relative to a reference of the AUV body (100), and transforming the initial positions (326) of the N particles (203) into initial object global coordinates (208) using the AUV’s global coordinates (334) to define the object’s approximate 3D location, the object global coordinates (208) being independent of the AUV body reference. A localisation system and a control system are also disclosed.