zeroth-order-optimization

Official implementation for the paper "Model-based Diffusion for Trajectory Optimization". Model-based diffusion (MBD) is a novel diffusion-based trajectory optimization framework that employs a dynamics model to run the reverse denoising process to generate high-quality trajectories.

[JMLR (CCF-A)] PyPop7: A Pure-PYthon LibrarY for POPulation-based Black-Box Optimization (BBO), especially *Large-Scale* algorithm variants. In the near future, we will add Learning-Based Optimizers as its extensions. [https://jmlr.org/papers/v25/23-0386.html]

ZO2 (Zeroth-Order Offloading): Full Parameter Fine-Tuning 175B LLMs with 18GB GPU Memory [COLM2025]

Square Attack: a query-efficient black-box adversarial attack via random search [ECCV 2020]

SOTA Black-box optimization (Python and JS) without the baggage

Official implementation for the paper "CoVO-MPC: Theoretical Analysis of Sampling-based MPC and Optimal Covariance Design" accepted by L4DC 2024. CoVO-MPC is an optimal sampling-based MPC algorithm.

[ICML‘24] Official code for the paper "Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark ".

Powell's Derivative-Free Optimization solvers.

[ICLR'24] "DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training" by Aochuan Chen*, Yimeng Zhang*, Jinghan Jia, James Diffenderfer, Jiancheng Liu, Konstantinos Parasyris, Yihua Zhang, Zheng Zhang, Bhavya Kailkhura, Sijia Liu

This repository contains the PyTorch implementation of Zeroth Order Optimization Based Adversarial Black Box Attack (https://arxiv.org/abs/1708.03999)

Robustify Black-Box Models (ICLR'22 - Spotlight)

Modular optimization library for PyTorch (work-in-progress).

Every Call is Precious: Global Optimization of Black-Box Functions with Unknown Lipschitz Constants

Benchmarking optimization solvers.

Zeroth-Order Regularized Optimization (ZORO): Approximately Sparse Gradients and Adaptive Sampling

Code for IEEE MLSP 2021 paper titled "Model-Free Learning of Optimal Deterministic Resource Allocations in Wireless Systems via Action-Space Exploration"

A curated collection of evolution strategies and zeroth-order optimization for large language models.

SCOBO: Sparsity-aware Comparison Oracle Based Optimization

Blockwise Direct Search (MATLAB version)

Hard-Thresholding Meets Evolution Strategies in Reinforcement Learning