Show newer

Accelerated Bregman gradient methods for relatively smooth and relatively Lipschitz continuous minimization problems arxiv.org/abs/2411.16743

Accelerated Bregman gradient methods for relatively smooth and relatively Lipschitz continuous minimization problems

In this paper, we propose some accelerated methods for solving optimization problems under the condition of relatively smooth and relatively Lipschitz continuous functions with an inexact oracle. We consider the problem of minimizing the convex differentiable and relatively smooth function concerning a reference convex function. The first proposed method is based on a similar triangles method with an inexact oracle, which uses a special triangular scaling property for the used Bregman divergence. The other proposed methods are non-adaptive and adaptive (tuning to the relative smoothness parameter) accelerated Bregman proximal gradient methods with an inexact oracle. These methods are universal in the sense that they are applicable not only to relatively smooth but also to relatively Lipschitz continuous optimization problems. We also introduced an adaptive intermediate Bregman method which interpolates between slower but more robust algorithms non-accelerated and faster, but less robust accelerated algorithms. We conclude the paper with the results of numerical experiments demonstrating the advantages of the proposed algorithms for the Poisson inverse problem.

arXiv.org

On quasi-convex smooth optimization problems by a comparison oracle arxiv.org/abs/2411.16745

On quasi-convex smooth optimization problems by a comparison oracle

Frequently, when dealing with many machine learning models, optimization problems appear to be challenging due to a limited understanding of the constructions and characterizations of the objective functions in these problems. Therefore, major complications arise when dealing with first-order algorithms, in which gradient computations are challenging or even impossible in various scenarios. For this reason, we resort to derivative-free methods (zeroth-order methods). This paper is devoted to an approach to minimizing quasi-convex functions using a recently proposed comparison oracle only. This oracle compares function values at two points and tells which is larger, thus by the proposed approach, the comparisons are all we need to solve the optimization problem under consideration. The proposed algorithm to solve the considered problem is based on the technique of comparison-based gradient direction estimation and the comparison-based approximation normalized gradient descent. The normalized gradient descent algorithm is an adaptation of gradient descent, which updates according to the direction of the gradients, rather than the gradients themselves. We proved the convergence rate of the proposed algorithm when the objective function is smooth and strictly quasi-convex in $\mathbb{R}^n$, this algorithm needs $\mathcal{O}\left( \left(n D^2/\varepsilon^2 \right) \log\left(n D / \varepsilon\right)\right)$ comparison queries to find an $\varepsilon$-approximate of the optimal solution, where $D$ is an upper bound of the distance between all generated iteration points and an optimal solution.

arXiv.org
Show older
Qoto Mastodon

QOTO: Question Others to Teach Ourselves
An inclusive, Academic Freedom, instance
All cultures welcome.
Hate speech and harassment strictly forbidden.