Quantum optimization, for both classical and quantum functions, is one of the most well-studied applications of quantum computing, but recent trends have relied on hybrid methods that push much of the fine-tuning off onto costly classical algorithms. Feedback-based quantum algorithms, such as FALQON, avoid these fine-tuning problems but at the cost of additional circuit depth and a lack of convergence guarantees. In this work we take the local greedy information collected by Lyapunov feedback control and develop an analytic framework to use it to perturbatively update previous control layers, similar to the global optimal control achievable using Pontryagin optimal control. This perturbative methodology, which we call feedback-based optimally controlled quantum states (FOCQS), can be used to improve the results of feedback-based algorithms, like FALQON, providing better convergence but still no formal guarantees of convergence. Furthermore, this perturbative method can be used to push smooth annealing-like control protocol closer to the control optimum, even providing an iterative approach, albeit with diminishing returns. In numerical testing, we show improvements in convergence and required depth due to these methods over existing quantum feedback control methods.