The analytical deep image prior (ADP) introduced by Dittmer et al. (2020) establishes a link between deep image priors and classical regularization theory via bilevel optimization. While this is an elegant construction, it involves expensive computations if the lower-level problem is to be solved accurately. To overcome this issue, we propose to use adaptive inexact bilevel optimization to solve ADP problems. We discuss an extension of a recent inexact bilevel method called the method of adaptive inexact descent of Salehi et al.(2024) to an infinite-dimensional setting required by the ADP framework. In our numerical experiments we demonstrate that the computational speed-up achieved by adaptive inexact bilevel optimization allows one to use ADP on larger-scale problems than in the previous literature, e.g. in deblurring of 2D color images.