Three-dimensional structured illumination microscopy (3D-SIM) doubles the resolution of fluorescence imaging in all directions and enables optical sectioning with increased image contrast. However, 3D-SIM has not been widely applied to imaging deep in thick tissues due to its sensitivity to sample-induced aberrations, making the method difficult to apply beyond 10 µm in depth. Furthermore, 3D-SIM has not been available in an upright configuration, limiting its use for live imaging while manipulating the specimen, for example, with electrophysiology. Here, we have overcome these barriers by developing a novel upright 3D-SIM system (termed Deep3DSIM) that incorporates adaptive optics for aberration correction and remote focusing, reducing artefacts, improving contrast, restoring resolution, and eliminating the need to move the specimen or the objective lens in volume imaging. These advantages are equally applicable to inverted 3D-SIM systems. We demonstrate high-quality 3D-SIM imaging in various samples, including imaging more than 130 µm into the Drosophila brain.

3D structured illumination microscopy (3D-SIM) doubles the resolution of fluorescence imaging in lateral and axial directions and increases contrast in both fixed and live specimens. However, 3D-SIM has so far not been widely applied to imaging deep in thick tissues due to its sensitivity to specimen-induced aberrations, making the method difficult to apply beyond 10 µm in depth. Furthermore, 3D-SIM has not been available in an upright configuration, limiting its use for live imaging while manipulating the specimen, for example with electrophysiology. Here, we have overcome these barriers by developing a novel upright 3D-SIM system (termed Deep3DSIM) that incorporates adaptive optics (AO) for aberration correction and remote focusing, reducing artefacts, and removing the need to move the specimen or objective. Both these advantages are equally applicable to inverted 3D-SIM microscopes. We demonstrate high-quality 3D-SIM imaging up to 130 µm into complex tissue and live sample manipulation, including human cells and Drosophila larval brains and embryos.### Competing Interest StatementThe authors have declared no competing interest.

Aberrations are a common problem in microscopes resulting in compromised imaging contrast and resolution. Adaptive optics (AO) can correct aberrations but requires either a wavefront sensor or a wavefront-sensorless AO method that requires multiple sample exposures. We created a machine learning (ML) approach that embeds physical understanding of the imaging process into a sensorless AO method. This enables correction of aberrations with as few as two sample exposures. The method was translated across different microscope modalities. This includes two-photon microscopy and three-photon microscopy of in vivo mouse neural activity, showing robustness to specimen motion and activity related intensity variations.

The resolution and contrast of microscope imaging is often affected by aberrations introduced by imperfect optical systems and inhomogeneous refractive structures in specimens. Adaptive optics (AO) compensates these aberrations and restores diffraction limited performance. A wide range of AO solutions have been introduced, often tailored to a specific microscope type or application. Until now, a universal AO solution - one that can be readily transferred between microscope modalities - has not been deployed. We propose versatile and fast aberration correction using a physics-based machine learning assisted wavefront-sensorless AO control (MLAO) method. Unlike previous ML methods, we used a specially constructed neural network (NN) architecture, designed using physical understanding of the general microscope image formation, that was embedded in the control loop of different microscope systems. The approach means that not only is the resulting NN orders of magnitude simpler than previous NN methods, but the concept is translatable across microscope modalities. We demonstrated the method on a two-photon, a three-photon and a widefield three-dimensional (3D) structured illumination microscope. Results showed that the method outperformed commonly-used modal-based sensorless AO methods. We also showed that our ML-based method was robust in a range of challenging imaging conditions, such as 3D sample structures, specimen motion, low signal to noise ratio and activity-induced fluorescence fluctuations. Moreover, as the bespoke architecture encapsulated physical understanding of the imaging process, the internal NN configuration was no-longer a "black box", but provided physical insights on internal workings, which could influence future designs.

Super-resolution microscopy allows the observation of sub-cellular structures with a resolution beyond diffraction limit of conventional fluorescence microscopy. However, most super-resolution microscopes have a limited imaging depth due to the inhomogeneous refractive index of the sample that leads to optical aberrations. Adaptive optics has been successfully adopted by many imaging techniques, including 3D Structured illumination microscopy (SIM). We use a fast deformable mirror to modulate the wavefront of fluorescence to compensate for optical aberration and changing focus position at the same time. Adaptive optics successfully extends the depth, the range and the speed of 3D-SIM imaging.

We have developed “Microscope-Cockpit” (Cockpit), a highly adaptable open source user-friendly Python-based Graphical User Interface (GUI) environment for precision control of both simple and elaborate bespoke microscope systems. The user environment allows next-generation near instantaneous navigation of the entire slide landscape for efficient selection of specimens of interest and automated acquisition without the use of eyepieces. Cockpit uses “Python-Microscope” (Microscope) for high-performance coordinated control of a wide range of hardware devices using open source software. Microscope also controls complex hardware devices such as deformable mirrors for aberration correction and spatial light modulators for structured illumination via abstracted device models. We demonstrate the advantages of the Cockpit platform using several bespoke microscopes, including a simple widefield system and a complex system with adaptive optics and structured illumination. A key strength of Cockpit is its use of Python, which means that any microscope built with Cockpit is ready for future customisation by simply adding new libraries, for example machine learning algorithms to enable automated microscopy decision making while imaging.

Custom-built microscopes often require control of multiple hardware devices and precise hardware coordination. It is also desirable to have a solution that is scalable to complex systems and that is translatable between components from different manufacturers. Here we report Python-Microscope, a free and open-source Python library for high-performance control of arbitrarily complex and scalable custom microscope systems. Python-Microscope offers simple to use Python-based tools, abstracting differences between physical devices by providing a defined interface for different device types. Concrete implementations are provided for a range of specific hardware, and a framework exists for further expansion. Python-Microscope supports the distribution of devices over multiple computers while maintaining synchronisation via highly precise hardware triggers. We discuss the architectural features of Python-Microscope that overcome the performance problems often raised against Python and demonstrate the different use cases that drove its design: integration with user-facing projects, namely the Microscope-Cockpit project; control of complex microscopes at high speed while using the Python programming language; and use as a microscope simulation tool for software development.

Bespoke microscopes often require control of multiple hardware devices and precise hardware coordination. It is also desirable to have a control solution that is scalable to more complex systems and translatable between components from different manufacturers. Here we report Python-Microscope, a free and open source Python library for high performance control of arbitrarily complex and scalable bespoke microscopes. Python-Microscope offers an elegant pythonic software platform to control microscopes, abstracting differences between physical devices by providing a defined interface for different device types. These include cameras, filter wheels, light sources, deformable mirrors, and stages. Concrete implementations are provided for a range of specific hardware and a framework is in place for further expansion. Python-Microscope supports the distribution of devices over multiple computers while maintaining synchronisation via highly precise hardware triggers. We discuss the architecture choices of Python-Microscope that overcome the performance problems often raised against Python and demonstrate the different use cases that drove its design: its integration in user facing projects, namely in the Microscope-Cockpit project; in controlling complex microscopes at high speed while using the Python programming language; and as a microscope simulation tool for software development.