Recent advances in neural rendering have unlocked unprecedented capabilities in 3D reconstruction and novel view synthesis, giving rise to applications such as virtual fly-throughs of a 3D scene reconstructed from a set of sparse, casually captured images. However, these renderings are viewed on a computer screen or conventional VR headsets as 2D images, greatly limiting the perceptual realism and immersiveness of such experiences. The rapid development in novel 3D scene representations calls for dedicated rendering algorithms that convert these readily-available 3D contents into formats that are compatible with emerging 3D display technologies, such as holographic displays. In this paper, we propose a wave-optics rendering pipeline that works with multiplane images (MPIs) for efficient and high-quality hologram synthesis. Our MPI-based computer-generated holography algorithm greatly outperforms state-of-the-art primitive-based CGH algorithms in terms of runtime, achieving speedups up to 250,000x while achieving comparable image quality, and significantly outperforms conventional layer-based CGH algorithms in terms of image quality. We validate our method extensively on a wide variety of 3D scene datasets both in simulation and through experimentally captured results, showing exceptional 3D focal stack and 4D light field reconstruction performance without sacrificing efficiency.
The holographic principle offers unprecedented capabilities for both near-eye and direct-view displays. However, the limited space-bandwidth product of current hardware, together with the lack of co-designed display architectures and rendering algorithms, remains a key challenge for practical deployment. This paper reviews recent advances in holographic display rendering and system design, with a particular focus on improving and efficiently utilizing the space-bandwidth product.
Mixed-reality (MR) display systems enable transformative user experiences across various domains, including communication, education, training and entertainment. To create an immersive and accessible experience, the display engine of the MR display must project perceptually realistic 3D images over a wide field of view observable from a large range of possible pupil positions, that is, it must support a large & eacute;tendue. Current MR displays, however, fall short in delivering these capabilities in a compact device form factor. Here we present an ultra-thin MR display design that overcomes these challenges using a unique combination of waveguide holography and artificial intelligence (AI)-driven holography algorithms. One of the key innovations of our display system is a compact, custom-designed waveguide for holographic near-eye displays that supports a large effective & eacute;tendue. This is co-designed with an AI-based algorithmic framework combining an implicit large-& eacute;tendue waveguide model, an efficient wave propagation model for partially coherent mutual intensity and a computer-generated holography framework. Together, our unique co-design of a waveguide holography system and AI-driven holographic algorithms represents an important advancement in creating visually comfortable and perceptually realistic 3D MR experiences in a compact wearable device.
Near-eye displays are fundamental technology in the next generation computing platforms for augmented reality and virtual reality. However, there are remaining challenges to deliver immersive and comfortable visual experiences to users, such as compact form factor, solving vergence-accommodation conflict, and achieving a high resolution with a large eyebox. Here we show a compact holographic near-eye display concept that combines the advantages of waveguide displays and holographic displays to overcome the challenges towards true 3D holographic augmented reality glasses. By modeling the coherent light interactions and propagation via the waveguide combiner, we demonstrate controlling the output wavefront using a spatial light modulator located at the input coupler side. The proposed method enables 3D holographic displays via exit-pupil expanding waveguide combiners, providing a large software-steerable eyebox. It also offers additional advantages such as resolution enhancement capability by suppressing phase discontinuities caused by pupil replication process. We build prototypes to verify the concept with experimental results and conclude the paper with discussion.
Light field (LF) displays can offer a quasi-natural three-dimensional (3D) viewing experience by tailoring the four-dimensional light information. However, the primary drawback of conventional LF displays is their limited image quality due to restricted information. To address this limitation, time-multiplexing techniques are employed, but the resulting system configurations are often impractical for achieving compact systems. Here, we present a compact time-sequential super multi-view LF near-eye display integrated with a front-lit, multi-directional illumination module for virtual reality application. This illumination module consists of an RGB light-emitting diode array and a waveguide, allowing us to exploit a high-speed reflective-type spatial light modulator and provide high-bit depth, full-color 3D scenes with quasi-continuous parallax in a compact form factor display. The prototype is implemented for a live demo and evaluated with both simulation and experiment. Our work provides a viable path towards the wide adoption of 3D virtual reality head-mounted displays.
In this work, we present a proof of concept Holographic Near Eye Display (HNED) that can, in principle, be very thin (few mm’s), have high resolution, enlarged eye box, provide wide Field of View (FOV), correct for the user’s prescription, and display 3D content, and thus avoid the Vergence-Accommodation Conflict (VAC). This optical architecture combines a Holographic Optical Element (HOE) waveguide, a Liquid Crystal over Silicon (LCoS) acting as a digital dynamic hologram, eye tracking, and Display Exit Pupil steering and switching. The waveguide expands the beam and acts as a front illuminator for the LCoS. So a small in-coupling element to the waveguide can illuminate the large area of the digital hologram. Several static multiplexed holograms were recorded within the in-coupling and out-coupling HOEs of the waveguide. The static in-coupling holograms did not contain any power, while the static out-coupling holograms had a lens function with around 35 mm focal distance. In-coupling and out-coupling static holograms were matched in pairs and met the Bragg condition at the same horizontal angle. The digital hologram, i.e., the LCoS, was illuminated with the converging illumination emerging from the static out-coupling hologram. The light was focused at a different position in the eye box for each static multiplexed hologram. Thus, by knowing the position of the user’s eye, the angle beam incident on the in-coupling element was adjusted, and the correct out-coupling element directed the light into the user’s eye pupil. Steering in the vertical direction was achieved by utilizing the “Bragg Degeneracy”1 of the static holograms.
How can we classify graph-structured data only with positive labels? Graph-based positive-unlabeled (PU) learning is to train a binary classifier given only the positive labels when the relationship between examples is given as a graph. The problem is of great importance for various tasks such as detecting malicious accounts in a social network, which are difficult to be modeled by supervised learning when the true negative labels are absent. Previous works for graph-based PU learning assume that the prior distribution of positive nodes is known in advance, which is not true in many real-world cases. In this work, we propose GRAB (Graph-based Risk minimization with iterAtive Belief propagation), a novel end-to-end approach for graph-based PU learning that requires no class prior. GRAB runs marginalization and update steps iteratively. The marginalization step models the given graph as a Markov network and estimates the marginals of latent variables. The update step trains the binary classifier by utilizing the computed marginals in the objective function. We then generalize GRAB to multi-positive unlabeled (MPU) learning, where multiple positive classes exist in a dataset. Extensive experiments on five real-world datasets show that GRAB achieves the state-of-the-art performance, even when the true prior is given only to the competitors.
What are the key structures existing in a large real-world MMORPG (Massively Multiplayer Online Role-Playing Game) graph? How can we compactly summarize an MMORPG graph with hierarchical node labels, considering substructures at different levels of hierarchy? Recent MMORPGs generate complex interactions between entities inducing a heterogeneous graph where each entity has hierarchical labels. Succinctly summarizing a heterogeneous MMORPG graph is crucial to better understand its structure; however it is a challenging task since it needs to handle complex interactions and hierarchical labels efficiently. Although there exist few methods to summarize a large-scale graph, they do not deal with heterogeneous graphs with hierarchical node labels.We propose GSHL , a novel method that summarizes a heterogeneous graph with hierarchical labels. We formulate the encoding cost of hierarchical labels using MDL (Minimum Description Length). GSHL exploits the formulation to identify and segment subgraphs, and discovers compact and consistent structures in the graph. Experiments on a large real-world MMORPG graph with multi-million edges show that GSHL is a useful and scalable tool for summarizing the graph, finding important structures in the graph, and finding similar users.
We present a novel near-eye display concept which consists of a waveguide combiner, a spatial light modulator, and a laser light source. The proposed system can display true 3D holographic images through see-through pupil-replicating waveguide combiner as well as providing a large eye-box. By modeling the coherent light interaction inside of the waveguide combiner, we demonstrate that the output wavefront from the waveguide can be controlled by modulating the wavefront of input light using a spatial light modulator. This new possibility allows combining a holographic display, which is considered as the ultimate 3D display technology, with the state-of-the-art pupil replicating waveguides, enabling the path towards true 3D holographic augmented reality glasses.
Holographic near-eye displays can deliver high-quality three-dimensional (3D) imagery with focus cues. However, the content resolution required to simultaneously support a wide field of view and a sufficiently large eyebox is enormous. The consequent data storage and streaming overheads pose a big challenge for practical virtual and augmented reality (VR/AR) applications. We present a deep-learning-based method for efficiently compressing complex-valued hologram images and videos. We demonstrate superior performance over the conventional image and video codecs.
How can we classify graph-structured data only with positive labels? Graph-based positive-unlabeled (PU) learning is to train a binary classifier given only the positive labels when the relationship between examples is given as a graph. The problem is of great importance for various tasks such as detecting malicious accounts in a social network, which are difficult to be modeled by supervised learning when the true negative labels are absent. Previous works for graph-based PU learning assume that the prior distribution of positive nodes is known in advance, which is not true in many real-world cases. In this work, we propose GRAB (Graph-based Risk minimization with iterAtive Belief propagation), a novel end-to-end approach for graph-based PU learning that requires no class prior. GRAB models a given graph as a Markov network and runs the marginalization and update steps iteratively. The marginalization step estimates the marginals of latent variables, while the update step trains a classifier network utilizing the computed priors in the objective function. Extensive experiments on five datasets show that GRAB achieves state-of-the-art accuracy, even compared with previous methods that are given the true prior.
Wolbachia is a genus of maternally transmitted bacteria having an endosymbiotic relationship with arthropod and nematode species. These bacteria manipulate host development, sex-determination, and reproduction. In addition, they are known to potentially suppress vector-borne diseases by interfering with pathogen transmission. Although the occurrence of Wolbachia infection in insects has been known, the underlying mechanisms that mediate their interactions remain unclear. To examine the influence of Wolbachia adaptation on the host, we infected wAlbA and wAlbB strain from Aedes albopictus into the C6/36 cell line derived from Ae. albopictus. The transient Wolbachia infection was characterized by induction of cell migration without cell proliferation. The production of nitrite and reactive oxygen species (ROS) was induced by transient Wolbachia infection. Cells with transient Wolbachia infection exhibited elevated expression of Toll-like receptor 6 (TLR6) and myeloid differentiation primary response 88 (Myd88). Conversely, the expression of TLR2, TLR4, TLR7, Cactus, Ankyrin, and Argonaute2 (AGO2) was inhibited upon Wolbachia infection. These results suggest that Wolbachia has an influence on the cell migration ability as well as host innate immune response in vitro. Considering these results, transient Wolbachia strain transfer in C6/36 cells might be an important approach for studying Wolbachia-host interactions and might help gain a deeper understanding of the early adaptation of Wolbachia in the original host insect.
Mosquitoes have adapted to various environmental conditions. Symbionts with mosquitoes impact this adaptation in different environments. In the field, mosquitoes could get exposed to antibiotics during their developmental period, which could reduce or eliminate their symbiotic microbes. However, the side effects of the antibiotics on the ovary and reproductive physiology of the Asian tiger mosquito,Aedes albopictusremains unknown. In this study, we investigated the effects of tetracycline and combinations of rifampicin and tetracycline at environmentally acceptable levels on the reproductive physiology of ovaries inAe. albopictus. Rifampicin and tetracycline in combination reduced the hatching rate and fertility ofAe. albopictuscompared to the untreated control group. These antibiotics induced histopathological damage and reactive oxygen species production in the ovaries. The combination of antibiotics decreased the expression of surface protein ofWolbachia(WSP) inAe. albopictus. Additionally, the expression of Toll like receptor 2 (TLR2) and Myd88 were triggered by the combinations. The findings demonstrate the detrimental effects of antibiotics, particularly combinations of rifampicin and tetracycline, on the reproductive capacity ofAe. albopictusfemales.
Abstract Foveated near-eye display is one of the most promising approaches to deliver immersive experience of mixed reality. However, it is challenged to conceive a compact optical system. Here, we introduce a method to use polarization optics via liquid crystal photonics to improve the foveated display performance. We demonstrate a benchtop prototype of this idea. We implement and combine two display modules for peripheral and foveal visions. A peripheral display consists of a polarization selective lens (PSL) module, a polarization selective diffuser (PSD), and a slanted projection system. An 80 $$^\circ$$ ∘ diagonal field of view is achieved by on-axis optical configuration of the PSL module and the PSD. A foveal holographic display is composed of a spatial light modulator (SLM), a volume grating lens, and a microelectromechanical system mirror possibly in combination with a switchable polarization selective grating module. The holographic reconstruction using the SLM enables accurate focus cue generation and high resolution above 30 cycles per degree within 15 $$^\circ$$ ∘ by 15 $$^\circ$$ ∘ field of view. We explore and discuss the liquid crystal photonics in the prototype that has a novel optical design using volume gratings with polarization selectivity.
Holographic optical elements (HOEs) have a wide range of applications, including their emerging use in virtual and augmented reality displays, but their design and fabrication have remained largely limited to configurations using simple wavefronts. In this paper, we present a pipeline for the design, optimization, and fabrication of complex, customized HOEs that enhances their imaging performance and enables new applications. In particular, we propose an optimization method for grating vector fields that accounts for the unique selectivity properties of HOEs. We further show how our pipeline can be applied to two distinct HOE fabrication methods. The first uses a pair of freeform refractive elements to manufacture HOEs with high optical quality and precision. The second uses a holographic printer with two wavefront-modulating arms, enabling rapid prototyping. We propose a unified wavefront decomposition framework suitable for both fabrication approaches. To demonstrate the versatility of these methods, we fabricate and characterize a series of specialized HOEs, including an aspheric lens, a head-up display lens, a lens array, and, for the first time, a full-color caustic projection element.
A compact noise filtering method for holographic head-mounted displays (HMDs) is proposed. Conventionally, twin, DC and high-order noise from the spatial light modulators is filtered by a spatial stop filter with a 4-f system. Since the 4-f system requires a long optical path length, the noise filtering system usually occupies most of the volume in holographic displays. In the proposed method, only thin angular stop filters (ASFs) are used for noise filtering. The ASFs do not require a bulky 4-f system while performing the same function as the conventional method. The proposed method is verified experimentally. Our study provides an important solution for realization of compact holographic HMDs.
Field collected populations of two mosquito species, Culex pipiens pallens and Aedes albopictus , were tested for susceptibility to five pyrethroid insecticides using a topical application bioassay. The laboratory strain of Cx . pipiens pallens (KS‐CP) demonstrated the most and the lowest susceptibility to deltamethrin and d ‐cis/trans prallethrine with LD 50 values of 0.00022 μg/♀ and 0.00178 μg/♀, respectively, while Ae . albopictus (KS‐AL) showed the greatest and the lowest susceptibility to phthalethrin and deltamethrin with LD 50 values of 0.00015 μg/♀ and 0.00085 μg/♀, respectively. Cx . pipiens pallens (JF‐CP) collected at a nearby rice field in Jeonju showed the greatest susceptibility to deltamethrin (LD 50 value of 0.0473 μg/♀) and the resistance ratios (RRs) and compared to the laboratory strain were 215 fold greater. JF‐CP demonstrated the lowest susceptibility to d ‐cis/trans phenothrin (0.7697 μg/♀) with RRs 711.0 fold greater than the KS‐CP. An Ae . albopictus (JF‐AL) collected at a nearby forest in Jeonju showed the highest susceptibility to deltamethrin (0.00203 μg/♀) with RRs 2.4 fold greater than the KS‐AL. JF‐AL demonstrated the lowest susceptibility to d ‐cis/trans prallethrine (LD 50 value 0.00646 μg/♀) with RRs 9.4 fold greater than the KS‐AL. JF‐AL demonstrated the greatest resistance to phthalethrin with RRs 39.3 fold greater than the KS‐AL. Resistance ratios of Cx . pipiens pallens between field and laboratory strains ranged from 215.0 to 711.0 and Ae . albopictus ranged from 2.4 to 39.3. These results indicate that the choice of effective insecticides to selected species of mosquito populations will result in more effective control against field strains of mosquito vectors and decrease environmental pollution.
Holographic optical elements (HOEs) have been used as important tools for implementing augmented reality (AR) and see-through displays because they are transparent and thin. Moreover, as HOEs usually come in the shape of a thin film, they can be bent, used for coating, or attached to curved surfaces. While they can be used to implement curved AR displays, however, the applications of the curved HOE have not been sufficiently studied. In this paper, an analysis method for curved/bent HOEs using the coupled-wave theory and the numerical ray tracing method is introduced. Using this method, the influence of the surface curvature on the optical characteristics of HOEs, including the aberration and diffraction efficiency, was analyzed. Also presented herein is a method of designing the optimal curvature that can reduce the aberration. Curved HOEs can be applied to see-through displays such as head-mounted displays (HMDs), head-up displays (HUDs), or transparent screens. They can be used to expand the field of view (FOV) and to reduce the form factor. The proposed analysis method provides a useful guideline for designing practical curved see-through displays.
Recently, augmented reality (AR) systems using holographic optical elements are getting the limelight. The holographic diffuser can be used as a screen in AR systems because it only diffuses incident light in a certain angle. To achieve a wide field-of-view (FOV), a diffusing angle of the screen should be sufficiently large. However, in the case of the holographic diffuser, it is difficult to achieve a high transmittance when the diffusing angle of the employed diffuser is large. In other words, by using conventional holographic screens, it is difficult to achieve a high transmittance and a wide FOV simultaneously. In this work, we propose an anisotropic holographic diffuser providing a wide FOV with relatively small diffusing angle by adopting a convex lens in the recording process. Furthermore, in the last section, a scheme of AR near-eye display system is suggested by using the proposed holographic screen and a special eye piece consisted of geometric phase lens.