Appears in: ICERI2022 Proceedings Publication year: 2022Pages: 2389-2394ISBN: 978-84-09-45476-1ISSN: 2340-1095doi: 10.21125/iceri.2022.0603Conference name: 15th annual International Conference of Education, Research and InnovationDates: 7-9 November, 2022Location: Seville, Spain

360-degree videos are consumed in diverse devices: some based in immersive interfaces, such as viewed through Virtual Reality headsets and some based in non-immersive interfaces, as in a computer with a pointing device or mobile devices with touchscreens. We have found, in prior work, significant differences in user behavior between these devices. From a dataset of the trajectories of the users’ head orientation in 775 video reproductions, we classify which kind of video was played (two values) and which of the four possible devices was used to reproduce these videos. We found that recurrent neural network models based on LSTM layers are able to classify the video type and the device used to play the video with an average accuracy of over 90% with only four seconds of trajectory. We are convinced that this knowledge can improve techniques to predict future viewports used in viewport-adaptive streaming when diverse devices are used.

360-degree videos have recently grown in popularity thanks to the popularization of virtual reality and its adoption by major online video streaming platforms. Being watched on a diverse array of interfaces (virtual reality headsets, computers, mobile devices, etc.), user behavior needs to be analyzed for all these devices. A 360-degree hypervideo production, a virtual tour around the streets of a city, is played in an experiment by 30 users with multiple devices through a web application which collects their behavior following the Experience API standard. The distribution of viewport centers when playing these 360-degree videos is analyzed and compared between device and video types, showing an evident difference in behavior when playing videos of different type.

One of the most recent trends in the evaluation of immersive virtual environments is the incorporation of user metrics. In this article, we conduct a user study on a 360° hypervideo, using a dashboard based on detailed metrics obtained from users’ interactions with 360° hypervideos. It is essential to evaluate the quality of experience to monitor service quality from the perspectives of consumers. We demonstrate a framework to examine the user experiences of 360° environments and evaluate them using the xAPI specification, facilitating the development of analytics solutions centered in the user experience; and how the graphs and related data composing the dashboard provide valuable information about ways of navigating and interacting with 360° video experiences, as well as the time invested in them. In the user study, we include the visual perception, attention, tracking and interaction of users watching Proemaid (a 360° multimedia production), collected from an interactive 360° video player in the form of xAPI statements. The Proemaid production has been played in a vast variety of contexts by stakeholders from government, technology and education, among others. Therefore, the quantitative results and the qualitative analysis of the user study are intended to outline a sketch of the users’ ways of navigating, interacting and investing time in 360° hypervideo productions. We consider that these metrics will be very interesting in the specification of new omnidirectional storyboards for film producers of content in 360°. Finally, we propose potential directions for empirical investigation that highlight its great potential in many fields.

Virtual reality (VR) applications have spread in the entertainment sector and are being introduced in other areas. Popularity of VR headsets as well as mobile headsets has created a lot of opportunities for VR applications, which can also be played in web pages thanks to HTML5’s WebVR. We propose a case study of a VR museum of Macintosh computers, which can be played using VR and non-VR interfaces as a web application. Participants are invited to explore the virtual world and interact with historic Apple Macintosh computers in an immersive, virtual world. The system has been implemented using A-Frame, a Mozilla-backed WebVR library that enabled us to create a multiplatform interface. User behavior is recorded in real time leveraging the experience API (xAPI) and may be analyzed both in real time and offline, so we can understand it better. This data, plus feedback received as part of the case study are discussed in the end of this article.

The arrival of 360° video to the everyday life creates the necessity of assessing both the audiovisual production and the playback environment offered to the final user. Leveraging the standard Experience API (xAPI), that considers collecting micro-interactions with e-learning content, we propose a platform to automatically collect the users' interaction with applications based on interactive 360° multimedia. To validate the platform, we introduce an example of educational activities based on interactive 360° videos and the tools used to first, annotate these videos and convert them into interactive activities; second, to perform said activity and collect the users' behavior via xAPI statements; and finally, to convert these statements to meaningful information in the form of user metrics and charts, both at individual level and also aggregated by activity, creating the possibility of finding singular and group behavior. This work concludes that the presented platform helps to analyze how users behave with omnidirectional interactive productions, with the aim of validating and improving its usability, ending with the discussion of future work ideas.

The theoretical calculation of Surface Site Interaction Points (SSIP) has been used successfully in some applications in the solid and liquid phase. In this work we propose a new set of optimizations for the search of SSIP using the Molecular Electrostatic Potential Surfaces (MEPS) calculated with Density Functional Theory and B3LYP/6‐31*G basis set. The measures that have been implemented are based on the search for the best agreement between experimental H‐bond donor and acceptor parameters ( α and β ) and the MEPS extremes exploring a range of electron density levels. Additionally, a parameterization as a function of atom types has been performed. The results show that the MEPS calculated at 0.01 au electron density level slightly improves the correlation with experimental data in comparison with the calculation over other density values. This fact is related to the bigger contribution of local electrostatics at higher density levels. The refinement has provided significant improvements to the correlation between theoretical and experimental data. Moreover, the proposed calculation over 0.01 au is six times faster on average than the computation at 0.002 au. The proposed methodology has been developed with the purpose to obtain high precision SSIP in a fast way and to improve their applications in virtual cocrystal screening, calculation of free energies in solution and molecular docking. © 2018 Wiley Periodicals, Inc.

360º video can consume up to six times the bandwidth of a regular video by delivering the entire frames instead of just the current viewport, introducing an additional difficulty to the delivery of this kind of multimedia. Many authors address this challenge by narrowing the delivered viewport using knowledge of where the user is likely to look at. To address this, we propose an automatic view tracking system based in xAPI to collect the data required to create the knowledge required to decide the viewport that is going to be delivered. We present a use case of an interactive 360º video documentary around the migratory crisis in Greece. In this case, omnidirectional content is recorded using a 6-camera array, rendered in an equirectangular projection and played later by an HTML5 web application. Interactive hotspots are placed on specific coordinates of the space-time of the production, introducing a connection of the viewer with the story by playing additional multimedia content. The current view and other usage data are recorded and permit us to obtain metrics on the user behavior, like most watched areas, with the goal to obtain that required knowledge.

Determining the position and magnitude of Surface Site Interaction Points (SSIP) is a useful technique for understanding intermolecular interactions. SSIPs have been used for the prediction of solvation properties and for virtual co‐crystal screening. To determine the SSIPs for a molecule, the Molecular Electrostatic Potential Surface (MEPS) is first calculated using ab initio methods such as Density Functional Theory. This leads to a high cost in terms of computation time and is not compatible with the analysis of huge molecular databases. Herein, we present a method for the fast estimation of SSIPs, which is based on the MEPS calculated from MMFF94 atomic partial charges. The results show that this method can be used to calculate SSIPs for large molecular databases with a much higher speed than the original ab initio methodology. © 2017 Wiley Periodicals, Inc.

Virtual screening (VS) is applied in the early drug discovery phases for the quick inspection of huge molecular databases to identify those compounds that most likely bind to a given drug target. In this context, there is the necessity of the use of compact molecular models for database screening and precise target prediction in reasonable times. In this work we present a new compact energy-based model that is tested for its application to Virtual Screening and target prediction. The model can be used to quickly identify active compounds in huge databases based on the estimation of the molecule's pairing energies. The greatest molecular polar regions along with its geometrical distribution are considered by using a short set of smart energy vectors. The model is tested using similarity searches within the Directory of Useful Decoys (DUD) database. The results obtained are considerably better than previously published models. As a Target prediction methodology we propose the use of a Bayesian Classifier that uses a combination of different active compounds to build an energy-dependent probability distribution function for each target.

In the recent years, consumption and usage of multimedia content has shifted to handheld devices, especially with the introduction of second screen applications. In this paper we address the situation of delivering multimedia information to users using modern and novel techniques to attract their attention in environments like fairs, parties or showrooms in an entertaining and educative way. Two different approaches are presented: an Augmented Reality application on mobile devices and a 360-degree video player remotely controlled by a handheld device. Both ideas focus on the use of mobile devices, as they have become a powerful tool always worn by its owners.

This paper presents a new methodology for the hardware implementation of neural networks (NNs) based on probabilistic laws. The proposed encoding scheme circumvents the limitations of classical stochastic computing (based on unipolar or bipolar encoding) extending the representation range to any real number using the ratio of two bipolar-encoded pulsed signals. Furthermore, the novel approach presents practically a total noise-immunity capability due to its specific codification. We introduce different designs for building the fundamental blocks needed to implement NNs. The validity of the present approach is demonstrated through a regression and a pattern recognition task. The low cost of the methodology in terms of hardware, along with its capacity to implement complex mathematical functions (such as the hyperbolic tangent), allows its use for building highly reliable systems and parallel computing.

The hardware implementation of neural network models allows to efficiently exploit their inherent parallelism. Here, we focus on the Liquid State Machine (LSM) methodology to build recurrent Spiking Neural Networks (SNN), particularly suited to process time-dependent signals. We propose a low cost hardware implementation of LSM networks based on the use of stochastic computing (SC) concepts. The functionality of the present approach is demonstrated for a time-series prediction task.

Spiking neural networks (SNN) are the last neural network generation that try to mimic the real behavior of biological neurons. Although most research in this area is done through software applications, it is in hardware implementations in which the intrinsic parallelism of these computing systems are more efficiently exploited. Liquid state machines (LSM) have arisen as a strategic technique to implement recurrent designs of SNN with a simple learning methodology. In this work, we show a new low-cost methodology to implement high-density LSM by using Boolean gates. The proposed method is based on the use of probabilistic computing concepts to reduce hardware requirements, thus considerably increasing the neuron count per chip. The result is a highly functional system that is applied to high-speed time series forecasting.

Efficient hardware implementations of neural networks are of high interest. Stochastic computing is an alternative to conventional digital logic that allows to exploit the intrinsic parallelism of neural networks using few hardware resources. We present a new stochastic methodology that extends the capabilities of classical stochastic computing. In particular, the present approach exhibits practically total immunity to noise. This is demonstrated evaluating the influence of the noise on the system's performance for a mathematical regression task.

In this work we review the basic principles of stochastic logic and propose its application to probabilistic-based pattern-recognition analysis. The proposed technique is the implementation of a parallel comparison of data with respect to various pre-stored categories. We design smart pulse-based stochastic-logic blocks to provide an efficient pattern recognition analysis. The proposed architecture can speed-up the screening process of huge databases by two orders of magnitude with respect classical software-based solutions, thus implying a great improvement in terms of total performance (speed and power dissipation).

Informal learning constitutes a large portion of our daily activities. With the development of digital technology easy access to useful information is only a touch or a click away. An innovative project TRAILER funded by European Commission proposed a new way of gathering information about informal learning. The goal was to enable a person to collect the data about its activities in informal learning and open this information to the public (formal institutions, friends through social media etc.). Informal learning can be done through various Web 2.0 platforms, mobile apps, personal learning environments and video games. One media that was unintentionally left out of the picture is Television (TV). For decades people are watching TV and are gaining useful knowledge. The progress in digital and smart television opened a new way of interaction with the television, where users are given the opportunity to interact with multimedia material. Also, internet protocol became an underlying protocol for streaming and managing television program. Today, the TV device has the ability of being an active participant on the Internet. People now have the opportunity to also share their learning experience created by watching a TV with the public. The goal of this paper is to extend the idea set in TRAILER project of collecting informal learning information to the very important media of television.

In this paper two improvements for the Hypervideo platform, used to represent augmented reality on Interactive TVs thanks to the hypervideo concept, are presented: the introduction of a second-screen application to the platform, enabling the user to obtain the additional information on its handheld device and delivering the video track through the broadcast channel, thanks to the HbbTV capability.

Minimal hardware implementations able to cope with the processing of large amounts of data in reasonable times are highly desired in our information-driven society. In this work we review the application of stochastic computing to probabilistic-based pattern-recognition analysis of huge database sets. The proposed technique consists in the hardware implementation of a parallel architecture implementing a similarity search of data with respect to different pre-stored categories. We design pulse-based stochastic-logic blocks to obtain an efficient pattern recognition system. The proposed architecture speeds up the screening process of huge databases by a factor of 7 when compared to a conventional digital implementation using the same hardware area.