Public outdoor surveillance cameras often have limited metadata describing their properties. Frequently, a public camera’s precise position, orientation, focal length, and image center are unknown; these attributes are necessary to precisely pinpoint the location of events seen in the camera. In this article, we ask: what is the minimal information needed to accurately estimate these properties for public cameras? We show, using a judicious combination of projective geometry, neural networks, and crowd-sourced annotations from human workers, that it is possible to, for example, localize 95% of the cameras in our test data set to within 12 m using a single image taken from the camera. This performance is an order of magnitude better than PoseNet, a state-of-the-art neural network that needs significantly more information than our approach, and can only estimate position and orientation (and not other properties). Finally, we show that the camera’s inferred pose and properties can help design a number of virtual sensors, all of which have good accuracy.

Autonomous vehicles use 3D sensors for perception. Cooperative perception enables vehicles to share sensor readings with each other to improve safety. Prior work in cooperative perception scales poorly even with infrastructure support. AutoCast enables scalable infrastructure-less cooperative perception using direct vehicle-to-vehicle communication. It carefully determines which objects to share based on positional relationships between traffic participants, and the time evolution of their trajectories. It coordinates vehicles and optimally schedules transmissions in a distributed fashion. Extensive evaluation results under different scenarios show that, unlike competing approaches, AutoCast can avoid crashes and near-misses which occur frequently without cooperative perception, its performance scales gracefully in dense traffic scenarios providing 2-4x visibility into safety critical objects compared to existing cooperative perception schemes, its transmission schedules can be completed on the real radio testbed, and its scheduling algorithm is near-optimal with negligible computation overhead.

With the rise of service computing, the increasing number and diversity of web services make it an intractable task to search for suitable services. Service composition, service selection and recommendation have become the focus of service computing. As the fundamental research of service network, service link prediction is used to explore the composition mode between services, which can facilitate the development of service composition, service selection and recommendation. However, the existing link prediction methods are mainly based on manual modeling and derivation, which cannot make full use of the global structure information and perform poorly in complex networks. The challenging problem in service link prediction is the heterogeneity and sparseness of the service network. Therefore, we propose a novel web service link prediction method based on a heterogeneous graph attention network. By analyzing the interaction between services, five types of neighbors that are associated with service links are chosen, and two levels of attention are applied to learn the importance of neighbors and calculate the embedding of services. In addition, in order to improve accuracy, we design a Service-TextRank algorithm to extract the key information of the service description. Extensive experimental results on real-world data-ProgrammableWeb validate the effectiveness of our approach.

Video cameras are among the most ubiquitous sensors in the Internet-of-Things. Video and audio applications, such as cross-camera activity detection, avatar extraction or language translation will, in the future, offload processing to an edge cluster of GPUs. Rim is a management system for such clusters that satisfies throughput and latency requirements of these applications, while enabling high cluster utilization. It uses coarse-grained knowledge of application structure to profile throughput of applications on resources, then uses these profiles to place applications on cluster nodes to achieve these goals. It dynamically adapts placement to load and failures. Experiments show that on maximal workloads on a testbed, Rim can satisfy requirements of all applications, but competing approaches designed for low-latency GPU execution cannot.

As cloud computing technology turning to mature, cloud services have become a trust-based service. Users’ distrust of the security and performance of cloud services will hinder the rapid deployment and development of cloud services. So cloud service providers (CSPs) urgently need a way to prove that the infrastructure and the behavior of cloud services they provided can be trusted. The challenge here is how to construct a novel framework that can effective verify the security conformance of cloud services, which focuses on fine-grained descriptions of cloud service behavior and security service level aggreements (SLAs). In this paper, we propose a novel approach to verify cloud service security conformance, which reduces the description gap between the CSP and users through modeling cloud service behavior and security SLA, these models enable a systematic integration of security constraints and service behavior into cloud while using UPPAAL to check the performance and security conformance. The proposed approach is validated through case study and experimentswith real cloud service based on Open- Stack, which illustrates CloudSec approach effectiveness and can be applied on realistic cloud scenario.

Malignant propagation events in networks, such as large-scale diffusion of computer viruses, rumors and failures, have caused massive damage to our society. Thus, it is critical to study how to identify the propagation source. However, existing source identification algorithms only quantify the impact mechanisms of part of the factors that affect the Maximum Likelihood Estimator (MLE) of propagation source, which result in reduced source identification accuracy. In this paper, through constructing a mathematical model for propagation process, we derive two node properties, called Average Eccentricity and Infection Force, which quantify the impact mechanisms of all the factors that affect the MLE of propagation source. And then, we design an AEIF source identification algorithm based on the above two node properties, which make AEIF algorithm has improved accuracy and lower time complexity than existing algorithm. Finally, in the experimental part, extensive simulations on various synthetic networks and real-world networks demonstrate the outperformance of AEIF algorithm than existing algorithms, and based on the experimental results, some assignment suggestions of parameters in AEIF algorithm are given.

Cashier-free shopping systems like Amazon Go improve shopping experience, but can require significant store redesign. In this paper, we propose Grab, a practical system that leverages existing infrastructure and devices to enable cashier-free shopping. Grab needs to accurately identify and track customers, and associate each shopper with items he or she retrieves from shelves. To do this, it uses a keypoint-based pose tracker as a building block for identification and tracking, develops robust feature-based face trackers, and algorithms for associating and tracking arm movements. It also uses a probabilistic framework to fuse readings from camera, weight and RFID sensors in order to accurately assess which shopper picks up which item. In experiments from a pilot deployment in a retail store, Grab can achieve over 90% precision and recall even when 40% of shopping actions are designed to confuse the system. Moreover, Grab has optimizations that help reduce investment in computing infrastructure four-fold.

With the rapid development of technologies in the field of service computing, and increasing of complex business requirements, more and more large-scale service ecosystem emerges. Thus, many researches of service ecosystem focus on issues related to optimization such as service recommendation and load balancing, so the API popularity prediction problem studied in this paper, which is basis for this service ecosystem optimization, becomes a research hotspot in this field. However, many existing researches are predicting the popularity of APIs based on API functions, QoS, history usage patterns and social relationships, which are difficult to obtain and cannot reflect the overall structure of the underlying service ecosystem. Therefore, we propose an innovative API popularity prediction method in service ecosystem based on Graph Neural Network (GNN). Concretely, a Global-Service Ecosystem Network (GSEN) model is proposed firstly, for modeling a given service ecosystem to a network that can depict the complex structure of service ecosystem and the functions, QoS, history usage patterns and social relationships of APIs. Then, a Graph Heterogeneous Spatiotemporal Convolutional Network (GHSCN) model is proposed to predict the popularity of APIs based on GSEN, and for getting better prediction accuracy, four different Heterogeneous Spatiotemporal Convolutional Kernels are proposed to extract the features of different elements which have different mechanisms to affect the popularity of target API. Finally, extensive experiments based on the data crawled from ProgrammableWeb.com show that our method achieves a superior performance in API popularity prediction, and the importance of the introduction of our model to service ecosystems.

Context awareness is an essential part of mobile and ubiquitous computing. Its goal is to unveil situational information about mobile users like locations and activities. The sensed context can enable many services like navigation, AR, and smarting shopping. Such context can be sensed in different ways including visual sensors. There is an emergence of vision sources deployed worldwide. The cameras could be installed on roadside, in-house, and on mobile platforms. This trend provides huge amount of vision data that could be used for context sensing. However, the vision data collection and analytics are still highly manual today. It is hard to deploy cameras at large scale for data collection. Organizing and labeling context from the data are also labor intensive. In recent years, advanced vision algorithms and deep neural networks are used to help analyze vision data. But this approach is limited by data quality, labeling effort, and dependency on hardware resources. In summary, there are three major challenges for today's vision-based context sensing systems: data collection and labeling at large scale, process large data volumes efficiently with limited hardware resources, and extract accurate context out of vision data. The thesis explores the design space that consists of three dimensions: sensing task, sensor types, and task locations. Our prior work explores several points in this design space. We make contributions by (1) developing efficient and scalable solutions for different points in the design space of vision-based sensing tasks; (2) achieving state-of-the-art accuracy in those applications; (3) and developing guidelines for designing such sensing systems.

Public cameras often have limited metadata describing their attributes. A key missing attribute is the precise location of the camera, using which it is possible to precisely pinpoint the location of events seen in the camera. In this paper, we explore the following question: under what conditions is it possible to estimate the location of a camera from a single image taken by the camera? We show that, using a judicious combination of projective geometry, neural networks, and crowd-sourced annotations from human workers, it is possible to position 95% of the images in our test data set to within 12 m. This performance is two orders of magnitude better than PoseNet, a state-of-the-art neural network that, when trained on a large corpus of images in an area, can estimate the pose of a single image. Finally, we show that the camera's inferred position and intrinsic parameters can help design a number of virtual sensors, all of which are reasonably accurate.

It is of significance to identify the source of malicious information in social networks, since this information diffusion is already a problem, which can seriously affect social stability. In this paper, we develop a propagation path based approach where the estimator of information source is chosen to be the root node associated with the propagation path that most likely leads to the monitored state of network. When the information diffusion process follows the Susceptible-Infected (SI) model and satisfying the instant forwarding hypothesis, we proved that the source estimator we proposed is the root node of the network shortest arborescence. Finally, multiple simulations on networks with different structure show that our method outperforms existing algorithms.

Detecting activities from video taken with a single camera is an active research area for ML-based machine vision. In this paper, we examine the next research frontier: near real-time detection of complex activities spanning multiple (possibly wireless) cameras, a capability applicable to surveillance tasks. We argue that a system for such complex activity detection must employ a hybrid design: one in which rule-based activity detection must complement neural network based detection. Moreover, to be practical, such a system must scale well to multiple cameras and have low end-to-end latency. Caesar, our edge computing based system for complex activity detection, provides an extensible vocabulary of activities to allow users to specify complex actions in terms of spatial and temporal relationships between actors, objects, and activities. Caesar converts these specifications to graphs, efficiently monitors camera feeds, partitions processing between cameras and the edge cluster, retrieves minimal information from cameras, carefully schedules neural network invocation, and efficiently matches specification graphs to the underlying data in order to detect complex activities. Our evaluations show that Caesar can reduce wireless bandwidth, on-board camera memory, and detection latency by an order of magnitude while achieving good precision and recall for all complex activities on a public multi-camera dataset.

In the future, the video-enabled camera will be the most pervasive type of sensor in the Internet of Things. Such cameras will enable continuous surveillance through heterogeneous camera networks consisting of fixed camera systems as well as cameras on mobile devices. The challenge in these networks is to enable efficient video analytics: the ability to process videos cheaply and quickly to enable searching for specific events or sequences of events. In this paper, we discuss the design and implementation of Kestrel, a video analytics system that tracks the path of vehicles across a heterogeneous camera network. In Kestrel, fixed camera feeds are processed on the cloud, and mobile devices are invoked only to resolve ambiguities in vehicle tracks. Kestrel's mobile device pipeline detects objects using a deep neural network, extracts attributes using cheap visual features, and resolves path ambiguities by careful association of vehicle visual descriptors, while using several optimizations to conserve energy and reduce latency. Our evaluations show that Kestrel can achieve precision and recall comparable to a fixed camera network of the same size and topology, while reducing energy usage on mobile devices by more than an order of magnitude.

Disaster and emergency response operations require rapid situational assessment of the affected area for timely and efficient rescue operations. A 3D map, collected after a disaster, can provide such awareness, but constructing this map quickly is a significant challenge. In this paper, we explore the design of a capability called QuickSketch that rapidly builds 3D representations of an unknown environment using crowdsourcing. QuickSketch employs multiple vehicles equipped with 3D sensors (stereo cameras) to explore different areas of an unknown territory and then combines 3D data from all the vehicles to build a single 3D map. QuickSketch annotates the 3D map with important landmarks and enables rapid contextualization of visual intelligence (photos) received from first responders and disaster victims to guarantee timely backup and rescue operations. Our evaluation results show that QuickSketch can stitch a 3D map for a large campus with sub-meter mapping accuracy under certain conditions, position landmarks an order of magnitude more accurately than other image matching techniques, and contextualize visual intelligence accurately.

Accurate positioning in urban areas is important for personal navigation, geolocation apps, and ride-sharing. Smartphones localize themselves using GPS position estimates, and augment these with a variety of techniques including dead reckoning, map matching, and WiFi localization. However, GPS signals suffer significant impairment in urban canyons because of limited line-of-sight to satellites and signal reflections. In this paper, we focus on scalable and deployable techniques to reduce the impact of one specific impairment: reflected GPS signals from non-line-of-sight (NLOS) satellites. Specifically, we show how, using publicly available street-level imagery and off-the-shelf computer vision techniques, we can estimate the path inflation incurred by (the extra distance traveled by) a reflected signal from a satellite. Using these path inflation estimates we develop techniques to estimate the most likely actual position given a set of satellite readings at some position. Finally, we develop optimizations for fast position estimation on modern smartphones. Using extensive experiments in the downtown area of several large cities, we find that our techniques can reduce positioning error by up to 55% on average.

Mobile advertisements influence customers' in-store purchases and boost in-store sales for brick-and-mortar retailers. Targeting mobile ads has become significantly important to compete with online shopping. The key to enabling targeted mobile advertisement and service is to learn shoppers' interest during their stay in the store. Precise shopper tracking and identification are essential to gain the insights. However, existing sensor-based or vision-based solutions are neither practical nor accurate; no commercial solutions today can be readily deployed in a large store. On the other hand, we recognize that most retail stores have the installation of surveillance cameras, and most shoppers carry Bluetooth-enabled smartphones. Thus, in this paper, we propose TAR to learn shoppers' in-store interest via accurate multi-camera people tracking and identification. TAR leverages widespread camera deployment and Bluetooth proximity information to accurately track and identify shoppers in the store. TAR is composed of four novel design components: (1) a deep neural network (DNN) based visual tracking, (2) a user trajectory estimation by using shopper visual and BLE proximity trace, (3) an identity matching and assignment to recognize shopper's identity, and (4) a cross-camera calibration algorithm. TAR carefully combines these components to track and identify shoppers in real-time. TAR achieves 90% accuracy in two different real-life deployments, which is 20% better than the state-of-the-art solution.

We propose Secure Pick Up (SPU), a convenient, lightweight, in-device, non-intrusive and automatic-learning system for smartphone user authentication. Operating in the background, our system implicitly observes users' phone pick-up movements, the way they bend their arms when they pick up a smartphone to interact with the device, to authenticate the users. Our SPU outperforms the state-of-the-art implicit authentication mechanisms in three main aspects: 1) SPU automatically learns the user's behavioral pattern without requiring a large amount of training data (especially those of other users) as previous methods did, making it more deployable. Towards this end, we propose a weighted multi-dimensional Dynamic Time Warping (DTW) algorithm to effectively quantify similarities between users' pick-up movements; 2) SPU does not rely on a remote server for providing further computational power, making SPU efficient and usable even without network access; and 3) our system can adaptively update a user's authentication model to accommodate user's behavioral drift over time with negligible overhead. Through extensive experiments on real world datasets, we demonstrate that SPU can achieve authentication accuracy up to 96.3% with a very low latency of 2.4 milliseconds. It reduces the number of times a user has to do explicit authentication by 32.9%, while effectively defending against various attacks.

Due to the critical security threats imposed by social worms, such as Twitter and Facebook, modeling and simulation study of the propagation dynamics of social worms is essential to predict their potential for damage and to understand the propagation characteristics. Modern social worms exhibit one new feature, reinfection-notification. It indicates that malicious messages are sent by neighbors whenever any susceptible or infected recipients open the malicious attachments, and a user will get reminders when new messages come to a social account. Meanwhile, there are two critical problems: dynamic host usage and temporal message processing. First, from a spatial perspective, previous models have not taken into account that public hosts in different locations are shared by several users arising from human mobility, namely, dynamic host usage. Second, from a temporal perspective, the problem of temporal message processing results from the improper assumption that during this period of message checking, unread messages all are read by default, or under the condition that some parts of messages have been read, but the rest of unread messages are no longer processed afterward. To address these problems, we present a novel social worm simulation model in this paper, which adopts "social network-based sharing" and "sorting and attenuation" methods. We perform comprehensive theoretical analyses and experimental evaluation to validate our simulation model. The results show that our model is more suitable for modeling the complicated propagation behaviors of modern social worms in hierarchical networks.

In the process of big data analysis and processing, a key concern blocking users from storing and processing their data in the cloud is their misgivings about the security and performance of cloud services. There is an urgent need to develop an approach that can help each cloud service provider (CSP) to demonstrate that their infrastructure and service behavior can meet the users' expectations. However, most of the prior research work focused on validating the process compliance of cloud service without an accurate description of the basic service behaviors, and could not measure the security capability. In this paper, we propose a novel approach to verify cloud service security conformance called CloudSec, which reduces the description gap between the cloud provider and customer through modeling cloud service behaviors (CloudBeh Model) and security SLA (SecSLA Model). These models enable a systematic integration of security constraints and service behavior into cloud while using UPPAAL to check the conformance, which can not only check CloudBeh performance metrics conformance, but also verify whether the security constraints meet the SecSLA. The proposed approach is validated through case study and experiments with a cloud storage service based on OpenStack, which illustrates CloudSec approach effectiveness and can be applied in real cloud scenarios.

Context awareness is crucial for ubiquitous computing, and position is an important aspect of context. In an ideal world, every stationary object or entity in the built environment would be associated with position, so that applications can have precise spatial context about the environment surrounding a human. In this paper, we take a step towards this ideal: by analyzing images from Google Street View that cover different perspectives of a given object and triangulating the location of the object, our system, ALPS, can discover and localize common landmarks at the scale of a city accurately and with high coverage. ALPS contains several novel techniques that help improve the accuracy, coverage, and scalability of localization. Evaluations of ALPS on many cities in the United States show that it can localize storefronts with a coverage higher than 90% and a median error of 5 meters.