Home insurance is a critical issue in the state of Florida, considering that residential properties are exposed to hurricane risk each year. To assess hurricane risk and project insured losses, the Florida Public Hurricane Loss Model (FPHLM) funded by the states insurance regulatory agency was developed. The FPHLM is an open and public model that offers an integrated complex computing framework that can be described in two phases: execution and validation. In the execution phase, all major components of FPHLM (i.e., data pre-processing, Wind Speed Correction (WSC), and Insurance Loss Model (ILM)) are seamlessly integrated and sequentially carried out by following a coordination workflow, where each component is modeled as an execution element governed by the centralized data-transfer element. In the validation phase, semantic rules provided by domain experts for individual component are applied to verify the validity of model output. This paper presents how the model efficiently incorporates the various components from multiple disciplines in an integrated execution framework to address the challenges that make the FPHLM unique.

Smart TVs allow consumers to watch TV, interact with applications, and access the Internet, thus enhancing the consumer experience. However, the consumers are still unable to seamlessly interact with the contents being streamed, as it is highlighted by TV-enabled shopping. For example, if a consumer is watching a TV show and is interested in purchasing a product being displayed, the consumer can only go to a store or access the Web to make the purchase. It would be more convenient if the consumer could interact with the TV to purchase interesting items. To realize this use case, products in the content stream must be detected so that the TV system notifies consumers of possibly interesting ones. A practical solution must address the detection of complex products, i.e, those that do not have a rigid form and can appear in various poses, which poses a significant challenge. To this end, a multi cue product detection framework is proposed for TV shopping. The framework is generic as it is not tied to specific object detection approaches. Instead, it utilizes appearance, topological, and spatio-temporal cues that make use of a related, easier to detect object class to improve the detection results of the target, more difficult product class. The three cues are jointly considered to select the best path that occurrences of the target product class can follow in the video and thus eliminate false positive occurrences. The empirical results demonstrate the advantages of the proposed approach in improving the precision of the results.

Smart TVs have realized the convergence of TV, Internet , and PC technologies, but still do not provide a seamless content interaction for TV-enabled shopping. To purchase interesting items displayed in a TV show, consumers must resort to a store or the Web, which is an inconvenient way of purchasing products. The fundamental challenge in realizing such a use case consists of understanding the multimedia content being streamed. Such a challenge can be realized by utilizing object detection to facilitate content understanding though it has to be executed as a computationally bound process so that consumers are provided with a responsive and exciting user interface. To this end, we propose a computational- and temporal-aware multimedia abstraction framework that facilitates the efficient execution of object detection tasks. Given computational and temporal rate constraints, the proposed framework selects the optimal video frames that best represent the video content and allows the execution of the object detection task as a computationally bound process. In this sense, the framework is computationally scalable as it can adapt to the given constraints and generate optimal abstraction results accordingly. Additionally, the framework utilizes “object views” as the basis for the frame selection process, which depict salient information and are represented as regions of interest (ROI). In general, an ROI can be a whole frame or a region that discards background information. Experimental results demonstrate the computational scalability of the proposed framework and the benefits of using the regions of interest as the basis of the abstraction process.

Semantic concept detection is among the most important and challenging topics in multimedia research. Its objective is to effectively identify high-level semantic concepts from low-level features for multimedia data analysis and management. In this paper, a novel re-ranking method is proposed based on correlation among concepts to automatically refine detection results and improve detection accuracy. Specifically, multiple correspondence analysis (MCA) is utilized to capture the relationship between a targeted concept and all other semantic concepts. Such relationship is then used as a transaction weight to refine detection ranking scores. To demonstrate its effectiveness in refining semantic concept detection, the proposed re-ranking method is applied to the detection scores of TRECVID 2011 benchmark data set, and its performance is compared with other state-of-the-art re-ranking approaches.

A multimedia semantic retrieval system based on hidden coherent feature groups (HCFGs) can support multimedia semantic retrieval on mobile applications. The system can capture the correlation between features and partition the original feature set into HCFGs, which have strong intragroup correlation while maintaining low intercorrelation. The authors present a novel, multimodel fusion scheme to effectively fuse the multimodel results and generate the final ranked retrieval results. In addition, to incorporate user interaction for effective retrieval, the proposed system also features a user feedback mechanism that helps refine the retrieval results.

The advent of smart TVs has reshaped the TV-consumer interaction by combining TVs with mobile-like applications and access to the Internet. However, consumers are still unable to seamlessly interact with the contents being streamed. An example of such limitation is TV shopping, in which a consumer makes a purchase of a product or item displayed in the current TV show. Currently, consumers can only stop the current show and attempt to find a similar item in the Web or an actual store. It would be more convenient if the consumer could interact with the TV to purchase interesting items.

Nevertheless, even though MCA has shown its eectiveness in CBMIR, its direct application to big data is not scalable. To train an MCA classi cation model, it is required to manipulate large matrices extractedfrom the training data, which impedes the useful utilization of MCA in today's pervasive big data environments. Existing works that utilize MCA for CBMIR tasks do not take into account scalability problems that arise when processing large amounts of data nor provide a framework that eectively utilizes multiple computers to speed up processing. e pertinent question is then how to improve the scalability of MCA and bring it onto the big data scale.

The booming multimedia technology is incurring a thriving multi-media data propagation. As multimedia data have become more essential, taking over a major potion of the content processed by many applications, it is important to leverage data mining methods to associate the low-level features extracted from multimedia data to high-level semantic concepts. In order to bridge the semantic gap, researchers have investigated the correlation among multiple modalities involved in multimedia data to effectively detect semantic concepts. It has been shown that multimodal fusion plays an important role in elevating the performance of both multimedia content-based retrieval and semantic concepts detection. In this paper, we propose a novel cluster-based ARC fusion method to thoroughly explore the correlation among multiple modalities and classification models. After combining features from multiple modalities, each classification model is built on one feature cluster, which is generated from our previous work FCC-MMF. The correlation between medoid of a feature cluster and a semantic concept is introduced to identify the capability of a classification model. It is further applied with the logistic regression method to refine ARC fusion method proposed in our previous work for semantic concept detection. Several experiments are conducted to compare the proposed method with other related works and the proposed method has outperform other works with higher Mean Average Precision (MAP).

Nowadays, only processing visual features is not enough for multimedia semantic retrieval due to the complexity of multimedia data, which usually involve a variety of modalities, e.g. graphics, text, speech, video, etc. It becomes crucial to fully utilize the correlation between each feature and the target concept, the feature correlation within modalities, and the feature correlation across modalities. In this paper, the authors propose a Feature Correlation Clustering-based Multi-Modality Fusion Framework (FCC-MMF) for multimedia semantic retrieval. Features from different modalities are combined into one feature set with the same representation via a normalization and discretization process. Within and across modalities, multiple correspondence analysis is utilized to obtain the correlation between feature-value pairs, which are then projected onto the two principal components. K-medoids algorithm, which is a widely used partitioned clustering algorithm, is selected to minimize the Euclidean distance within the resulted clusters and produce high intra-correlated feature-value pair clusters. Majority vote is applied to subsequently decide which cluster each feature belongs to. Once the feature clusters are formed, one classifier is built and trained for each cluster. The correlation and confidence of each classifier are considered while fusing the classification scores, and mean average precision is used to evaluate the final ranked classification scores. Finally, the proposed framework is applied on NUS-wide Lite data set to demonstrate the effectiveness in multimedia semantic retrieval.

This demo paper presents a system based on PostgreSQL and the AH-Tree that supports Content-Based Image Retrieval (CBIR) through similarity queries. The AH-Tree is a balanced, tree-based index structure that utilizes high-level semantic information to address the well-known problems of semantic gap and user perception subjectivity. The proposed system implements the AH-Tree inside PostgreSQL's kernel by internally modifying PostgreSQL's GiST access mechanism and thus provides a DBMS with a viable and efficient content-based multimedia retrieval functionality.

Homeowner insurance is a critical issue for Floridians because of the periodic threat hurricanes pose to Florida. Providing fairness into the rate-making policy process, the state of Florida has developed the Florida Public Hurricane Loss Model (FPHLM), an open, public hurricane risk model to assess the risk of wind damage to insured residential properties. For each input property portfolio, the FPHLM processes a large amount of data to provide expected losses over tens of thousand of years of simulation, for which computational efficiency is of paramount importance. This paper presents our work in integrating the atmospheric component into the FPHLM using MapReduce, which resulted in a highly efficient computing platform for generating stochastic hurricane events on a cluster of computers. The experimental results demonstrate the feasibility of utilizing MapReduce for risk modeling components.

In this paper, we propose a Correlation based Feature Analysis (CFA) and Multi-Modality Fusion (CFA-MMF) framework for multimedia semantic concept retrieval. The CFA method is able to reduce the feature space and capture the correlation between features, separating the feature set into different feature groups, called Hidden Coherent Feature Groups (HCFGs), based on Maximum Spanning Tree (MaxST) algorithm. A correlation matrix is built upon feature pair correlations, and then a MaxST is constructed based on the correlation matrix. By performing a graph cut procedure on the MaxST, a set of feature groups are obtained, where the intra-group correlation is maximized and the inter-group correlation is minimized. Finally, one classifier is trained for each of the feature groups, and the generated scores from different classifiers are fused for the final retrieval. The proposed framework is effective because it reduces the dimensionality of the feature space. The experimental results on the NUSWIDE-Lite data set demonstrate the effectiveness of the proposed CFA-MMF framework.

In this paper we propose a novel framework for object retrieval based on automatic foreground object extraction and multi-layer information integration. Specifically, user interested objects are firstly detected from unconstrained videos via a multimodal cues method, then an automatic object extraction algorithm based on Grab Cut is applied to separate foreground object from background. The object-level information is enhanced during the feature extraction layer by assigning different weights to foreground and background pixels respectively, and the spatial color and texture information is integrated during the similarity calculation layer. Experimental results on both benchmark data set and real-world data set demonstrate the effectiveness of the proposed framework.

This paper presents the framework and results of the team “Florida International University - University of Miami (FIU-UM)” in TRECVID 2012 Semantic Indexing (SIN) task [3] [13]. Four runs of the SIN results were submitted, and the summary of the four runs is as follows: • F A FIU-UM-1-brn 1: Fusion of the results generated from three models, corresponding to the rest of the three runs.

To be effective multimedia retrieval mechanisms, index methods must provide not only efficient access but also meaningful retrieval by addressing challenges in multimedia retrieval. This article presents the AH + -tree, a height-balanced, tree-based index structure that efficiently incorporates high-level affinity information to support Content-Based Image Retrieval (CBIR) through similarity queries. The incorporation of affinity information allows the AH + -tree to address the problems of semantic gap and user perception subjectivity inherent to multimedia retrieval. Based on the Affinity-Hybrid Tree (AH-Tree), the AH + -tree utilizes affinity information in a novel way to eliminate the I/O overhead of the AH-Tree while maintaining the same functionality and quality of results. We explain the structure of the AH + -tree and implement and analyze algorithms for tree construction and similarity queries (range and nearest neighbor). Experimental results demonstrate the superior I/O efficiency of the AH + -tree over that of the AH-Tree and the M-tree without a detrimental impact on real-time costs of the retrieval process.

This paper presents a summary of the team “Florida International University - University of Miami (FIU-UM)” in TRECVID 2011 tasks [1]. This year, the FIU-UM team participated in the Semantic Indexing (SIN) and Instance Search (INS) tasks. Four runs of the SIN results were submitted. F A FIU-UM-1 1: KF+Meta&Relation+Audio+SPCPE&SIFT - Fuse the results from Subspace Modeling and Ranking (SMR) using the Key Frame-based low-level features (KF), LibSVM classification using Metadata from those meta-xmlfiles associated with the IACC videos as well as the relationships between semantic concepts (Meta&Relation), Gaussian Mixture Models (GMM) using the Mel-frequency cepstral coefficients (MFCC) audio features, and the simultaneous partition and class parameter estimation (SPCPE) algorithm with scale-invariant feature transform (SIFT) interesting points matching (SPCPE&SIFT). F A FIU-UM-2 2: KF+Meta&Relation - Fuse the results from SMR using KF and LibSVM using meta information and relationships between semantic concepts. F A FIU-UM-3 3: KF+Audio+SPCPE&SIFT - Fuse the results from SMR using KF, GMM using MFCC audio features as well as SPCPE&SIFT matching.

This paper presents the AH+-tree, a balanced, tree-based index structure that efficiently supports Content-Based Image Retrieval (CBIR) through similarity queries. The proposed index structure addresses the problems of semantic gap and user subjectivity by considering the high-level semantics of multimedia data during the retrieval process. The AH+-tree provides the same functionality as the Affinity-Hybrid Tree (AH-Tree) but utilizes the high-level semantics in a novel way to eliminate the I/O overhead incurred by the AH-Tree due to the process of affinity propagation, which requires a complete traversal of the tree. The novel structure of the tree is explained, and detailed range and nearest neighbor algorithms are implemented and analyzed. Extensive discussions and experiments demonstrate the superior efficiency of the AH+-tree over the AH-Tree and the M-tree. Results show the AH+-tree significantly reduces I/O cost during similarity searches. The I/O efficiency of the AH+-tree and its ability to incorporate high-level semantics from different machine learning mechanisms make the AH+-tree a promising index access method for large multimedia databases.