Recent years have seen increasing popularity of storing and managing personal multimedia data using online services. Preserving confidentiality of online personal data while offering efficient functionalities thus becomes an important and pressing research issue. In this paper, we study the problem of content-based search of image data archived online while preserving content confidentiality. The problem has different settings from those typically considered in the secure computation literature, as it deals with data in rank-ordered search, and has a different security-efficiency requirement. Secure computation techniques, such as homomorphic encryption, can potentially be used in this application, at a cost of high computational and communication complexity. Alternatively, efficient techniques based on randomizing visual feature and search indexes have been proposed recently to enable similarity comparison between encrypted images. This paper focuses on comparing these two major paradigms of techniques, namely, homomorphic encryption-based techniques and feature/index randomization-based techniques, for confidentiality-preserving image search. We develop novel and systematic metrics to quantitatively evaluate security strength in this unique type of data and applications. We compare these two paradigms of techniques in terms of their search performance, security strength, and computational efficiency. The insights obtained through this paper and comparison will help design practical algorithms appropriate for privacy-aware cloud multimedia systems.

Recent years have witnessed tremendous growth in the generation and consumption of digital images. Monitoring and evaluating image quality is an important issue for online and mobile media applications. Conventional quality assessment work mostly focus on intensity level distortion caused by operations that do not change image size. In this work, we study the problem of quality assessment for images having undergone content-adaptive resizing, also known as retargeting operations. We design a reduced-reference algorithm to analyze the structural distortion caused by retargeting and propose a quality score that achieves positive correlation with human observations. The quality score and distortion analysis from the proposed algorithm provide rich information for both objective and subjective quality assessment tasks.

Scale-Invariant Feature Transform (SIFT) is one of the most popular local image features that are widely used in computer vision, image processing and image retrieval. In this paper we study the relation between the SIFT descriptor and its matching accuracy. We propose a method to quantitatively assess the quality of a SIFT feature descriptor in terms of robustness and discriminability. This would enable us to gain a better understanding of the strength and limitations of SIFT in emerging applications of SIFT-based image hash, and also to improve matching accuracy and efficiency in applications such as object search. The experimental results demonstrate the effectiveness of the proposed method.

Secure signal processing is an emerging technology to enable signal processing tasks in a secure and privacy-preserving fashion. It has attracted a great amount of research attention due to the increasing demand to enable rich functionalities for private data stored online. Desirable functionalities may include search, analysis, clustering, etc. In this paper, we discuss the research issues and challenges in secure video processing with focus on the application of secure online video management. Video is different from text due to its large data volume and rich content diversity. To be practical, secure video processing requires efficient solutions that may involve a trade-off between security and complexity. We look at three representative video processing tasks and review existing techniques that can be applied. Many of the tasks do not have efficient solutions yet, and we discuss the challenges and research questions that need to be addressed.

Seam carving is an adaptive multimedia retargeting technique to resize multimedia data for different display sizes. This technique has found promising applications in media consumption on mobile devices such as tablets and smartphones. However, seam carving can also be used to maliciously alter image content and when combined with other tampering operations, makes tampering detection very difficult by traditional multimedia forensic techniques. In this paper, we study the problem of seam carving estimation and tampering localization using very compact side information called forensic hash. The forensic hash technique bridges two related areas, namely robust image hashing and blind multimedia forensics, to answer a broader scope of forensic questions in a more efficient and accurate manner. We show that our recently proposed forensic hash construction can be extended to accurately estimate seam carving and detect local tampering.

Technology advancements in areas of mobile computing, social networks, and cloud computing have rapidly changed the way we communicate and interact. The wide adoption of media-oriented mobile devices such as smartphones and tablets enables people to capture information in various media formats, and offers them a rich platform for media consumption. The proliferation of online services and social networks makes it possible to store personal multimedia collection online and share them with family and friends anytime anywhere. Considering the increasing impact of digital multimedia and the trend of cloud computing, this dissertation explores the problem of how to evaluate trustworthiness and preserve confidentiality of online multimedia data.The dissertation consists of two parts. The first part examines the problem of evaluating trustworthiness of multimedia data distributed online. Given the digital nature of multimedia data, editing and tampering of the multimedia content becomes very easy. Therefore, it is important to analyze and reveal the processing history of a multimedia document in order to evaluate its trustworthiness. We propose a new forensic technique called "Forensic Hash", which draws synergy between two related research areas of image hashing and non-reference multimedia forensics. A forensic hash is a compact signature capturing important information from the original multimedia document to assist forensic analysis and reveal processing history of a multimedia document under question. Our proposed technique is shown to have the advantage of being compact and offering efficient and accurate analysis to forensic questions that cannot be easily answered by convention forensic techniques. The answers that we obtain from the forensic hash provide valuable information on the trustworthiness of online multimedia data.The second part of this dissertation addresses the confidentiality issue of multimedia data stored with online services. The emerging cloud computing paradigm makes it attractive to store private multimedia data online for easy access and sharing. However, the potential of cloud services cannot be fully reached unless the issue of how to preserve confidentiality of sensitive data stored in the cloud is addressed. In this dissertation, we explore techniques that enable confidentiality-preserving search of encrypted multimedia, which can play a critical role in secure online multimedia services. Techniques from image processing, information retrieval, and cryptography are jointly and strategically applied to allow efficient rank-ordered search over encrypted multimedia database and at the same time preserve data confidentiality against malicious intruders and service providers. We demonstrate high efficiency and accuracy of the proposed techniques and provide a quantitative comparative study with conventional techniques based on heavy-weight cryptography primitives.

With the increasing popularity of digital multimedia such as images and videos and the advent of the cloud computing paradigm, a fast growing amount of private and sensitive multimedia data are being stored and managed over the network cloud. To provide enhanced security and privacy protection beyond traditional access control techniques, privacy preserving multimedia retrieval techniques have been proposed recently to allow content-based multimedia retrieval directly over encrypted databases and achieve accurate retrieval comparable to conventional retrieval schemes. In this paper, we introduce a security definition for the privacy preserving retrieval scenario and show that the recently proposed schemes are secure under the proposed security definition.

Digital multimedia such as images and videos are prevalent on today's internet and cause significant social impact, which can be evidenced by the proliferation of social networking sites with user generated contents. Due to the ease of generating and modifying images and videos, it is critical to establish trustworthiness for online multimedia information. In this paper, we propose novel approaches to perform multimedia forensics using compact side information to reconstruct the processing history of a document. We refer to this as FASHION, standing for Forensic hASH for informatION assurance. Based on the Radon transform and scale space theory, the proposed forensic hash is compact and can effectively estimate the parameters of geometric transforms and detect local tampering that an image may have undergone. Forensic hash is designed to answer a broader range of questions regarding the processing history of multimedia data than the simple binary decision from traditional robust image hashing, and also offers more efficient and accurate forensic analysis than multimedia forensic techniques that do not use any side information.

In recent years, digital images and videos have become increasingly popular over the internet and bring great social impact to a wide audience. In the meanwhile, technology advancement allows people to easily alter the content of digital multimedia and brings serious concern on the trustworthiness of online multimedia information. Forensic hash is a short signature attached to an image before transmission and acts as side information for analyzing the processing history and trustworthiness of the received image. In this paper, we propose a new construction of forensic hash based on visual words representation. We encode SIFT features into a compact visual words representation for robust estimation of geometric transformations and propose a hybrid construction using both SIFT and block-based features to detect and localize image tampering. The proposed hash construction achieves more robust and accurate forensic analysis than prior work.

The advancement of information technology is rapidly integrating the physical world where we live and the online world where we retrieve and share information. One immediate example of such integration is the increasing popularity of storing and managing personal data using third-party web services, as part of the emerging trend of cloud computing. Secure management of sensitive data stored online is becoming one of the critical research issues in cloud computing and online privacy protection. In this paper, we propose techniques to achieve content based multimedia retrieval over encrypted databases, which can be used for online management of multimedia data while preserving data privacy. We propose two types of secure retrieval schemes by combining cryptographic techniques, such as order preserving encryption and randomized hash functions, with image processing and information retrieval techniques, such as visual words representation, inverted index, and min-hash. The first type of retrieval schemes scramble visual features extracted from images and allow similarity comparison of the features in their encrypted forms. The second type of schemes encrypt the state-of-the-art search indexes without significantly affecting their search capability. The two types of schemes are complementary and represent different tradeoffs between user-side computational complexity and communication overhead. Retrieval results on an encrypted color image database and security analysis under different attack models show that retrieval performance comparable to conventional plaintext retrieval schemes can be achieved over encrypted databases while ensuring data confidentiality.

Performing information retrieval tasks while preserving data confidentiality is a desirable capability when a database is stored on a server maintained by a third-party service provider. This paper addresses the problem of enabling content-based retrieval over encrypted multimedia databases. Search indexes, along with multimedia documents, are first encrypted by the content owner and then stored onto the server. Through jointly applying cryptographic techniques, such as order preserving encryption and randomized hash functions, with image processing and information retrieval techniques, secure indexing schemes are designed to provide both privacy protection and rank-ordered search capability. Retrieval results on an encrypted color image database and security analysis of the secure indexing schemes under different attack models show that data confidentiality can be preserved while retaining very good retrieval performance. This work has promising applications in secure multimedia management.

This paper addresses the problem of image retrieval from an encrypted database, where data confidentiality is preserved both in the storage and retrieval process. The paper focuses on image feature protection techniques which enable similarity comparison among protected features. By utilizing both signal processing and cryptographic techniques, three schemes are investigated and compared, including bit-plane randomization, random projection, and randomized unary encoding. Experimental results show that secure image retrieval can achieve comparable retrieval performance to conventional image retrieval techniques without revealing information about image content. This work enriches the area of secure information retrieval and can find applications in secure online services for images and videos.