ContextController is a content augmentation system that enables the real time collation and presentation of contextual information for linear TV broadcasts. The goal of this project is to enable viewers to supplement their TV viewing experience with externally sourced and summarized knowledge. Through this viewers with varied background knowledge are provided the ability to discover new contextual information, which aids in the improved understanding of the original content. Although this system can be used with a variety of content types this work focuses on News related broadcasts.

The consumption of video in context of social interactions is becoming reality, as borders between television, video, microblogging and social networking are disappearing. This paper discusses the problem of quality of experience and synchronized video viewing across heterogeneous device and networks ecosystem. The proposed solution, which is still work in progress, builds upon network coding and traffic engineering to achieve maximum goodput across devices and networks, while preserving fairness between flows and meeting quality of experience requirements. The proposed approach has been designed for both real time and non real time content. We present preliminary results for the non-real case, in which the viewers use essentially FTP-like downloads to view and exchange the content.

The purpose of this work is to provide a method for exploiting pervasive wireless communication capabilities that are often underutilized on smart devices (e.g., phones, tables, cameras, TVs, etc.) in an opportunistic and collaborative way. This goal can be accomplished by sharing device resources using their built-in WiFi adapter. In this paper we explain why the standard ad-hoc mode for building mobile peer-to-peer networks is not always the best choice and we propose an alternative self-organizing approach in which an opportunistic infrastructure-mode WiFi network is built. The particularity of this network is that each device can either be an access point or a client and change its role and wireless channel over time. This contribution advances the state of the art by using a context-aware approach that considers actual frequency allocation to other devices and monitored traffic. We finally show that our approach increases the average speed for delivering messages to a level that in several situations outperforms previous work in the area, as well as a simple single-channel ad-hoc WiFi network.

ShAir is a middleware infrastructure that allows mobile applications to share resources of their devices (e.g., data, storage, connectivity, computation) in a transparent way. The goals of ShAir are: (i) abstracting the creation and maintenance of opportunistic delay-tolerant peer-to-peer networks; (ii) being decoupled from the actual hardware and network platform; (iii) extensibility in terms of supported hardware, protocols, and on the type of resources that can be shared; (iv) being capable of self-adapting at run-time; (v) enabling the development of applications that are easier to design, test, and simulate. In this paper we discuss the design, extensibility, and maintainability of the ShAir middleware, and how to use it as a platform for collaborative resource-sharing applications. Finally we show our experience in designing and testing a file-sharing application.

We present an 8-dimensional (8D) display that allows glasses-free viewing of 3D imagery, whist capturing and reacting to incident environmental and user controlled light sources. We demonstrate two interactive possibilities enabled by our lens-array-based hardware prototype, and realtime GPU-accelerated software pipeline. Additionally, we describe a path to deploying such displays in the future, using current Sensor-in-Pixel (SIP) LCD panels, which physically collocate sensing and display elements.

Recently, several camera designs have been proposed for either making defocus blur invariant to scene depth or making motion blur invariant to object motion. The benefit of such invariant capture is that no depth or motion estimation is required to remove the resultant spatially uniform blur. So far, the techniques have been studied separately for defocus and motion blur, and object motion has been assumed 1D (e.g., horizontal). This article explores a more general capture method that makes both defocus blur and motion blur nearly invariant to scene depth and in-plane 2D object motion. We formulate the problem as capturing a time-varying light field through a time-varying light field modulator at the lens aperture, and perform 5D (4D light field + 1D time) analysis of all the existing computational cameras for defocus/motion-only deblurring and their hybrids. This leads to a surprising conclusion that focus sweep, previously known as a depth-invariant capture method that moves the plane of focus through a range of scene depth during exposure, is near-optimal both in terms of depth and 2D motion invariance and in terms of high-frequency preservation for certain combinations of depth and motion ranges. Using our prototype camera, we demonstrate joint defocus and motion deblurring for moving scenes with depth variation.

We describe a mobile application for sharing user-authored photo content in realtime called CoCam. CoCam is a collaborative content sharing framework based on opportunistic P2P proximal networking. CoCam users who are located in the same physical space can automatically share the photos they create as well as receive photos from other users around them. Since CoCam is based on an opportunistic P2P network middleware, users are not required to know each other in advance. It is also not necessary for them to agree on the same service provider nor coordinate the network configuration, infrastructure and security settings. This middleware automatically discovers other peers and handles the organization of ad-hoc network connections. With CoCam, we demonstrate that users are able to share and enjoy shared photos and video streams without the effort of manual setup and cost associated with the 3G/4G network.

In this paper, we propose CoCam, a framework for mobile phones that enables uncoordinated real-time image and video collaboration between different users sharing the same context, in the same physical location. CoCam addresses the complexity and difficulty introduced when users wish to collaborate, create, and share media contents at the same time. CoCam is based on a middleware that creates a self-organizing ad-hoc network within the context of a shared event in a proximal physical space (a common scene). This middleware automatically handles the context detection, as well as the network configuration and peer discovery. It also enables real-time content sharing while reducing the burden of complicated settings and configurations by the users. As a result of operation tests and user study with the prototype implementation, we have verified that CoCam is feasible and has the potential to enrich the users' experience when sharing contents in an event scenario.

In this paper, we examine the implementation and usage scenarios for WeatherPlay, a web site that collects weather and travel TV clips, microblog entries, amateur videos, and outdoor data. In order to present a comprehensive picture of people's outdoor experiences, WeatherPlay geolocates this media and places it in both a map and video gallery context.

People can have more insights and social experiences when they collaborate on collecting, revisiting, and utilizing their contents, such as images and videos; however, designing a social space that offers rich co-creation and exploration of multimedia contents remains a challenge. I propose a new system, SparkInfo, which enables users to create, exchange and augment their multimedia elements in ways that are personally unique and sociable. SparkInfo is designed for a group of people, who have created multimedia elements for the same purpose or at the same event, to collect their elements in one place and have a meaningful experience of their co-created media resources. SparkInfo provides a social space for the co-creation of multimedia resources. In the process of exploring and embellishing their materials, SparkInfo users can create new ideas, stories, and information. By utilizing this process, the users are able to experience how SparkInfo can embody the cycle of knowledge building, re-mixing, and sharing.

ATTN-SPAN is a digital content summarization and contextualization platform designed to compile video footage into personalized episodes. Viewers are able to associate opinions and externally sourced static content to moments within the video, creating a far richer viewing experience and supporting a community knowledge base around civic information. The pilot project utilizes C-SPAN footage to create daily episodes for each user of the system based on his or her political interests.

WorldTV is a proposed TV app for the discovery of user-generated video from around the world. We describe a two-screen solution (a TV app and accompanying mobile app to control what appears on the television screen) would let viewers browse and play geotagged videos plotted on a 3D model of the earth. The content would include video from viewers' own social networks as well as user-generated news and event video filtered according to viewers' interests, preferences, and social connections. In a world in which user-generated online video content is being uploaded and consumed at an exponential rate, the idea of using a globe-based navigation metaphor has powerful appeal. It is instantly approachable and easy to use, especially for people with global networks of friends, family, and colleagues living in other countries. We also argue that the widespread availability of geotagged video provides an additional impetus for using a geo-focused UI over more traditional browsing and discovery mechanisms.

magine a display that behaves like a window. Glancing through it, viewers perceive a virtual 3D scene with correct parallax, without the need to wear glasses or track the user. Light that passes through the display correctly illuminates the virtual scene. While researchers have considered such displays, or prototyped subsets of these capabilities, we contribute a new, interactive, relightable, glasses-free 3D display. By simultaneously capturing a 4D light field, and displaying a 4D light field, we are able to realistically modulate the incident light on rendered content. We present our optical design, and GPU pipeline. Beyond mimicking the physical appearance of objects under natural lighting, an 8D display can create arbitrary directional illumination patterns and record their interaction with physical objects. Our hardware points the way towards novel 3D interfaces, in which users interact with digital content using light widgets, physical objects, and gesture.

ATTN-SPAN is a digital content summarization and contextualization platform designed to compile video footage into personalized episodes. Viewers are able to associate opinions and externally sourced static content to moments within the video, creating a far richer viewing experience and supporting a community knowledge base around civic information. The pilot project utilizes C-SPAN footage to create daily episodes for each user of the system based on his or her political interests.

CommenTV enables users to share their comments in parallel with time-based content. CommenTV utilizes existing web technologies to enhance current TV or video player solutions with a new type of comment system. CommenTV is able to take and display texts, images, and related videos as social comments. In this way, the CommenTV users can experience audiovisual content that blends with their social networks by sharing comments. Ultimately, CommenTV can be a social bookmarking system for audiovisual content that informs content creators about how viewers perceive their content and lets viewers participate in creating a more social content.

Some of the world's most pressing problems can be traced to inequity between people, both in the present and over time. Long periods of inequity can lead to both social and environmental degradation. In its 2011 Human Development Report (HDR), the United Nations incisively examines some of the complex relationships between socioeconomic equity and environmental sustainability. Members of the MIT Media Lab and The DuKode Studio created "Networks in Equity and Sustainability", a visualization tool that shows, via a network graph, how nations are multi-dimensionally linked. Examining linkages with this tool can illuminate potential partnerships between cultures. As the tool is expanded in the future, it will support intercultural discussions in the global effort for a world that is more equal today and more sustainable over time.

Most organizations have a wealth of knowledge about themselves available online, but little for a visitor to interact with on‐site. At the MIT Media Lab, we have designed and deployed a novel intelligent signage system, the Glass Infrastructure (GI), that enables small groups of users to interact physically through a touch‐screen display with this data and to discover the latent connections between people, projects, and ideas. The displays are built on an adaptive, unsupervised model of the organization and its relationships developed using dimensionality reduction and commonsense knowledge that automatically classifies and organizes the information. The GI is currently in daily use at the lab. We discuss the AI model's development, the integration of AI into a human‐computer interaction (HCI) interface, and the use of the GI during the lab's peak visitor periods. We show that the GI is used repeatedly by lab visitors and provides a window into the workings of the organization.

People can have more insights and social experiences when they collaborate on collecting, revisiting, and utilizing their contents, such as images and videos; however, designing a social space that offers rich co-creation and exploration of multimedia contents remains a challenge. We propose a new system, SparkInfo, which enables users to create, exchange and augment their audiovisual elements in ways that are personally unique and sociable. SparkInfo is designed for a group of people, who have created audiovisual elements for the same purpose or at the same event, to collect their elements in one place and have a meaningful experience of their co-created media resources. SparkInfo provides a social space for the co-creation of audiovisual and multimedia resources. In the process of exploring and embellishing their materials, SparkInfo users can create new ideas, stories, and information. By utilizing this process, the users are able to experience how SparkInfo can embody the cycle of knowledge building, re-mixing, and sharing.

MatchMaker is an automated collaborative filtering system that recommends friends to people on Facebook by analyzing and matching people's online profile with the profiles of TV characters. The goal of MatchMaker is to produce friend recommendations with rich contextual information through collaborative filtering in the existing social network. Using relationships in TV programs as a parallel comparison matrix, MatchMaker projects these relationships into reality to help people find friends whose personality and characteristics have been voted to suit them well by their social network. MatchMaker also encourages more TV content viewing by using the social network context and connections to provoke people's curiosity of TV characters whom they have been matched with in their social network.

Social television, in essence, is a way to share and experience media with others. Rather than attempt to bridge physical distance by creating new social experiences, we have built a system called TakeoverTV that augments canonical social interactions by addressing three key questions: How can we better support social gatherings around media in public and private places?; How can we enable conversation across our devices rather than forcing us to communicate through them?; and How can we help people come to agreements about what media they want to watch together? Primarily aimed at public spaces like bars, TakeoverTV is a system that lets local users influence and interact with the movies and television programs shown on public displays. Our system collects the media preferences of all people physically present at a given location, lets users start a vote among all members of the space, and allows everyone to participate using tangible physical objects like tables in addition to smart devices. Over time, public establishments can evolve complex identities based on the preferences and votes of their patrons while generating valuable analytic data for patrons, establishment owners, content-providers, and advertisers alike.