Ph.D. Students

• Akshay Krishnan
• Mengyu Yang
• Jim James
• Lingjun Zhao (co-advised with Lu Gan)

Past Ph.D. Students

• Benjamin Wilson now at Latitude AI
• Cusuh Ham now at Adobe
• Patrick Grady now at Meta Reality Labs
• Amit Raj now at Google
• Sean Foley now at NASA
• John Lambert now at Google Deepmind
• Patsorn Sangkloy now at NYU
• Samarth Brahmbhatt now at Overland AI
• Nam Vo now at Roku
• Genevieve Patterson
• Libin "Geoffrey" Sun now at Apple

Previous Postdoc

• Pierre-Yves Laffont now at Meta

Visiting Student Alumni

• Tsung-Yi Lin now at Nvidia
• Huaijin "George" Chen now at U Hawaii
• Mathias Eitz

Undergrad and MS Lab Alumni

• Wenqi Xian, Yipin Zhou, Hang Su, Sirion Vittayakorn

New: Book chapter with expanded geolocalization experiments.

Large-Scale Image Geolocalization
James Hays and Alexei Efros.
Multimodal Location Estimation of Videos and Images. Pages 41-62. 2014.

Paper, Bibtex

Previously published as:
IM2GPS: estimating geographic information from a single image
James Hays and Alexei Efros.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2008.
Paper. CVPR 2008 Project Page. Google Tech Talk.

Abstract: Estimating geographic information from an image is an excellent, difficult high-level computer vision problem whose time has come. The emergence of vast amounts of geographically-calibrated image data is a great reason for computer vision to start looking globally - on the scale of the entire planet! In this paper, we propose a simple algorithm for estimating a distribution over geographic locations from a single image using a purely data-driven scene matching approach. For this task, we will leverage a dataset of over 6 million GPS-tagged images from the Internet. We represent the estimated image location as a probability distribution over the Earth's surface. We quantitatively evaluate our approach in several geolocation tasks and demonstrate encouraging performance (up to 30 times better than chance). We show that geolocation estimates can provide the basis for numerous other image understanding tasks such as population density estimation, land cover estimation or urban/rural classification.

Scene Completion Using Millions of Photographs
James Hays and Alexei Efros.
Transactions on Graphics (SIGGRAPH 2007). August 2007, vol. 26, No. 3.

Abstract: What can you do with a million images? In this paper we present a new image completion algorithm powered by a huge database of photographs gathered from the Web. The algorithm patches up holes in images by finding similar image regions in the database that are not only seamless but also semantically valid. Our chief insight is that while the space of images is effectively infinite, the space of semantically differentiable scenes is actually not that large. For many image completion tasks we are able to find similar scenes which contain image fragments that will convincingly complete the image. Our algorithm is entirely data-driven, requiring no annotations or labelling by the user. Unlike existing image completion methods, our algorithm can generate a diverse set of image completions and we allow users to select among them. We demonstrate the superiority of our algorithm over existing image completion approaches.

Interactive Tensor Field Design and Visualization on Surfaces
Eugene Zhang, James Hays, and Greg Turk.
IEEE Transaction on Visualization and Computer Graphics, 2007, Vol 13(1), pp 94-107.

This research project was primarily Eugene's work and I played only a small role.

Image De-fencing
Yanxi Liu, Tamara Belkina, James Hays, and Roberto Lublinerman.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2008.

We introduce a novel image segmentation algorithm that uses translational symmetry as the primary foreground/background separation cue. We use texture-based inpainting to recover an un-occluded background.

Discovering Texture Regularity as a Higher-Order Correspondence Problem
James Hays, Marius Leordeanu, Alexei Efros, and Yanxi Liu.
European Conference on Computer Vision (ECCV) 2006.

We find arbitrarily distorted regular patterns in real images by treating lattice-finding as a higher-order assignment problem. We leverage previous work from Marius Leordeanu and Martial Hebert to approximate the optimal assignment under second-order constraints.

Source code available upon request, although this code by Minwoo Park et al. is likely more accurate and faster.

Quantitative Evaluation of Near Regular Texture Synthesis Algorithms
Steve Lin, James Hays, Chenyu Wu, Vivek Kwatra, and Yanxi Liu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2006

Quantitative evaluation is difficult for texture synthesis. Ground truth is not well defined. But for certain textures you can objectively decide whether an algorithm has failed or not. Regular and near-regular textures imply a definite structure that should be preserved. We tested several popular algorithms on a large group of structured textures. In addition to the CVPR 2006 paper, a more detailed technical report is available.

Digital Papercutting
Yanxi Liu, James Hays, Ying-Qing Xu, and Harry Shum SIGGRAPH 2005 Sketch

Papercutting is a widespread and ancient artform which, as far as we could tell, had no previous computational treatment. We developed algorithms to analyze the symmetry of papercut patterns and produce efficient folding and cutting plans.

Near-Regular Texture Analysis and Manipulation
Yanxi Liu, Steve Lin, and James Hays. SIGGRAPH 2004

Abstract: A near-regular texture deviates geometrically and photometrically from a regular congruent tiling. Although near-regular textures are ubiquitous in the man-made and natural world, they present computational challenges for state of the art texture analysis and synthesis algorithms. Using regular tiling as our anchor point, and with user-assisted lattice extraction, we can explicitly model the deformation of a near-regular texture with respect to geometry, lighting and color. We treat a deformation field both as a function that acts on a texture and as a texture that is acted upon, and develop a multi-modal framework where each deformation field is subject to analysis, synthesis and manipulation. Using this formalization, we are able to construct simple parametric models to faithfully synthesize the appearance of a near-regular texture and purposefully control its regularity.

Image and Video Based Painterly Animation
James Hays and Irfan Essa. NPAR 2004.

We extend previous non-photorealistic rendering work to handle video significantly better by temporally constraining brush stroke properties in addition to other improvements.