Fast computational camera based on two arrays of lenses

Details Fast computational camera based on two arrays of lenses Fast computational camera based on two arrays of lenses

2011-06-07

2011-06-07

Chan, Jason (Department: 2622)

Television

Camera, system and detail

Optics

C348S315000, C348S335000, C348S343000, C348S344000

Reexamination Certificate

active

07956924

ABSTRACT:
Method and apparatus for a fast (low F
umber) computational camera that incorporates two arrays of lenses. The arrays include a lenslet array in front of a photosensor and an objective lens array of two or more lenses. Each lens in the objective lens array captures light from a subject. Each lenslet in the lenslet array captures light from each objective lens and separates the captured light to project microimages corresponding to the objective lenses on a region of the photosensor under the lenslet. Thus, a plurality of microimages are projected onto and captured by the photosensor. The captured microimages may be processed in accordance with the geometry of the objective lenses to align the microimages to generate a final image. One or more other algorithms may be applied to the image data in accordance with radiance information captured by the camera, such as automatic refocusing of an out-of-focus image.

REFERENCES:
patent: 725567 (1903-04-01), Ives
patent: 2039648 (1936-05-01), Ives
patent: 3985419 (1976-10-01), Matsumoto et al.
patent: 4180313 (1979-12-01), Inuiya
patent: 4193093 (1980-03-01), St. Clair
patent: 4849782 (1989-07-01), Koyama et al.
patent: 5076687 (1991-12-01), Adelson
patent: 5361127 (1994-11-01), Daily
patent: 5400093 (1995-03-01), Timmers
patent: 5659420 (1997-08-01), Wakai et al.
patent: 5729011 (1998-03-01), Sekiguchi
patent: 5946077 (1999-08-01), Nemirovskiy
patent: 6097541 (2000-08-01), Davies et al.
patent: 6137937 (2000-10-01), Okano et al.
patent: 6268846 (2001-07-01), Georgiev
patent: 6301416 (2001-10-01), Okano et al.
patent: 6339506 (2002-01-01), Wakelin et al.
patent: 6351269 (2002-02-01), Georgiev
patent: 6476805 (2002-11-01), Shum et al.
patent: 6738533 (2004-05-01), Shum et al.
patent: 6838650 (2005-01-01), Toh
patent: 7054067 (2006-05-01), Okano et al.
patent: 7620309 (2009-11-01), Georgiev
patent: 7732744 (2010-06-01), Utagawa
patent: 2001/0012149 (2001-08-01), Lin et al.
patent: 2001/0050813 (2001-12-01), Allio
patent: 2002/0140835 (2002-10-01), Silverstein
patent: 2003/0231255 (2003-12-01), Szajewski et al.
patent: 2004/0114807 (2004-06-01), Lelescu et al.
patent: 2004/0223214 (2004-11-01), Atkinson
patent: 2005/0088714 (2005-04-01), Kremen
patent: 2008/0152215 (2008-06-01), Horie et al.
patent: 2008/0165270 (2008-07-01), Watanabe et al.
patent: 2008/0193026 (2008-08-01), Horie et al.
patent: 2009/0041381 (2009-02-01), Georgiev
patent: 2009/0041448 (2009-02-01), Georgiev
patent: 2009/0140131 (2009-06-01), Utagawa
patent: 2010/0026852 (2010-02-01), Ng et al.
patent: 1548481 (2005-06-01), None
patent: 0137025 (2001-05-01), None
patent: 2006/057838 (2006-06-01), None
patent: 2007/115281 (2007-10-01), None
J. Neumann, et al., “Eyes from Eyes Analysis of Camera Design Using Plenoptic Video Geometry,” Dec. 2001, XP002509893.
Zaharia, R., et al., “Adaptive 3D-DCT Compression Algorithm for Continuous Parallax 3D Integral Imaging,” Signal Processing, Image Communication, Elsevier Scient Publishers, Amsterdam, NL, vol. 17, No. 3, Mar. 1, 2002.
Ren NG, et al., “Light Field Photography with a Hand-Held Plenoptic Camera,” retrieved from Internet Mar. 6, 2007, XP002423384.
Yang, J C, et al., “A Real-Time Distributed Light Field Camera,” Rendering Techniques 2002, Eurographics Workshop Proceedings, PIS, Italy, Jun. 26-28, 2002.
International Search Report and Written Opinion from PCT/US2008/080238, mailed Jan. 23, 2009.
T. Adelson and J. Bergen, “The plenoptic function and the elements of early vision,” From M. Landy and J. A. Movshon (eds), Computational Models of Visual Processing (pp. 3-20). Cambridge, MA: MIT Press (1991), 18 pages.
Tanida, J. Yamada, K., “TOMBO: thin observation module by bound optics,” Lasers and Electro-Optics Society, 2002. LEOS 2002. The 15th Annual Meeting of the IEEE, Issue Date: 2002, pp. 233-234 vol. 1.
M Christensen, M Haney, D Rajan, S Douglas, and S Wood, “Panoptes: A thin agile multi-resolution imaging sensor,” Government Microcuircuit Applications and Critical, Technology Conference (GOMACTech-05)(Jan. 2005), 4 pages.
D Capel and A Zisserman, “Computer vision applied to super resolution,” Signal Processing Magazine(Jan. 2003), 10 pages.
P Sloan, M Cohen, and S Gortler, “Time critical lumigraph rendering,” Proceedings of the 1997 symposium on Interactive 3D graphics (Jan. 1997), 7 pages.
S Todt, C Rezk-Salama, A Kolb, and K.-D Kuhnert, “Fast (Spherical) Light Field Rendering with Per-Pixel Depth,” Technical Report, Computer Graphics Group, University of Siegen, 2007, 8 pages.
John Nickolls, Ian Buck, Michael Garland, and Kevin Skadron, “Scalable parallel programming with cuda,” Queue 6, 40-53 (2008), 40 pages.
John E. Stone, David Gohara, and Guochun Shi, “OpenCL—The open standard for parallel programming of heterogeneous systems.” Comput. in Sci. and Eng., 12:66-73, 2010, 66 pages.
“PyOpenGL, The Python OpenGL® Binding” downloaded from http://pyopengl.sourceforge.net/ on Dec. 21, 2010, 2 pages.
Mark Segal, Kurt Akeley. The OpenGL Graphics System: A Specification (Version 3.2 (Core Profile)—Dec. 7, 2009), 104 pages.
John Kessenich, Dave Baldwin, Randi Rost. The OpenGL Shading Language, Version: 4.00, Document Revision: 7, Feb. 12, 2010, 160 pages.
Georgiev T., Lumsdaine A.: Depth of field in plenoptic cameras. In Eurographics 2009—Annex (Apr. 2009), pp. 5-8.
Shree K. Nayar, Vlad Branzoi, Terry E. Boult, “Programmable Imaging using a Digital Micromirror Array”, Shree K. Nayar, Vlad Branzoi, Terry E. Boult, In Conf. on Computer Vision and Pattern Recognition, pp. I:436-443, 2004.
Georgeiv, et al. “Spatio-Angular Resolution Tradeoff in Integral Photography,” Proc. EGSR, 2006, 10 pages.
Levoy, Mark et al., “Light Field Microscopy,” Stanford University, Submitted to Siggraph 2006, 5 pages.
Shree K. Nayar, Vlad Branzoi, Terry E. Boult, “Programmable Imaging using a Digital Micromirror Array”, Shree K. Nayar, Vlad Branzoi, Terry E. Boult, In Conf. on Computer Vision and Pattern Recognition, pp. I:436-443, 2004.
Levoy, Mark, “Optical Recipes for Light Microscopes,” Stanford Computer Grpahics Laboratory Technical Memo 2006-001, Jun. 20, 2006, 10 pages.
Adelson T., Wang J.: “Single Lens Stereo with a Plenoptic Camera,” IEEE Transactions on Pattern Analysis and Machine Intelligence (1992), 99-106.
Brown M., Lowe D. G.: “Unsupervised 3d Object Recognition and Reconstruction in Unordered Datasets,” In Proceedings of 5th International Conference on 3D Imaging and Modelling (3DIM) (2005), pp. 21-30.
Gortler S. J., Grzeszczuk R., Szeliski, R., Cohen M. F.: “The Lumigraph,” ACM Trans. Graph. (1996), 43-54.
Georgiev T., Wainer M.: “Morphing Between Multiple Images,” Tech. Rep. (1997).
Isaksen A., McMillan L., Gortler S. J.: “Dynamically Reparameterized Light Fields,” ACM Trans. Graph. (2000), 297-306.
Xiao J., Shah M.: “Tri-view Morphing,” Computer Vision and Image Understanding 96, 3 (2004), 345-366.
Lippmann G.: “Epreuves reversible donnant la sensation du relief (“Reversible Prints Providing the Sensation of Depth”)”, Journal of Physics 7 (1908), pp. 821-825.
Lippmann G.: “Epreuves Reversibles Photographies Integrales,” Academie des sciences (Mar. 1908), pp. 446-451.
Lee S., Wolberg G., Shin S.: “Polymorph: Morphing Among Multiple Images,” IEEE Computer Graphics and Applications (1998).
Park, S., Park, M., and Kang, M., “Super-resolution image reconstruction: a technical overview,” Signal Processing Magazine, 2003, 16 pages.
NG R.: “Fourier Slice Photography,” Proceedings of ACM SIGGRAPH 2005 (Jan. 2005).
Naemura T., Yoshida T., Harashima H.: “3d Computer Graphics Based on Integral Photography,” Optics Express, vol. 8, 2 (2001).
Zitnick C. L., Jojic N., Kang S.: “Consiste

Affiliated with

Also associated with

Rating

Fast computational camera based on two arrays of lenses does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Fast computational camera based on two arrays of lenses, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Fast computational camera based on two arrays of lenses will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-2709771