A Look at Google’s Patent Applications for Light Field Technology

“The global market size for virtual and augmented reality has reached $20.4 billion and its projected economic impact by 2020 is $15.6 billion. To harness that growth for its own benefit, Google will have to innovate.”

Google’s U.S. patent application 20190124318—originally assigned to Lytro, an American developer of light-field cameras, before it went defunct in March 2018—was published in April 2019. Google is actively working on concepts constructing Virtual Reality (VR) and Augmented Reality (AR) and has spun out products like Daydream, ARCore, Cardboard, Jump, Tilt Brush and Blocks, all of which benefit from light field technology. CNET reported last year that a number of Lytro employees were heading to Google after Lytro folded, and that Google would likely acquire some Lytro patents.

Global market research and management consultation company, Global Market Insights, reports that the light field market will grow to $1.5 billion by 2024, rising from $650 million in 2017. The 2017 figures point to media and entertainment industries like gaming, theaters and amusement parks, making up 25% of the overall industry shares. The adoption of the technology in cameras is predicted to drive this growth.

Uses of Light Field Technology

Matt Hirsch of Lumii (@lumiidisplay.com), a company based out of Boston that makes holographic prints for packaging, explained the uses of light field technology and how his company has evolved with it.

“From a historical perspective, the term light field was popularized as a branch of computer science,” Hirsch said. “There was a lot of work on image-based rendering using ray tracing efficiently to render realistic scenes.”

Hirsch said that, for many years, light field technology was “an easy way” of handling all of the lighting data in a rendered scene: data on light, the direction it moves and its interaction with objects. If you can imagine, Hirsch said, a high dimensional function that parameterizes all the aspects of how light travels, then you have a basic idea of how exciting light field theory is.

Most of the practical benefits in the early work of light field technologies were with graphics; making scenes seem real on the computer screen. Today, with VR goggles and other goggle-less images, which Lumii produces on packages, light field technology is a boon to creating images that look real because of the utilization of considerable amounts of data on how light travels through a space.

The Google Patents

The U.S. Patent Application No. 20190124318, published in April 2019 to Google, titled Capturing Light-Field Images with Uneven and/or Incomplete Angular Sampling, describes a method in which a light field camera has an image sensor and a microlens array to configure light at that image sensor. One of the possibilities described is the ability to refocus the image after the photo is taken, something that Lytro was famous for pioneering. Light field cameras can be used to capture a four-dimensional light field comprising two spatial dimensions and two angular dimensions. These cameras can capture more versatile images in which focus distance, center of perspective and depth of field can be varied. This patent application describes the possibility that complete and even sampling of the two angular dimensions may not even be required and that a combination of high-resolution image data and depth data may be attained with the same result.

Other patents originally assigned to Lytro describing light field technologies have also been assigned to Google in the last year:

U.S. Patent No. 10129524, titled Depth-Assigned Content for Depth-Enhanced Virtual Reality Images, discloses an example of a light field picture that contains information about the direction of light as it arrives at a sensor. The images generated from the rendering of this data can be associated with different depths.

U.S. Patent No. 10205896, titled Automatic Lens Flare Detection and Correction for Light-Field Images. One of the obstacles of light field image data is the occurrence of lens flare. However, this patent discloses a method in which flare affected pixel values are corrected to remove the flare from the image. This is possible in light field digital images because they typically encode more data for every pixel related to the trajectory of light rays.

It seems likely that Google will continue to republish Lytro’s patent applications in the near future. According to Statista, an online statistics organization, the global market size for VR and AR has reached $20.4 billion and its projected economic impact by 2020 is $15.6 billion. To harness that growth for its own benefit, Google will have to innovate, and light field development seems like its best bet.

Share

Warning & Disclaimer: The pages, articles and comments on IPWatchdog.com do not constitute legal advice, nor do they create any attorney-client relationship. The articles published express the personal opinion and views of the author as of the time of publication and should not be attributed to the author’s employer, clients or the sponsors of IPWatchdog.com.

Join the Discussion

3 comments so far.

  • [Avatar for Anon]
    Anon
    June 16, 2019 08:10 am

    Mr. Morgan, I take the “/s” to be a “Sarcasm” sign, right?

    But to your comment, I would implore someone (anyone) to actually provide a single “objective physical structure” description of this mythical “general purpose computer” that somehow is “magically” able to apply these ideas (abstract or otherwise) that may appear on the computing system element of “software” without first changing that computer in order to become programmed with that “software.”

    There is no such thing as “just implement” without having to change the machine with the new software component.

    There is no such thing as ANY computer (the mythical general purpose one or otherwise) that can — with NO changes — simply and magically have the new functionality as represented by the new software component. If this were so, then you would not NEED the new software component and the machine would inherently already have the new functionality already present.

    The elephant in the room is that the so-called “general purpose computer” is merely a machine capable of being changed in a number of patent-equivalent design choice manners of “wares.” There is a fundamental fallacy being hoisted every time the notion of “general purpose computer” is being used by the courts (and those advocating that the courts’ rhetoric be given factual and legal weight).

  • [Avatar for George D. Morgan]
    George D. Morgan
    June 15, 2019 10:22 pm

    Just abstract ideas implemented on a general purpose computer. /s

  • [Avatar for Pro Say]
    Pro Say
    June 15, 2019 07:13 pm

    Abstract! No patent for you!

    Oh, but wait …

    NONE of Google’s apps or patents are abstract … while ALL of everyone else’s (and in particular those who’s patents they’re infringing) are.

    According to Google, that is.

    Hypocrites gonna hypocratize.