Apple Seeks Patent on Gaze Detection Capabilities

Apple Inc. of Cupertino, CA, has been a regularly featured corporation in our Companies We Follow series. The multinational corporation has been a major name in consumer electronics and computer software, owing largely to the market success of the iPad and the iPhone. Recently, Apple has been making more moves into media application development, as is suggested by recent agreements with Warner Music and others to provide streaming radio services.

This week, we’re featuring a number of interesting new patents and published applications from the U.S. Patent & Trademark Office that have been assigned to Apple. A few of the applications we’ve chosen to profile include more efficient systems of detecting user inputs. One application describes a system allowing devices to enter a low-power mode based on user gaze detection, conserving battery power. Another application would protect a system for better facial recognition during photo processing of image files.

Other documents assigned to Apple showcase the corporation’s focus on aiding user communication and providing a more user-intuitive device experience. One application featured here was filed to protect a system of analyzing a user’s media preferences for gaming environments, while another improves a user’s ability to share a pinned location on a map with others.

An patent awarded to Apple this week protects a richer system for accessibility software, allowing users to enhance their reading experience rather than rely on continuous audio playback.

[Companies-5]

Electronic Devices with Gaze Detection Capabilities
U.S. Patent Application No. 20130135198

Computing devices come in many forms in today’s market. Along with desktops and laptops, handheld devices have proliferated, and many other interactive computer systems are widely available in vehicles and more. Especially among smaller devices, space for various components comes at a premium. Speakers, cameras and battery packs, among other components, must be designed to fit in small spaces. In the case of battery packs, this can reduce the amount of power that can be made available to a device.

Apple has devised a new system of conserving battery power that relies on gaze detection to determine which mode the computer should operate in. An electronic device would operate in active mode if it detects that a user’s gaze has been directed towards the display screen. When the user’s gaze is diverted from the screen, the device then enters sleep mode to conserve battery power.

Claim 1 of this Apple patent application describes:

“A method for using a portable electronic device having an accelerometer and gaze detection circuitry, the method comprising: with the accelerometer, determining whether a measured acceleration level for the portable electronic device has exceeded a given threshold value; and when it has been determined that the acceleration level of the portable electronic device has exceeded the given threshold value, disabling the gaze detection circuitry.”

Location Sharing
U.S. Patent Application No. 20130137462

Electronic device owners can communicate with each other in various ways. Besides calling and texting, these devices contain many applications that allow users to play each other in games, communicate via social networks and more. At times, it becomes advantageous to device owners to allow better sharing of information by allowing two or more of these applications to interact directly.

Apple is hoping to protect a new method of sharing location information between electronic device users with this patent application. Users would be able to drop a pin onto a map application and store that location as a bookmark. That location pin could then be sent to another user with a modified message using a short message service (SMS) application. In this way, one person could easily share a home location or another destination with a friend or co-worker.

As Claim 1 explains, Apple is seeking protections for:

“A non-transitory computer-readable medium having instructions stored thereon, which, when executed by a processor, cause the processor to perform operations comprising: receiving a message and geographic location data from a device; processing the message to determine if the message is associated with geographic location data; and presenting the message in a graphical object having a graphical element indicating that the message is associated with the geographic location data, if the message is associated with the geographic location data; obtaining input to the graphical element; and presenting a map display with the geographic location of the device identified, in response to the input to the graphical element.”

Allowing Media and Gaming Environments to Effectively Interact and/or Affect Each Other
U.S. Patent Application No. 20130130803

Gaming applications have become an incredibly popular feature of handheld electronic devices. A variety of strategy, action and puzzle games exist, some of which feature high-quality graphics, social score leader boards and other inventive features. However, gaming applications do not have much interaction with a device’s media environment beyond being presented as an application. Having access to data about a user’s media environment would provide a more user-intuitive gaming experience.

[Companies-3]

This patent application from Apple has been filed to protect a system of interaction between gaming applications and media environments. This system would allow a gaming environment to be designed that takes inputs from a user’s media preferences, specifically for music. A “musical profile” could be built based on the user’s music library that affects the music played during game play. The description also states that the interactivity could work both ways and the application might suggest music based on a user’s library or preferences.

Claim 1 of this patent application would give Apple protection over:

“A computer-implemented method comprising: accessing media content from a plurality of media assets stored in a media environment associated with a computing system, the media environment being an environment other than a gaming environment; obtaining game data associated with the gaming environment; determining, based on the game data, an output set of media assets selected from the plurality of media assets; and simultaneously outputting the output set of media assets and gaming content in the gaming environment.”

Detecting Skin Tone in Images
U.S. Patent Application No. 20130129209

Handheld electronic devices with camera components installed in the device housing have grown in capabilities recently where photo processing is concerned. Users can easily crop images or perform a number of operations using photography applications. At times, it’s useful for a software program to be able to detect how many faces exist in a picture. Enhanced methods of facial recognition for determining a subject’s identity would also be helpful for gleaning information from an image.

This Apple system of detecting skin tone in images uses image processing techniques to determine where a face exists within a stored image. Once a face has been detected, a sample is gathered from the face that will contain different shaded areas of the subject’s skin. Different pixels can be evaluated against a red-green-blue (RGB) color model to determine a tone within a certain color space that is “characteristic of human flesh,” according to this application’s description. This system can also be configured to determine skin tone even if an image filter has been applied to the photo.

As Claim 1 states, Apple wants to protect:

“A program storage device, on which are stored instructions comprising instructions for causing a computer system to: select a sample portion of an image that may depict a human face, comprising a plurality of pixels; indicate a likelihood that the image does not depict a human face if the pixels in the sample portion are substantially neutral; and indicate a likelihood that the image depicts a human face if the pixels in the sample portion correspond to skin tone colors.”

Assisted Reader
U.S. Patent No. 8452600

Accessibility software provides many options for device users who are visually or hearing impaired. For example, visually impaired device owners can use accessibility software for audio playback of eBooks or other documents to aid in comprehension. However, this audio playback is can typically only be operated continuously, which isn’t very useful for those who can read large portions of the book and only need audio playback in limited cases.

This USPTO patent protects a Apple invention that provides for an enhanced assisted reading mode in addition to a continuous assisted reading mode. In the enhanced assisted reading mode, an electronic device can receive touch inputs through the display screen to determine which words or lines require processing from accessibility software.

Claim 1 of this patent document gives Apple the right to protect

“A method performed by one or more processors of an assisted reading device, the method comprising: providing a user interface on a display of the assisted reading device, the user interface displaying text of a content item and configured to distinguish between a first type of gesture for selecting a continuous assisted reading mode and a second type of gesture for selecting an enhanced assisted reading mode of the device and a respective portion of the displayed text to be read in the enhanced assisted reading mode; receiving a first touch input on the user interface; upon determining, based on the first touch input, that the first type of gesture has been entered: invoking the continuous assisted reading mode; and continuously outputting audio for each word in a currently displayed portion and all subsequent portions of the content item until an end of the content item is reached or a user input for stopping or pausing the continuous assisted reading mode is received; and upon determining, based on the first touch input, that the second type of gesture has been entered: invoking the enhanced assisted reading mode; receiving a second touch input for selecting a desired level of reading granularity; configuring the assisted reading device to provide the selected level of reading granularity; based on a location of the first touch input on the user interface and the selected level of granularity, selecting the respective portion of the displayed text to be read in the enhanced assisted reading mode; and outputting audio for each word in the selected portion of the displayed text.”