Facial Recognition Technology Raising Privacy Concerns

The Federal Trade Commission is seeking public comments on the issues raised at a recent FTC workshop exploring facial recognition technology and the privacy and security implications raised by its increasing use. The December 8, 2011, public workshop, “Face Facts: A Forum on Facial Recognition Technology,” focused on the current and future commercial applications of facial detection and recognition technologies, and discussed current uses of these technologies, possible future uses and benefits. Now the focus shifts to the potential harms, namely the numerous potential privacy and security concerns.

Facial detection and recognition technologies have been adopted in a variety of new contexts, ranging from online social networks to automobiles to Automatic Teller Machines and must more. Indeed, there are a variety of facial recognition innovations already patented and many with patent pending status. A search of the United States Patent and Trademark Office database for the term “facial recognition” within the Abstract of an issued patent or published patent application returns 181 results.

As I read through the patents and patent applications discussed below, and the many more I did not include, I started to wonder whether anyone has any reasonable expectation of privacy at all any more. I am a big fan of the CBS drama Person of Interest, and the surveillance system from that show that ferrets out dangerous on both macro and micro levels doesn’t seem quite so far fetched. I suppose that is why the FTC is seeking comments on facial recognition technologies and the government is attempting to get its hands around the enormous issues and promulgate some rules or guidelines.

Without trying to be exhaustive, here is a sampling of some of the facial recognition technologies the United States Patent and Trademark Office has seen to date. The breadth and scope of applications will likely take those who cherish personal privacy by surprise.

[Google_468_60]

1. Facial based image organization and retrieval method
U.S. Patent No. 7,916,976

This invention relates to a system and set of processes for organizing image collections. The system detects individuals in each image uploaded into the system using facial recognition or similar methods. The user and viewers of the images may then view dynamic albums based on the interrelationships of individuals in images. Users and viewers may browse all images with an individual or see albums of images with two selected individuals or similar combinations based on the relationships between users.

This innovation seems similar to what Facebook now offers, which when implemented caught me by surprise and seemed a little too intrusive for my liking.

2. Event matching in social networks
U.S. Patent Application No. 20110211737

This Microsoft patent application relates to an innovation where images from two image databases may be correlated based on identifying a common event, which may be determined by image metadata as well as image content. The image metadata can include timestamps, geotagging metadata, or other tags, as well as input from a social network application in some embodiments. The image content may also include analysis to find common persons based on facial recognition or color histograms, common background components, or other common features. The common event may be used to identify images that may be shared among the participants of the event by a social network application, as well as other purposes.

It is no doubt it is the “as well as other purposes” that will give privacy advocates pause. Like so many of these technologies on this list and within the walls of the USPTO you can understand why this would be useful, even desirable, for those who create or assemble content online.

3. Facial Recognition in groups
U.S. Patent Application No. 20070174272

This IBM patent application relates to a system for group facial recognition that includes a target extractor for extracting target images from a scene image; a target image classifier for comparing a target image to a database of known identities and allocating classification scores for identities; a relationship database providing relationship scores between known identities; and means for applying the relationship scores to the classification scores to improve classification of a target image.

The usefulness of this type of invention is almost certainly obviously apparent. For among other things it would be wonderful for law enforcement to be able to rely on high speed facial recognition technology to scan a group to identify whether known terrorists are present. Of course, there are any number of potential uses that should scare the pants right off you! Can you imagine a technology like this falling into the hands of a totalitarian regime?

4. Facial-recognition vehicle security system
U.S. Patent No. 7,602,947

This patent relates to a movable-vehicle security system that includes facial-recognition technology. The invention relies upon a scanner, such as a infrared camera, directed at the face of a person in the driver’s seat. The scanner produces output control signals that are applied to enable or disable operation of the vehicle, with or without a key. In one particular embodiment, the system includes a camera mounted below the roof and inside the rear window, directed at the rear-view mirror, and coupled to a facial-recognition computer, which in turn is coupled to an enabling element for the vehicle, such as a starter motor.

This type of innovation, which deals with security and enabling a vehicle only for the intended driver, seems less objectionable, at least at first. But the thought of having a camera on focusing on you in your car could be a little unnerving to some, perhaps many. Also, would the bank that lent you money to purchase the car be able to turn off your access to the car for failure to make a payment? These and other questions will need to be answered as facial recognition technology continues to proliferate.

5. System and method for performing rapid facial recognition
U.S. Patent Application No. 20100239130

As facial recognition technology continues to take hold it will only get faster and faster, making it more useful in far more contexts. This invention is an example of the need for speed in order to make facial recognition technologies more useful. Distributed facial recognition is performed by multiple feature recognition modules that are interconnected and scattered on the network, hence can greatly increase the recognition speed and decrease the requirements in hardware specification. Moreover, the recognition accuracy is further increased by employing an identification module to identify the recognition results.

The invention begins it work when the image capture device captures a facial image. The feature extraction module extracts features of the facial image to generate a set of feature data that is broadcasted to the response recognition computing units. The feature recognition modules then perform distributed facial recognition for generating recognition results as a response. The identification module identifies the recognition results to accomplish the recognition of an individual’s identity.

[Google_468_60]

6. Remote and digital data transmission and satellite location…
U.S. Patent Application No. 20100177193

This invention utilizes urban surveillance cameras for facial recognition. A photograph is taken and sent to the command center by means of a data link in real time. The photograph is sent to a global module which transmits it to a security filter for its analysis and once the information is confirmed it sends it back to the module, and from here it is sent to the individuals identification module. The system edits the photograph with all data acquired from an associated series of microphones. By coupling the photographs with sounds the system can recognize the sound signatures of such things as gunshots.

Again, the utility for this invention is clear. If you read the patent it explains that the goal is to assist with data collection for security personnel investigating kidnappings, for example. But with that amount of data collected and the transmission of data one has to be a little nervous. Can you imagine those who might want to tap into such data?

7. ATM with facial recognition features
U.S. Patent Application No. 20050167482

This invention relates to a self-service automated transaction machine that permits an authorized user to perform transactions therewith based on facial recognition. The machine includes one or more cameras from which one or more facial images are captured and which in turn are used in determining whether to grant machine access to an individual adjacent the machine. The machine may also include biometric type reading devices which are capable of reading a physical feature associated with a merchant user. These may include for example fingerprint readers, retina scanners, iris scanners, voice recognition features or combinations of any of the above.

To further the Commission’s understanding of the privacy issues inherent in the use of facial recognition technology, the Federal Trade Commission seeks public comments on the various issues, including but not limited to:

What are the current and future commercial uses of these technologies?
How can consumers benefit from the use of these technologies?
What are the privacy and security concerns surrounding the adoption of these technologies, and how do they vary depending on how the technologies are implemented?
Are there special considerations that should be given for the use of these technologies on or by populations that may be particularly vulnerable, such as children?
What are best practices for providing consumers with notice and choice regarding the use of these technologies?
Are there situations where notice and choice are not necessary? By contrast, are there contexts or places where these technologies should not be deployed, even with notice and choice?
Is notice and choice the best framework for dealing with the privacy concerns surrounding these technologies, or would other solutions be a better fit? If so, what are they?
What are best practices for developing and deploying these technologies in a way that protects consumer privacy?

Because all comments will be made publicly available on the FTC website, you should not include any trade secrets, confidential information, or sensitive personal information.

Public comments can be filed in electronic form by using this FTC form or in paper. Paper comments should refer to “Face Facts: A Forum on Facial Recognition — Project Number P115406” and include this reference both in the text and on the envelope. They should be mailed or delivered to the Federal Trade Commission at the following address: 600 Pennsylvania Avenue N.W., Room H-113 (Annex P), Washington, DC 20580.

The FTC is requesting that comments filed in paper form be sent by courier or overnight service, if possible, because U.S. postal mail in the Washington, DC area and at the Commission is subject to delay due to heightened security precautions.

The deadline for filing comments is January 31, 2012.

Gene Quinn Gene Quinn is a patent attorney and a leading commentator on patent law and innovation policy. Mr. Quinn has twice been named one of the top 50 most influential people [...see more]

Warning & Disclaimer: The pages, articles and comments on IPWatchdog.com do not constitute legal advice, nor do they create any attorney-client relationship. The articles published express the personal opinion and views of the author as of the time of publication and should not be attributed to the author’s employer, clients or the sponsors of IPWatchdog.com.

Join the Discussion

10 comments so far.

PTOExaminer
January 1, 2012 08:55 pm
“How can consumers benefit from the use of these technologies?”

I believe facial recognition software is good only for identification purposes (as opposed to verification). Verification (or authentication) requires 100% accuracy whereas identification, for many applications, need not be absolutely correct all the time. For example, Facebook’s facial recognition software helps users automatically tag friends in pictures – it is not the end of the world if it tags incorrectly. However, if a facial recognition based verification system thinks I am someone else and grants me access to secure systems then the end of the world may be closer.

I believe biometrics based security only appears to be more advanced than traditional approaches. This may be because we’re weaned to be amazed by the fact that no two people have the same fingerprints, or because Hollywood knows how to impress us with it. Traditional two-factor authentication is significantly easier and cheaper to implement (“something you know” & “something you have”) and has a proven track record.

Having created facial recognition security software, I honestly feel that it is a sham (as well as biometrics based security in general). There are just too many human variables to account for. There are just too many ways of circumventing such technology. I have seen too many companies “upgrade” to biometrics based security only to revert to previous security protocols months later because it just doesn’t work the way the salesman said it would.
Stan E. Delo
December 29, 2011 06:24 pm
EG-
Another feature of the American experiment that seems to have been forgotten now and then, is that the Constitution clearly states that the member States of the United States of America shall have their own rules of governance, and that the Federal government will respect their rights and wishes. In other words, State rights come first and foremost, given that the state laws do not violate any Federal laws. By the people and for the people, on a regional or perhaps on a larger scale if it seems to be warranted. The way Congress has been behaving these days though, I seem to find it hard to believe that they will actually do the right thing at the end of which day?

Best wishes for a brand New Year!
Stan~
EG
December 29, 2011 05:47 pm
Stan,

Thanks for bringing up yet ANOTHER GLARING EXAMPLE of why our Bill of Rights is important. The “McCarthy Era” in the 50’s was another regrettable instance of American government showing its ugly “Dark Side.” Our Founding Fathers were right: we want to minimize the intrusion of government in our private lives, not increase it as some would have you believe is best for us.
Stan E. Delo
December 29, 2011 03:28 pm
Gene-
Interesting post, which goes right to the heart of what the American experiment has obviously done so well in the past to make absolutely certain that Americans do not have their Constitutionally granted rights abridged due to the vagaries of what certain individuals in our government might happen to want to do for reasons that may not ultimately be in the best interests of our personal freedoms, and the protection from the abuse or subversion of those rights. I am reminded of the witch hunt conducted by Senator McCarthy towards the end of the Cold War, as part of the astoundingly ill-advised HUAC efforts to hunt down and “get rid” of *suspected* Communists living in the US at the time. They destroyed hundreds of careers with nearly NO evidence to support their claims, and a whole bunch of folks went to prison for many years because of the efforts of a few Senators, whether they actually deserved it or not..
EG
December 29, 2011 12:43 pm
BJR,

Excuse, I meant to say that de Toqueville said “absolute power will corrupt absolutely,” which is definitely what will happen when government isn’t made to “toe the line.”
EG
December 29, 2011 12:41 pm
“Stop reading George Orwell, and start reading the U.S. Constitution. The moment we as U.S. citizens stop understanding and believing in the Constitution, is the day we [lose] our rights.”

BJR,

I have read the Constitution (many times), and my concern about government abusing information it has obtained from its citizens is very valid (and the worry about “Big Brother” is very real). I’m not a libertarian (although would likely be characterize as a social/fiscal conservative), but de Toqueville was correct in saying that “absolute power will be abused absolutely” which is true when you have government not “toeing the line” strictly on our Bill of Rights because of expediency or an “emergency” (the forcing of U.S. CITIZENS of Japanese ancestry in concentration camps during WWII that was permitted under Koramastu v. U.S. is a glaring reminder of the “Dark Side” of our federal government). Like Gene said, anyone who believes you can trust government (and especially its officials/employees to “obey the rules”) is very naive.

The big problem here is that there is usually no recourse by the affected citizen to correct, stop, be remediec for, etc., the abuse by government offiicial/employees because of the doctrines of soverign immunity (under the Holmesian rubric of the “King can do no wrong”), qualified/absolute immunity, etc. For example, there have been repeated instances where the IRS is seized or garnished assets/property based on incorrect information it has (acts which would cause you or I to end of in jail or on the receiving end of a civil lawsuit), causing the affected person to be unable to use their property, and only after much suffering, was it discovered that the IRS acted incorrectly. But no apologize from the IRS for having unnecessarily harmed that individual, and no recompense from the IRS/government for the damage caused.

Our Founding Fathers put the Bill of Rights (First through Tenth Amendments) into our Constitution because of the abuses the English Crown of those Rights when we were “colonies” and the desire to avoid those. Even our Supreme Court has denigrated those Rights with the Kelo v. City of New London in allowing “takings” under the Fifth Amendment which essentially amounted to a transfer of private property from one private entity to another private entity. (The Fifth Amendment was intended to be a brake on such “takings” by government, yet the Supreme Court in Kelo essentially turned the Fifth Amendment on its head for the first time in my memory). The only thing that has changed since the Bill of Rights came in is that governmental intrusion into our lives has GREATLY INCREASED. That makes vigilance by us to hold government accountable to respect our Bill of Rights even more important now.

Also, as I stated, I didn’t object to the finger printing for the purposes of verifying who I was for the two tests. But by what “right” does government have to use that information for purposes unrelated to the verifying my identity for those two tests without my permission? The answer is it has NONE. Any waiver of such Rights must be “voluntary” (i.e., it cannot be “coerced”) and requires the permission having them.

You also state that you carry a “concealed weapons permit.” But if the Supreme Court hadn’t upheld the Second Amendment that DC case against efforts to exert gun control (including banning guns period), you wouldn’t necessarily have such a permit. I’m not an owner of any guns, but I do respect the Second Amendment saying that others may have them. Again, if you don’t force the government to “toe the line,” it won’t happen, and our Bill of Rights will be “flushed down the toilet.”
BJR
December 29, 2011 11:12 am
Gene,

As to your first comment, why are you upset that the FBI has a file on you then? What could possibly be the reason?

As to your second comment, I am one citizen in 300 million (counting just citizens of the U.S.). There is no way the U.S. government would have resources to watch me do anything. Even if these facial recgonition systems work in a fully autonomous manner, it still takes manpower to review the information gleaned. Further, I can’t stop the U.S. governement from taking my picture in a public forum, even on my front step. And this cannot be consided an illegal search or invasion of privacy. Stop reading George Orwell, and start reading the U.S. Constitution. The moment we as U.S. citizens stop understanding and believing in the Constitution, is the day we loose our rights. This has, admittedly, already happened due in part to the nation’s lack of vigilance.

As to your third comment, when I requested to sit for the bar, I gave the feds as much information as necessary to perform the background check, an no more. Likewise, I don’t give my name here because I do, in fact, relish in my privacy, and choose not to give complete strangers like yourself information about me that they don’t need to know. For all I know, people could be reviewing these comments and obtain enough information to do me harm. I care about my privacy as much as you or more. I’m just not stupid about it.
Gene Quinn
December 29, 2011 10:48 am
BJR-

First, I never said or implied that I am fearful of the FBI having a file on me.

Second, it seems pretty clear to me that you are extraordinarily naive. A government that spies on its people has the ability to turn into a totalitarian regime. The fact that you choose to ignore the teachings of thousands of years of history that prove that to be true is a YOU problem, not a problem for those that recognize historical realities.

Third, your comments suggest that you have nothing to hide and are not afraid of being an open book. Your comments suggest that you have no particular desire for personal privacy. At the same time you choose to use your initials on this forum. Seems rather hypocritical if you ask me.

-Gene
BJR
December 29, 2011 10:43 am
Gene at comment 1,

I thought when you requested to sit for the state bar you had to provide an FBI background check a a prerequisite to sit. If you didn’t want a file opened on you with the FBI, you shouldn’t have sat for the bar. Further, apart from the “slippery slop” arguement that all conspriacy therorist put forth, why are you so scared to have a file on you with the FBI. The guilty flee when none persue. I am glad the FBI has a file on me that shows I sat for a state bar, I have a valid passport, and a concealed weapons permit. It shows that I exercise my constitutional rights, and am a good enough citizen to obtain these additional rights.
EG
December 29, 2011 08:59 am
Gene,

Interesting that the FTC is interested in this technology. Biometrics (of which I would include facial recognition technologies) has been around for quite some time, incluidng for use in securely accessing bank accounts, fund accounts, etc.

Frankly, I’m more concerned with what the government will do with such technology in terms of privacy, includiing 4th Amendment issues. From personal example, I was fingerprinted twice, first when I was planning to take the LSAT, and later when I was planning to sit for the Virginia Bar Exam. I didn’t mind the fingerprinting to make sure it was me who were taking the respective exams. But what bothers me the most is that those fingerprints have likely ended up in some government database, probably of the FBI’s, and without asking for my permission to do so; that’s “Big Brother” 1984 style. And given the abuses that have occurred within the government of such information (incluidng SSNs) with no realistice recourse to get the wrong punished or rectified, that’s what scares me the most about facial recognition technologies.