What Can We Learn from the FTC’s Patent Assertion Entity Study?

Federal Trade Commission, Washington, DC.

Any day, the FTC will release the results of its multi-year examination of patent assertion entities (PAEs). Using its Section 6(b) compulsory process authority, the FTC has been collecting information related to PAEs, which the agency defines as companies that are in the business of buying and asserting patents. The FTC intends to use this information to publish a study that will examine how PAEs conduct business and to develop a better understanding of how their activities affect innovation and competition. FTC officials have suggested that the study results could help inform public policy decisions with respect to PAEs and intellectual property issues and pointed to Congressional interest as part of the justification for the study.

The FTC study will provide some potentially useful data with respect to PAE activity. But any efforts by the FTC or others to base policy decisions, including law enforcement priorities, principally on the study results would be seriously misguided. To understand why requires a basic understanding of statistical methods.

Broadly speaking, research can be classified as either exploratory or confirmatory. Exploratory research is often the starting point for research and is premised on general questions, rather than testable hypotheses. Exploratory studies seek to identify potential relationships between variables, which can then serve as a basis for further investigation. Definitive answers are usually not possible from exploratory research alone.

Confirmatory studies, which are sometimes called hypothesis testing, are used when the researcher already has a testable theory, and the objective of the research is to see if the theory is valid. Well-designed confirmatory research meets rigorous conditions and can provide definitive answers to specific questions.

The FTC’s PAE study is exploratory. The specific issues the FTC intends to address in the study are classic exploratory-type questions, such as how PAEs organize their corporate structure and what types of patents PAEs hold. In addition, the number of participants in the study—25 PAEs and 15 other firms—is too small to generate the definitive conclusions we seek from a confirmatory study.

The FTC seems to have understood the exploratory nature of its study. In a submission to OMB, the FTC explained that the study would consist of two “case studies,” which are a type of exploratory study only relevant to the cases or the samples of data studied and whose findings cannot be extrapolated further. And like all exploratory studies, the FTC acknowledged that “the findings of the case study should be viewed as descriptive and probative for future studies.” This of course means an introductory step, not one that will—or can—provide definitive results.

Another important aspect of the FTC study is that it is not intended to describe all PAEs, or the overall relationship between PAEs and operating companies. It’s very unlikely that obtaining data from just 25 PAEs will provide a representative sampling of PAEs given that the universe of PAEs is largely unknown and probably very diverse. Likewise, all of the non-PAEs in the study are from the wireless chipset industry, so the study’s results cannot be generalized beyond that industry.

The study designers also recognized these limitations. They explained in their OMB submission that because of sampling concerns, “the findings of [its] case studies will not be generalizable to the universe of all PAE activity.” Likewise, they explained that “it is not statistically valid to extrapolate the findings from the case study to the population of PAEs, manufacturers, or NPEs.” In other words, the study’s findings are limited to the surveyed firms, and not to PAEs or manufacturers generally.

In sum, the study designers recognized that the PAE report is unlikely to offer any conclusive results as to PAE activity. But they might be used later to help generate additional (confirmatory) studies to reach statistically-significant conclusions on which public policy could be based. So far, so good.

The problem is that in my experience both lawmakers and regulators routinely ignore important statistical limitations of federal studies. I say this with the experience of having worked for over 20 years as a federal government statistician. All too often policymakers use federal studies in ways beyond their intended purposes, with the result that legislation or regulation may be based on a flimsy and potentially inaccurate understanding of the underlying problem or the costs or benefits of proposed government action.

That the FTC’s study designers did an admirable job explaining many of the limitations of their study unfortunately tells us little about whether the final report will show similar restraint or whether policy makers—including the FTC itself—will use the study results in ways contrary to what the study was designed to do. Regardless of the report’s findings, one should be suspicious of those who claim that the study supports a particular policy proposal or enforcement initiative. Any such use is based on a failed understanding of the value of the data, and the corresponding limits of the study design.