Some patent applications are difficult to get agreement on. The examiner won’t allow and the applicant won’t abandon. The net result is that office actions and responses go back and forth with no apparent resolution in sight. We propose that progress with these difficult patent applications can be tracked by looking at two separate but interrelated metrics, “applicant effectiveness” and “examiner effectiveness”. These two metrics can then be used to diagnose and correct problems in patent prosecution and examination.
“Applicant effectiveness” is the fraction of office actions that have a new piece of prior art cited against the first independent claim. “Examiner effectiveness” is the fraction of responses that have a new amendment to the first independent claim. These metrics are described as “effectiveness” since they are measures of how skilled applicants and examiners are in convincing the other party that they are right. If an examiner cites a new piece of prior art against an independent claim, then it was the applicant who was effective in his or her last response. If an applicant amends an independent claim, then it was the examiner was effective in his or her last office action. We limit out attention to the first independent claim since this is the bottleneck in getting all claims allowed. It also saves time in collecting the data.
200 random patent applications
Figures 1A and 1B illustrate how applicant effectiveness and examiner effectiveness vary with each successive office action for a set of 200 randomly selected patent applications. The applications were filed between 2005 and 2007 and most have been recently allowed or abandoned. Hence these graphs represent the current state of affairs in patent examination. Data was gathered by reviewing office actions and responses in the USPTO’s Patent Application Information Retrieval system (PAIR).
Each bar in Figure 1A shows the percent of office actions that cite new art on top of the percent of office actions that have all claims allowed. This gives a total effectiveness of the applicants’ prior submissions. In the case of the first office action, we considered the references the applicant cited in an initial information disclosure statement (IDS) as “prior cited” art.
Total effectiveness starts out at about 80% and then gradually decreases to about 50% by the fourth office action. Allowance rates, on the other hand, start out relatively low at 15% but then increase to 25% to 35%. These low numbers indicate that there is substantial room for improvement. Determining why these ratios are low and evaluating alternative strategies for increasing them would bring substantially lower patent prosecution costs and lower pendency rates to applicants, particularly those with large portfolios.
Figure 1B shows the total examiner effectiveness for the same set of 200 randomly selected patent applications. Each bar shows the percent of responses with amended claims on top of the percent of abandonments. Each applicant response to an examiner office action was inspected to determine if the first independent claim was amended or not. A judgment call was made as to whether or not the amendment narrowed the claim to overcome a prior art rejection or if it was simply to correct a clerical defect or statutory defect, such as a 101 or 112 rejection. Amendments that were made to correct clerical or statutory defects were not counted towards measuring how effective an examiner was in convincing an applicant that a claim was not allowable over the cited prior art.
The effectiveness of examiners is about the same as the effectiveness of applicants. Initial office actions are about 80% effective in getting an applicant to amend or abandon. Later office actions are only about 60% effective. Here again, there is substantial room for improvement.
37 difficult patent applications
Figures 2A and 2B illustrate a block of 37 difficult patent applications. These applications were all filed by a single applicant in a single art unit with filing dates in the range of 2005 to 2006. The applicant is a major bank. The art unit is computer implemented business methods. These 37 applications have had a total of 180 office actions since active examination began. 26 are still pending. 10 have been allowed and 1 has been abandoned.
Figure 2A shows that the overall applicant effectiveness is significantly below average. It’s not due to the inability of the applicant to overcome a given rejection, however. It’s due to the failure of the applicant to convince the examiner that the claims are allowable. Additional analysis of the file wrappers of these applications revealed two areas where improvements could be made that might address this problem. The first is more consistent and thorough prior art searching before filing. The second is more consistent assignment of attorneys to individual cases.
This applicant has a much lower rate of initial IDS filings than average. Applicants in our random sample of 200 applications filed initial IDSs 80% of the time. This applicant filed initial IDSs only 50% of the time. This means that the applicant is largely blind as to what prior art is out there that the examiner could rely on. Hence examiners find it relatively easy to find new art to cite against the applicant’s amendments or arguments.
This applicant also has a much higher rate of turnover of attorneys than average. We found a 50% turnover in attorneys from one office action to the next. The average turnover for our random sample of 200 applications, however, was only 10%. This means that the applicant’s attorneys are focusing only on the office actions in front of them and not on any sort of long term strategy to get claims allowed.
The applicant can test the validity of this analysis by randomly selecting a subset of its pending applications, applying corrective action, and monitoring its “applicant effectiveness” to see if there is an improvement. Significant results should be apparent in a few months.
Turning to the examiners, Figure 2B shows that examiner effectiveness for these difficult applications is also lower than average. The problems appear to be related to prior art and turnover, but in a very different manner than for the applicant.
The office actions in these cases had very few non-patent literature citations. This may be fine for most art units, but we propose that it is inadequate for business method inventions. Stronger art, such as peer-reviewed literature from financial journals, may be much more effective at convincing this applicant to significantly amend its claims or even abandon. Finding and reviewing scholarly literature will require more effort from examiners, but the benefits would be immediately apparent from higher examiner effectiveness, higher abandonment rates and reduced backlog.
Regarding turnover, it appears as if the examiners in this art unit are surprisingly well experienced. Examiner turnover from one office action to the next was less than 1%. The average for our random sample of 200 applications was 10%. This means that these examiners are much more experienced than the applicant’s attorneys and may be taking a lot for granted in their office actions. A potential solution to this problem might be a more thorough explanation of the rationale for a given rejection, particularly where considerations specifically related to business methods are involved.
Some patent applications are difficult to examine. The effectiveness of both applicants and examiners in advancing prosecution can be independently measured by looking at how each party responds to the submissions of the other. If an examiner cites new art against a claim, then the applicant was effective in overcoming the examiner’s prior rejection. If the applicant amends a claim, then the examiner was effective in showing that the applicant’s prior claim was anticipated or obvious. Effectiveness data can then be used to diagnose where the difficulties are in patent examination and then to evaluate corrective action. This can lead to lower legal costs, faster agreement on allowable claims and a more efficient patent examination process.
 Other researchers, such as Cotropia, Lemely, and Sampat, interpret examiners citing new art over applicants’ IDSs as indicative of examiners’ “preference” for their own art. We, on the other hand, interpret it as an indicator of how well the applicant has presented his or her initial case for patentability.
 The final bar on the graph labeled 4+ shows the combined effectiveness of the fourth, fifth, sixth and higher actions.
 This is based on our analysis of the file wrappers and is also consistent with other researchers.
 See “KSR Training Examples: TC3600 Business Methods”, USPTO examiner training supplement found at http://www.uspto.gov/web/offices/pac/dapp/opla/ksr/ksr_3600_bm_supplement.pdf