How to extract FAR clauses from RFPs in under 3 minutes
If you’re a proposal manager, contracts manager or program manager working US federal programs, you’ll know the importance of the FAR (Federal Acquisition Regulations) and DFAR (Defense Federal Acquisition Regulations).
You need to carefully review certain FAR / DFAR clauses for compliance, especially those concerning OCI (Organisational Conflict of Interest), Security and Intellectual Property.
Prime contractors also need to manage any flow down clauses that apply to sub-contractors. Equally, sub-contractors need to clearly understand what flow-down clauses apply to them.
So contract managers and program managers manually comb through and review the FAR clauses. A very time consuming, but critical process. This applies after contract award when the contract kicks off, but also during the proposal process.
To help the process, most contract managers create a type of FAR requirements matrix, often using a spreadsheet. This allows them to track compliance with the FAR mandates. It also helps manage the various stakeholders or functional groups delivering the program. The FAR requirements matrix is a critical tool. It often dovetails into a responsibility matrix, sometimes called a RACI matrix.
Recently we noticed something interesting. Some of our customers were using VisibleThread Docs to automatically create the FAR matrix using VisibleThread custom dictionaries. They used dictionaries to search for specific FAR and DFAR clause numbers.
Now this was a really interesting use case, so we were curious.
- How are FAR clauses checked?
- Can these checks be automated & what are our customers doing?
- What kind of time savings can you expect for automatic extraction?
In this blog post, I’ll share what we found. If you’re a contracts manager, program manager or proposal manager, you’ll find it interesting.
How are FAR/DFAR clauses checked?
First, a little context; many FAR clauses are standard and tend not to change. These are typically pretty easy to identify as they have a fairly uniform structure in the RFP paperwork.
However, contractors pay particular attention to certain clauses including;
- Conflict of Interest FAR clauses,
- IP (Intellectual Property) FAR clauses,
- Security related FAR clauses.
These same considerations apply for DFARs too.
Extracting OCI (Organisational Conflict of Interest) FAR clauses
The FAR is large currently numbering 672 different clauses. So, for the sake of explaining how to automate the extraction, we’ll focus on a subset of the FAR, those clauses that are related to OCI. There are 15 of those.
The FAR breaks OCI concerns into 3 broad categories; biased ground rules, impaired objectivity and unequal access to information. An example of biased ground rules might be if the contractor had some part in writing or influencing the statement of work. For more detailed examples, see here.
And to give a sense here are some of the specific FAR clauses for OCI:
For the full list of the 15 OCI clauses go here: https://www.acquisition.gov/far/html/Subpart%209_5.html
How to scan multiple documents for OCI clauses?
So, back to our original question; can we automatically extract FAR clauses from RFPs in under 3 minutes?
Absolutely, here’s how. We just need to create a ‘dictionary’ of search terms that reflect specific FAR clause numbers. Then we can run it against a collection of docs. Here’s the result in the case of the 15 OCI clauses:
In the example above, we analyzed 7 different docs. We searched for OCI terms like ‘9.501’, ‘9.502’ and so on. This ‘birds eye’ view means we can instantly see the location of the FAR clauses across not just one doc, but a collection of docs. That’s pretty powerful.
Here’s what we can immediately infer in this case:
- 5 of the 7 docs contain FAR references. The numbers show frequency of occurrence for each doc.
- We can see which docs have which particular FAR clauses. For instance, I’ve flagged in red an RFP that has a total of 9 occurrences. This comprises 3 references to 9.505-1, 2 to 9.505-2 etc.
- 2 of the 7 docs contain no FAR clauses at all.
- On the other axis, certain FAR clauses did not show anywhere in the 7 docs. These missing clauses include; 9.501, 9.502, 9.504 etc.
So far so good. We can now easily see which of the 7 docs contain specific FAR terms. Next, how do we extract those clauses from the doc itself to arrive at our FAR Matrix?
Extracting the OCI Clauses to a Matrix
For this, we use the same dictionary but this time, point it at the doc only. This view shows the specific occurrences of the OCI related FAR clauses and shows the content in the doc too.
As you can see, this view lists every paragraph where the clause occurs by page. The FAR number is highlighted using color-coding for easy recognition.
We’re almost there, we just need to get this into excel form. So, just click ‘Export to Excel’ and you have a FAR matrix. Here’s what that output looks like. It’s very similar to the previous view except it’s an Excel spreadsheet. This provides a lot of flexibility since you can easily add new control columns or sort/slice the data.
What about the rest of the FAR & the DFAR?
Can we generalise this approach to the rest of the FAR and DFAR? Certainly, let’s see how.
Here’s the same view as before showing the collection of 7 docs.
The only difference is that this time we’re using a much bigger dictionary of search terms. This full FAR dictionary contains all 672 FAR clauses.
The really nice thing is that we were able to switch the dictionaries with just 1 click, clicking ‘Use different dictionary’. And exactly like our OCI example, we can analyze a collection of documents OR deep dive into a specific doc and extract the FAR matrix for that doc too.
Here, for example, is a snippet of the Excel output when we apply this full FAR dictionary to our prior RFP example.
What kind of Time Savings can you expect?
The single biggest advantage when automating this extraction process is time savings. Let’s quantify that.
Here are the steps you take when extracting manually:
- Step 1: Open the doc in PDF or MS Word
- Step 2: Search for the specific FAR clause using standard ‘find’.
- Step 3: Check if there are any hits. If there are, then copy and paste the relevant paragraph of text into the spreadsheet.
- Step 4: Repeat step 2 & 3 until you’ve checked all the FAR clauses.
Time Savings Scenario 1: Checking 15 OCI clauses
So let’s assume you have the OCI clause list that we referenced before. You need to search for 15 specific OCI terms; including ‘9.501’, ‘9.502’ etc. Then if you see hits, you need to copy/paste to the FAR matrix. The RFP that we analyzed previously had 106 pages and 5 of the 15 terms had hits.
Time taken =
- Step 2: 15 x 30 seconds = 5 minutes (450 secs) . Assumes it takes 30 seconds to enter the term and determine if there are occurrences or not.
- Step 3: 5 x 2 minutes = 10 minutes. Assumes it takes 2 minutes to identify and copy/paste the various clauses found for each FAR term.
This gives us a total of 17.5 minutes for a single 106-page doc with just 15 terms to check.
With the automated approach, steps 2 and 3 took 1 minute. Recall that we found 5 terms occurring in the doc immediately and at a single click we had the output in a spreadsheet.
So for this scenario, it’s:
17.5 minutes (for manual) vs. 1 minute (for automated).
Time Savings Scenario 2: Checking 672 FAR clauses
Using the same approach as scenario 1 but this time with the full FAR, it’s a lot more time consuming. For our 106 page RFP doc, we had 155 hits, and we need to check for 672 FAR clauses.
So, here’s how our calculations play out.
Time taken =
- Step 2: 672 x 30 seconds = 336 minutes or 5.6 hours (20,160 secs) . Assumes it takes 30 seconds to enter the term and determine if there are occurrences or not.
- Step 3: 155 x 2 minutes = 310 minutes or 5.1 hours. Assumes it takes 2 minutes to identify and copy/paste the various clauses found for each FAR term. NOTE: 155 of the 672 clauses had hits.
This gives us a total of 646 minutes or 10.76 hours for this 106-page doc.
If we use the automated approach then steps 2 and 3 become extremely fast. In our case, in fact, it was 2 minutes. We were able to extract in the output to a spreadsheet in a single click.
So for this 2nd scenario, it’s:
10.76 hours (for manual) vs. 2 minutes (for automated).
The value of reliably extracting the full content is that it dramatically shortcuts the time-consuming copy/paste process to populate the FAR spreadsheet.
- FAR and DFAR clauses are critical to track.
- Contract and Program Managers often create FAR matrices.
- Creating this kind of matrix is critical but very time-consuming.
- You can automate the extraction process and save days per contract, without any risk of missing requirements.
I hope this post was helpful. Good luck with your contracts.
Acknowledgments: I am very grateful to a number of 3rd parties who helped my understanding of the FAR and clarified many of the challenges for contract managers. In particular, big thanks to Maxine Tolbert of SBC Solutions, Carl Gouaux of CWG and associates, Melissa Howell of Government Acquisitions and Bridget Anderson of Deltek. Thanks guys for your insights.