During early 2011, FIT systems, a Florida based SDVOSB (service disabled veteran-owned small business) asked these questions:

1. Can we quickly determine whether to bid / no-bid or team
2. Can we save time and avoid reading poorly qualified solicitations?
3. Can we compare multiple opportunities for ‘good fit’ in a few minutes?

In February 2012, Jim Creutz, Business Development Director at FIT Systems and I got chatting. Jim shared what they were doing with VisibleThread to support these efforts. I was intrigued.

Of course, they were using VT for the normal stuff; making better quality props with readability analysis, generating starter compliance matrices in minutes, and driving more efficient pink and red team reviews. But, Jim showed me how they were also using VisibleThread to augment the very early stages of Capture Planning too to help bid / no-bid and teaming decisions. Now this was new!

They had started using language analysis to scan solicitation docs for what Jim called ‘affinity’. Basically, he was identifying the right opportunities to bid or team on with language indicators. As I blogged here on win themes, it is always amazing to see how customers apply VisibleThread, so this was another really interesting case.

So, if you are a Capture Manager or Proposal Manager involved in Business Development at all, you will find this post useful. If you want to see this technique in the flesh, Jim shared what he is doing in a webinar we ran on the 10th April. Click here for the slides and a full recording.

I am very interested to hear whether you have attempted to analyse solicitation docs using a similar approach? If so, how did it go? Let us know your thoughts on this approach. Do you think it can work in your case? Is it helpful? Would it save time? Post comments and let us know.

So, let’s get back to Jim. Here’s how he did it.

Step 1: Define Affinity Characteristics & weightings

Jim set up what he calls affinity characteristics in a spread-sheet.

Here are three categories he defined:

• Experience
• Technical
• Business Attributes

Under each category, he listed phrases and terms that would indicate ‘fit’ or ‘lack of fit’. Here are a sample of some of the characteristics he defined:

In total, Jim defined a total of 77 technical indicators to scan for, 23 individual business indicators, and 15 experience indicators.

Next, for each indicator, Jim assigned a weighting. Positive weightings between 1 and 5 indicate good fit, with 5 as best fit.

Negative weightings indicate poor fit. These ranged from -1 to -5, with -5 as worst fit. So, now Jim had an Excel spread-sheet with categorized indicators and weightings.

Jim also defined a categorized ‘Concept List’ in VisibleThread so that he would be able to scan for the indicators. The concept list will allow us to explicitly scan the documents later as part of step 3. So, onto the next step, uploading docs…

Step 2: Take your docs from GovWinIQ or FedBizOpps and upload to VisibleThread

Jim’s company subscribes to GovWinIQ from Deltek. This means they have a steady stream of solicitations coming in on a daily basis. For those not using Deltek’s product, you can still get your solicitations from FedBizOpps of course.

So, Jim takes the solicitation docs that are worth a second look and uploads them to a VisibleThread folder.

He tends to name his folders either by date range or, if a large number are coming at him from a specific agency/dept., he’ll add the agency name. So he might have folders called ‘2012 Week April 2  - Solicitations’ or ‘Army Solicitations – April 2012’ or ‘Navy Solicitations – April 2012’ etc.

By adding to categorized folders like this, over time he can assess trends also. But that’s a separate benefit.

Step 3: Run a Scan

Next Jim runs the scans against the doc. You might recall back in step 1, Jim had defined a ‘concept list’ holding his technical, business and experience indicators. So, a scan is really a way to uncover any references to those indicators.

So, scanning is pretty simple. He does this:

1. He selects the folder. He clicks ‘Run Concept List Analysis…’, selects his ‘affinity’ concept list and then.
2. He reviews Concept Coverage across the uploaded docs. In other words he is presented with a matrix type view.

The screenshot below is an example of how this looks in VisibleThread. Listed across the top are the documents; RFPs, SOWs and PWSs for candidate solicitations.

The left shows the indicators; technical business and experience. Each column represents a solicitation document. The numbers indicate coverage (number of hits).

Step 4: Crunch the numbers in Excel

Finally, Jim now takes the above information and exports it to Excel.

He places it in the context of a detailed spread-sheet he has developed.

Jim’s final output is what he calls a ‘normalized score’. This is a calculated number based on the sum of each attribute’s Frequency Density X Weighting.

Now this may sound a little tricky, but here’s how he did it:

1. Take the weighting as outlined earlier, 1 to 5 for positive, -1 to -5 for negative.
2. Calculate Density by taking the total word count in the document and devising a % based on (number hits / total word count *100). This gives him a density percentage value.
3. Multiple the weighting against the density percentage value to give a normalised score for each attribute.

Here an example Excel report for one of Jim’s solicitations:

Jim has also developed a bunch of excel sheets that can show how all the solicitations compare graphically.

Here we see 6 solicitations compared by ‘Business Attributes’ using one of Jim’s comparisons. The green checks should be considered further. The ‘question’ one may be worth a read. We should likely pass on the red x solicitations.

If you want to learn more on this, Jim shared what he is doing in a webinar we ran on the 10th April. Click here for the slides and a full recording.

What has changed for Jim and his team?

The business benefits for Jim are very clear:

1. Time savings – Every day one of Jim’s colleagues had been spending between 1 and 2 hours reviewing new opportunities. Now that time is cut, because 1.) they are reading fewer more qualified solicitations and 2.) it is faster to identify exactly where the hot buttons are.
2. Improve Win Rate - Better decisions around the right bids to pursue – using objective metrics for ‘apples to apples’ comparisons of multiple opportunities.
3. Pass Fast on inappropriate stuff - Faster turnaround time in determining when to dig deeper and when to pass. If you know when to pass it allows you focus on better qualified opps.
4. Better win theme development - This is a side effect from Jim’s work. Because he’s got a more detailed sense of what the hot buttons in the solicitations are, he can now do a much more accurate job of establishing win themes and items to emphasize in the repsponse.

Summary Takeaways

I will leave you with some of my own summary observations:

• Jim has shown how automation can drive very quick qualification cycles. While his math skills are to be applauded, any proposal shop could apply this with VisibleThread. Drop us a note if you want help.
• Jim has evolved one way to do the task that is performing well ‘under fire’, and in very timeframes of under 5-7 minutes for collections of quite large docs.
• The business benefits of chasing the right opps are profound.

So, to see it in action, see Jim’s webinar slides and recording.

——-

If you want to see how we scan documents generally, check out these 3-minute demos to get a good sense.

If you want to see how you can do plain language scans or explore some of our other capability using some of our pre-canned pdfs/docs, just sign up for a no-obligation 30-day free trial of VisibleThread here: http://www.visiblethread.com/products/evaluate/.

OR drop us a mail and we can work directly with you.

And we continue to work closely with proposal managers who manually create the compliance matrix. Depending on size, they spend anything from several hours to days; creating it, honing it, getting it right.

Of course, no one reading this post will be too surprised at the time proposal managers spend creating a strong compliance matrix. After all, if your prop is non-compliant, you may as well pack up and head home. From the perspective of government agencies and commercial entities, a non-compliant response can be a great excuse for culling the field.

So back in early 2011, we had some questions:

• Can we automate some or all of the process? if so which parts?
• Can we bring initial creation of the compliance matrix, down from many hours to minutes, (well actually under 10 minutes to be precise)?
• If we automate the process can we keep it simple? Is it possible with no ‘prep work’?
• If we automate the process, is the output readable?
• Can we ‘shred’ both PDF and word formatted solicitations?

We wanted to answer these questions with real-world data and started down this path in early 2011. We are fortunate; we work with some of the biggest federal contractors (>$10b/annum revenue), right down to small businesses. Roughly speaking about 75% are government contractors, the remainder commercial. So we had a good test bed.

We have come a long way on our journey, so I wanted to blog on our findings and answer the questions posed above. We have learned a lot. You may find it useful.

What do I mean by compliance?

Shipley defines compliance like this; ‘you have agreed to meet all requirements as asked in the bid request’. I like this definition. It is clear; it leaves no room for vagaries. It also aligns closely with the view held by the close to 100 organisations we have worked with over the last year. So, let’s go with this definition.

I should also say that compliance is not responsiveness, that is; compliance is not about communicating your differentiation. Larry Newman of Shipley wrote about this topic here; Compliance vs. Responsiveness. If you haven’t read it, you should.

So, it’s compliance with requirements that we’re interested in. Can we save time with automation?

General Findings

To help answer our questions, we took a sample set of data.

Who was in the sample?

1. Our sample included 28 organizations
2. 72% (20) of these are federal contractors
3. 26% (8) are primarily chasing commercial business
4. All 28 are North American based.

Who’s using a compliance matrix?

1. Of our total sample, 79% (22) use a compliance matrix in their proposal development process.
2. All 20 of the federal contractors use a compliance matrix.
3. Of the 8 commercial vendors, only 2 use a compliance matrix and it depends on the bid size.

Government / Federal Findings

For the 20 federal contractors, is the process manual or automated?

1. 17 of the 20 federal contractors create the matrix manually and internally.
2. Of the remaining 3, 2 use automation in some form. Both have internally developed a set of MS Word Macros. These macros identify ‘shall’, ‘will’, ‘must’… and create a first pass matrix in excel.
3. 1 of the 20 federal contractors outsources the creation of the compliance matrix to a 3^rd party for a fee based on number of pages in the solicitation.
4. 100% of the federal contractors create the compliance matrix either as the very first or second step of their proposal development process.

How do people create a compliance matrix?

If we consider the 20 federal contractors, we see a similar approach.

It begins with a formatted grid. In most cases, this is a Microsoft Excel spread sheet. The grid typically contains these columns:

• From the solicitation doc; Col 1: ‘Page no’, Col 2: ‘Para’, Col 3: ‘text extract’.

• From the response; Col 4: ‘Page no / Para no’, Col 5: ‘volume’. Sometimes we have seen additional headings such as: ‘Compliant (yes/No)’ and ‘Person responsible’.

These are the common steps we see when creating the matrix:

1. If the solicitation is in PDF format, convert to MS Word.
2. In MS Word, using search, identify all occurrences of ‘shall’, ‘will’, ‘must’ and any other indicators. In some cases, this is restricted to only looking at certain sections, for example L & M.
3. For each occurrence found, populate a row in the compliance spreadsheet. This will include the page or paragraph number and the text of the compliance statement.
4. Once all occurrences are in the spread sheet, then for each row, fill in the details relating to where in the response we meet the requirement.

So, can we create a first pass matrix in under 10 minutes?

It would be naive to suggest that tools could magically crank out a ‘fully-baked’ compliance matrix.

But… could we cut out a lot of the drudge work? cut several hours down to a few minutes? Well the good news is yes, read on.

Here are the original questions we asked, along with some answers:

Can we automate some or all of the process? if so, which parts?

We have shown that steps 2 and 3 are good candidates for automation.

As of Feb 29^th 2012; from our original federal sample of 20 federal contractors, 8 now regularly use VisibleThread for performing steps 2 and 3. 1 is actively comparing the results of their own internal word macros with VisibleThread outputs and considering a change.

Can we bring initial creation of the compliance matrix, down from many hours to minutes, (well actually under 10 minutes to be precise)?

We have seen during the course of our studies that with automation, the total time for steps 2 and 3 above is between 3 and 5 minutes. This consists of uploading a PDF or MS Word solicitation, selecting the ‘dictionary’ of terms to scan with (e.g. ‘shall’, ‘will’), and generating a first pass excel spread sheet.

If we automate the process can we keep it simple? Is it possible with no ‘prep work’?

One of the key risks when automating any parsing process is this; users often need to ‘prep’ the automation. This often involves some kind of special massaging or formatting of the solicitation content, so that ‘tags’ or some other delimiter helps the parse algorithms.

Often this results in nothing more than automation getting in the way of the process and such products can become ‘shelfware’. So, complexity is the enemy.

We prioritized simplicity over sophistication. For instance we only output page no, not paragraph number, but this means the output is very accurate and extremely fast. We were able to generate a compliance matrix in 3 clicks / steps.

• Step 1: Upload & analyze the PDF or Word doc (Time required 30-90 secs, hits 90 secs for lengthy docs in excess of 200 pages)
• Step 2: Run specific Compliance scan (Time: 10-20 secs, depending on doc size)
• Step 3: Export to Excel (Time: 5-10 secs, depending on doc size)

If we automate the process, is the output readable?

This was a big concern. If you automate an initial document parse, how will the output show? Will it be readable?

We went through 4 significant algorithmic tweaks to get this right. Our first tests back in early 2011 resulted in some pretty scrappy outputs. In the early days, probably the biggest mistake we made was how we ‘chunked’ (parsed) the text. We didn’t provide enough context text in our outputs. We got some really great feedback from a few major beltway integrators who tried it. This helped us tweak the algorithm, so these days the outputs strike the right balance in a majority of cases.

To give a flavour, I took an arbitrary PWS (Performance Work Statement) issued on FedBizOpps just last week (Feb 22^nd), uploaded it and did a quick shred. (Total time: 90 secs)

Here is a screenshot of the resulting first pass compliance matrix:

- Can we ‘shred’ both PDF and word formatted solicitations?

On this front, we wanted to make sure that we could handle PDF and Word docs equally well, with sensible outputs. Technically speaking, while the parsing mechanics are slightly different, our results show very readable exports in both cases.

Summary Takeaways

I will leave you with some summary observations:

• Certain of your peers already have automation internally developed that helps efficiency.

• Tools are not the panacea, but the right tool definitely can save you a lot of time.

• We have evolved one way to do the task that is performing well ‘under fire’, and in short timeframes of under 5 minutes for quite large docs.

• You don’t always need a compliance matrix. For example, commercial proposals generally don’t require it. This is due to the small size and fixed format of responses.

If you want to see how we scan documents, check out these 3-minute demos to get a good sense.

If you want to see how you can generate a first pass compliance matrix in under 10 minutes, or even just to play around with some of our pre-canned pdfs/docs, just sign up for a no-obligation 30-day free trial of VisibleThread here: http://www.visiblethread.com/products/evaluate/.

OR drop us a mail and we can work directly with you.

“To ensure optimum delivery of services for its membership, Mini Healthcare of Elbonia will leverage the administrative and legal structure of its parent company, Maxi Healthcare, Inc. to administer a cost-effective combination of local plan operations and shared corporate resources that ensure a person-centered, outcomes-focused approach to delivering managed care services.”

It was taken from a proposal he was reviewing (actual names removed). You should take a look at the entire thread; it has a lot of good stuff around clichés. As author of the Shipley proposal guide, Larry is extremely well qualified on the subject of ‘trash talk’ in proposals.

Good writing, an on-going challenge for proposal professionals.

So, I wanted to blog on the topic of readability metrics, and how they can help eliminate this kind of rubbish.

If you work proposals as a proposal manager or team reviewer, you likely spend a lot of time editing contributions from colleagues.

Similar to Larry’s example, you will see marketing boilerplate filled with clichés and trite language, commonly in executive summary sections. You will see technical contributions from SMEs (Subject Matter Experts), often dense and hard to understand.

It is likely you are spending valuable time tidying the content, editing it out. But if you’re anything like our customers, you face these two issues:

1. Not enough time, since editing content is hard, at the end of a development cycle you don’t have enough time to sufficiently fix it. So, your company ends up submitting sloppy responses, despite your best efforts.
2. Offering feedback is emotive. If you do offer feedback to colleagues on their writing, it is by definition subjective, and a very delicate conversation. Here’s how it often goes:
   • You say: ‘I’m not sure the way you’re writing this is all that clear’
   • Your SME hears: ‘Your content sucks’.
   • Your SME is thinking: ‘I know what I’m talking about; you’re just glorified window dressing. I’ve done the hard work!’

We work with many proposal managers who privately admit frustration with this status quo. Some of these same people use VisibleThread scans to help unblock the emotive dynamic in 2 above.

Here’s how. Imagine we change the conversation to: “32% of the sentences in your content are longer than 25 words, whereas winning proposals tend to have 5% or less”. Then, metrics become a tool to drive way better quality in an objective way. Indeed, metrics are especially useful when dealing with technical SMEs. They respect metrics way more than opinion.

What are these readability metrics and how to apply them?

So, let’s revisit Larry’s example statement. I repeat the example here for ease. Below, you will see the metrics:

“To ensure optimum delivery of services for its membership, Mini Healthcare of Elbonia will leverage the administrative and legal structure of its parent company, Maxi Healthcare, Inc. to administer a cost-effective combination of local plan operations and shared corporate resources that ensure a person-centered, outcomes-focused approach to delivering managed care services.”

Sentence Length: 50 Words

Clichés: Multiple, including: ‘outcomes-focused approach’, ‘person-centered’ etc.

Passive Voice: none

So looking at the metrics, we see; long sentences and clichés.

If there is any positive in Larry’s example, it is the absence of passive language. Unfortunately, even without passive language, a sentence of 50 words is a disaster. You need to put it on a diet and bring out the critical customer benefits, ideally using bullets or lists. So, identifying long sentences is our first port of call.

Now let’s say you have a lengthy proposal. If you scan the full proposal and identify all examples of long sentences and passive voice, then you can prioritize areas for edits. This saves review time.

So, this is exactly what our customers are doing with VisibleThread. They upload proposals at pink or red team stage. In minutes, they can review the statistics, pin pointing long sentences and passive language. This forms the basis of rapid prioritization of edits.

Sharing the metrics with SMEs and marketing teams means that the quality of base contributions starts to improve too.

Let’s see how this works in reality.

To show you how this works, I took 6 recent proposal documents from 4 contractors (3 federal, 1 commercial). I uploaded them to VisibleThread for analysis.

For fun and so that we have a reasonable benchmark of ‘good writing’; I analysed Isaac Asimov’s; ‘The Best of Isaac Asimov’ alongside. (Worth a read if you have a moment.)

So, here are the long sentence statistics. The rank column gives you a fairly clear understanding of best and worst performers. The % measure is the proportion of total sentences exceeding 25 words.

Based on this report, we should immediately pay closest attention to ‘Contractor 2: XXX-Combined NearFinal.doc’ and ‘Contractor 4: T-1 Vol 2 TechnicalPart2.docx’.

Uncovering long sentences is the first step to cleaner and tighter docs.

Once you have long sentences down to a nice 5% or below, then it’s onto passive language removal and so on down the line.

Summary Takeaways

Readability metrics have these advantages:

1. Rapid Insight to Weak Content: They provide rapid insight into the weakest content in a proposal much sooner than manual review.
2. Encourage better writing from colleagues: They can help your colleagues tighten their submissions, so that ultimately a better quality response is submitted. With tighter inputs, you have more time to focus on being truly responsive to customer need.
3. Remove Subjective and emotive discussion, introduce objectivity: You say to me: ‘I’d like to suggest some edits’. As an SME, I hear ‘Your content sucks’. As an SME, I say to myself: ‘well, you can take a flying leap!’
   Metrics provide objectivity and move the conversation to a different place. If I change the conversation to: “32% of the sentences in your content are longer than 25 words, whereas winning proposals tend to have 5% or less”, then metrics become a way to drive towards better quality.
4. Alleviate Editorial Bottlenecks: Metrics and better quality base submissions shift the onus back to contributors. They provide a needed line of defense for the proposal manager.

Check out these 3-minute demos to how we scan documents with VisibleThread.

If you want to see how your own proposal docs rank for long sentences and passive, just sign up for a no-obligation 30-day free trial of VisibleThread in the cloud here: http://www.visiblethread.com/products/evaluate/. Or use it simply to see how your marketing pdfs or docs rank.

During 2011, we used VisibleThread to scan more than 120 separate bids (yes, we’ve been counting). The scanned bids included RFPs (Request for Proposal), Task Orders and commercial RFIs (Request for Information). We scanned bid responses from some of the largest integrators around the beltway, and smaller 8A companies too.

I spoke on Fed News Radio in March about the results we were seeing. I spoke again in October at the APMP NCA (Association of Proposal Management Professionals - National Capital Area) chapter conference with updated statistics. You should grab the full slide deck here which covers examples and metrics on our findings.

Despite this, we continued to see technically competent, well-priced bids, shot in the foot due to poor language.

So in this post, I wanted to share some more real examples that will help highlight this issue. At the end of this post, you’ll see a full cliché checklist. Feel free to copy this for use on your own bids. I also wanted to outline why I think this is a difficult issue and what we can do about it. I address this towards the end of this post.

Some Examples

Let me give a couple of simple examples from one Task Order.

Example 1: The following text appeared in section 1 of a technical volume, immediately following the table of contents.

“Our team is diverse with expertise in IT support unmatched by other small business consortiums.”

Introductory sections are read first. From the reviewer’s point of view, these summary areas set the tone for the rest of the response. You are basically saying upfront; I am responding to your need by using fluffy and marketing language. I will not bore you with actual facts to back up my claims.

So in this example, we claim that our company XXX is ‘unmatched’ in some capacity.

Put yourself in the reviewer’s position. If you saw this, what would you think? It is either not credible or lacks sufficient evidence. When VisibleThread encounters a term such as unmatched, it offers this guidance in the review report. I hope you see why it makes sense.

Likely a trite claim & ‘market-ese’, remove it, otherwise state the evidence and drop the ‘unmatched’ word. It simply comes across as not being credible.

As a side note, you may wonder why expertise is also flagged above in maroon color. VisibleThread flags this as a liability issue, as it is hard to defend. From a legal standpoint, it is easy to attack should an issue arise post bid win.

Example 2: In the same Task Order, the next paragraph contained:

“We demonstrate our diversity as a team by illuminating the fact that over seventy percent of Team XXX members are socio-economically disadvantaged companies.”

This long sentence needs to go on a diet. The first half of it is pure waste; the second half masks the real information. Try this instead:

“We are diverse because seventy percent of Team XXX members are socio-economically disadvantaged companies.”

This reduces the statement from 23 to 14 words. It emphasizes the metric. If this were a genuine color team review, I would go further and suggest this rewrite:

We are diverse because:

-          seventy percent of Team XXX members are socio-economically disadvantaged companies.

-          (2^nd point that supports the claim)

-          (3^rd point that supports the claim)

Lists drive concise responses. Secondly, lists are really easy to review. Our studies in website content analysis show that lists are way more effective way to communicate. Thirdly, lists put written text on a diet. Any fear of increasing page count is more than offset by a lowered word count.

I should point out that lists have an interesting side effect; they force you to group similar concepts and reduce repetition. This is a bigger subject than just this point. I’ll pen a separate post on this topic in the next while.

All told, lists are a good thing. Don’t be afraid to put them into your summary sections and emphasize metrics & numbers. You’ll be amazed at how better the result is.

Takeaways – why is it so hard?

Amid all the scans, what stood out was a continued problem with wordy and trite language in bids. I asked myself, why is it so hard?

The issue I think comes down to these points:

1. Time Constraint: Who the heck has time to properly scrub a doc when trying to meet incredibly tight delivery deadlines?
2. Review Process too late: If your process only allows for language review at red team review phase, then it’s invariably too late to fix amid all the other conflicting challenges.
3. Editing Content is hard: In many organisations especially tecnical heavy ones, ‘fixing the language’ is considered almost easy and an afterthought. This viewpoint needs to change. Editing content out is a skilled task and takes time.
4. Lack of Automation: If you try to scrub docs manually using search in word, it’s way too manual. You need some form of automated approach, be it MS Word macros or tooling designed for the job like VisibleThread.

Poor langauge kills bids, if reviews are left too close to the end, it is extremely hard to fix. So, we need to monitor the response using automation tools and eliminate language issues much earlier in the development cycle.

The full cliché Check List

So, I did promise I would share our list of ‘baddie’ words. Below you will find the full list we check for. It continues to expand.

If you’d like to review your own proposal docs online and see how they rank, contact us or sign up for a 30 day free evaluation of VisibleThread for Docs at: http://www.visiblethread.com/products/evaluate/ Our software can save you a ton of time and dramatically improve submission quality.

let’s call time on the cliché in 2012!

So, last week, on October 3^rd we scanned 31 federal agency websites with VisibleThread. The scans revealed some agencies doing really well, others remain challenged.

We plan to publish the full results soon. I wanted to share in this blog post some initial findings. These are items that caught our eye as we reviewed the scans. We hope these will be helpful for US federal agencies looking to comply with the Plain Writing act of 2010.

Why the Act?

Let’s remind ourselves why congressman Braley sponsored this act and why president Obama signed it.

Reason 1 - Improve Customer Engagement: Clarity in documents and websites means citizens are more likely to achieve their goals. Whether you are filing a tax return or trying to understanding labor rights; clear instructions mean we have a higher chance of completing an action.

Reason 2 - Reduce Cost: If the Feds can make it easier for people to engage, it means lower cost. It means lower cost for government due to less follow-up calls, letters and less time spent chasing. It cuts down on possible litigation due to misunderstood terms and conditions.

Plain Language is the mechanism that drives clarity for websites and documents.

The scanned results

Here are just two of the websites that we scanned:

Pension Benefit Guaranty Corporation (www.pbgc.gov) & Federal Highway Administration (www.fhwa.dot.gov). The metrics:

The Pension Benefit Guaranty Corporation scored a little over 20% when measured for long sentences. Long sentences exceed 25 words. In the context of web communications, this score is not good. The Fed Highway Administration wasn’t far behind at close to 18%. Equally, passive density was too high for the sites. Note that we did limit the scan to the first 100 pages of content, in both cases.

Hidden in the weeds - badness

One of the specific areas that caught our attention was the FAQ (Frequently Asked Questions) section of the pension site at: http://www.pbgc.gov/about/faq.html

Aside from the usage of an acronym ‘FAQ’ which is likely unfamiliar to the target demographic (i.e. aged persons), we found many issues with the content itself.

Now, you should note that the FAQ pages are reasonably well laid out and clear. So, the web design guys have done pretty well. The real issues lie in the content under the expandable answers. Under here, lurks some really bad stuff that we picked up in the scan. It’s the proverbial ‘under the rug’ area of any website.

For instance, at: http://www.pbgc.gov/about/faq/pg/general-faqs-about-pbgc.html , in answer to the very first question: ‘What is the Pension Benefit Guaranty Corporation (PBGC)?’ we get this:

A: PBGC is a federal agency created by the Employee Retirement Income Security Act of 1974 (ERISA) to protect pension benefits in private-sector traditional pension plans known as defined benefit plans. If your plan ends (this is called “plan termination”) without sufficient money to pay all benefits, PBGC’s insurance program will pay you the benefit provided by your pension plan up to the limits set by law. (Most people receive the full benefit they had earned before the plan terminated.) Our financing comes from insurance premiums paid by companies whose plans we protect, from our investments, from the assets of pension plans that we take over as trustee, and from recoveries from the companies formerly responsible for the plans, but not from taxes. Your plan is insured even if your employer fails to pay the required premiums.

Each long sentence is highlighted in light blue. Maroon highlights passive language.

So, let’s assume we understand the purpose of the agency, the next question ups the ante. In answer to ‘What can PBGC tell me about my pension?’ we have:

A: If you’re a participant in a pension plan that PBGC insures but that has not yet been taken over (“trusteed”), PBGC generally will have no information specifically about you, your plan, or your benefits under that plan. PBGC gathers this information only after PBGC trustees the plan. This means you’ll find only general information below about our insurance programs and how they operate, including our guarantees. If PBGC has trusteed your pension plan, we’ll have only limited information about your plan and your benefits until we’ve had time to obtain and review your plan’s records-typically after several months.

The rest of this area of the website has equally lengthy and complex language. Clearly, not achieving the aim of the Plain Writing act.

Long Sentences and Passive Voice, why important?

VisibleThread analyses websites and documents for a number of metrics. While other metrics (complex density, hidden verbs, readability) are useful; long and passive sentences are the ‘low hanging fruit’ of the Plain Language world. If all you do today, is just cut your long sentences down and write in active voice, clarity will immediately improve.

To re-iterate, two of the easiest ways you can reduce complexity & introduce clarity are:

1. Split long sentences and
2. Convert passive into active voice.

If you want more on this, read this excellent piece from the University of Wisconsin on passive voice. Then check out how putting wordy sentences on a diet helps too.

Easy Steps to simplify your website content

What simple steps can you take to improve website quality?

1. Scan for and eliminate Long sentences: Aside from being harder to interpret, long sentences often mask several different information concepts. They are thus harder to understand. If you split the sentence, it is an excellent first step to achieve clarity. For websites, where readers are actually visually skimming, long sentences are especially inappropriate.

1. Rewrite passive with a verb driven approach: Active (verb preceding noun) rather than passive (subject preceding verb) language makes for clearer requirements. It forces an explicit reference to the ‘actor’. The actor is the person or entity responsible for performing the action. This reduces ambiguity and improves likelihood your customers will complete an action.

Summary Takeaways

-          One year on, some agencies are still falling short of complying with the Plain Writing act.

-          We scanned several agency websites with VisibleThread and found some poor examples.

-          If you eliminate long sentences & passive language, clarity will improve.

In this post, I wanted to look at the IT space to see how we can improve documented requirements.

As background, people like Tom and Kai Gilb argue for improving IT requirements quality by rewriting them in a notation (Planguage) that is measurable and testable. This is really great if your team is able and willing to adopt that approach. In most business organisations however, it is a bridge too far. This is because BAs (Business Analysts) are not conditioned to translating informal English into mathematical statements.

Here I argue for a less radical and more digestible alternative; running a basic plain language check. This leads to dramatically improved requirements quality. More importantly, it is very workable in pretty much all organisations.

How does Plain Language analysis help?

For starters, let’s set some context on Plain Language. Plain Language (sometimes called Plain Writing) is all about clear, concise communication. If you want to see how and why the US government is sponsoring plain language, check here.

Let me first give a non-IT related before and after example from http://www.plainlanguage.gov/examples/before_after/index.cfm :

“When the process of freeing a vehicle that has been stuck results in ruts or holes, the operator will fill the rut or hole created by such activity before removing the vehicle from the immediate area.”

vs

“If you make a hole while freeing a stuck vehicle, you must fill the hole before you drive away.”

This example shows two things; (1) a long complicated sentence and (2) passive language. You can see the passive highlighted in maroon. Both sentences attempt to communicate the exact same thing. You be the judge of which is clearer.

Are you agile or waterfall? Does it matter?

You may be capturing your initial requirements using the ‘big fat doc’ approach or you may capture them as stories in a 100% agile environment. Either way, if you apply Plain Language checks, quality will improve.

Since we work with a good number of corporates who still use BRDs (Business Requirements Definition), I will show examples and analysis from BRDs but all of the principles of plain language apply equally to requirements captured as part of an agile approach.

The value of identifying long Sentences

Long sentences often mask multiple requirements and can often contain multiple meanings & ambiguity. We categorise them as sentences in excess of 25 words.

Here is an example of a long sentence in a recent BRD for a financial institution:

“Prices received electronically daily from the Fund Accountant via FTP and email, except for the Growth Stock Fund, which prices monthly and for the bi-monthly price receipt for Fixed Income  funds”

So, here are the immediate issues:

1. Multiple Requirements: There are actually 3 rules embedded in this sentence. A rule pertaining to ‘general pricing’, a rule pertaining to the ‘Growth Stock Fund’ and rules for ‘Fixed Income Funds’
2. Ambiguity: Is bi-monthly, once every two months or twice a month?
3. Ambiguity: Is daily business day only?
4. Ambiguity: Are Growth Stock Funds and Fixed Income funds also received electronically? Or is the ‘except’ clause inclusive of the receipt mechanism?

So scanning for long sentences tends to reveal multiple different requirements and exceptional scenarios. It often uncovers significant degrees of looseness. All of this leads to delivery risk and poor project estimatation.

What is Passive Language and why it matters?

It is common to find that people are unsure what passive language is. Let me give a simple example first:

“Quality is monitored” vs. “We monitor quality”

Notice that the first version contains ‘is monitored’. It does not specify the actor, i.e. who or what is responsible for monitoring. The formal definition of passive language is a statement where the noun precedes the verb. In our example, ‘quality’ is the noun, with ‘is monitored’ coming second, masking the active form ‘to monitor’.

Passive language also allows you omit specifying the person or entity responsible for performing the verb.

For IT systems, where completeness of specification is important, that omission can lead to guesswork and misinterpretation by developers. If you convert passive into active language, it will often reveal missing actors and incomplete specifications.

Omitting the actor also means less testable requirements. Agile stories and use cases by their nature emphasize the active form, which is why they are very effective capture mechanisms. On the other hand, BAs can tend to write general business & technical requirements in the passive voice.

Here is just one example from a real IT program spec in a BRD (Business Requirements Definition):

“Those transactions/fields that the user is authorized to input will be submitted to ZHG on the mainframe.”

Immediate issues:

1. Missing actor. Who or what authorizes?
2. Who or what submits to ZHG (renamed from the real system)?
3. We need an exact set of conditions that allow each step to occur. This statement actually contains two distinct rules. It would be better to split them as two separate statements (or steps) and link them in a flow or story.

Further, the following set of questions pop out:

4. Are transactions and fields using different permissions?
5. What are the transactions?
6. What are the fields?

While points 4 thru 6 are not explicitly associated with passive analysis, this kind of looseness is often spotted in the vicinity where passive language is present.

When we scanned the entire doc, it showed that 18.75% of all sentences in the BRD contained passive language.

So reviewing just for passive language often uncovers significant degrees of looseness and risk and the amount of passive language present increases risk.

Simple Steps to reduce Risk

What simple steps can you take to improve quality?

1. Scan for and eliminate Long sentences: Aside from being harder to interpret, long sentences often mask several different requirements. They are thus hard to track and estimate for. Splitting up the sentence is a first step.

1. Rewrite passive with a verb driven approach: Active (verb preceding noun) rather than passive (noun preceding verb) language makes for clearer requirements. It forces an explicit reference to the ‘actor’. The actor is the person or entity responsible for performing the action. From an IT standpoint, this reduces ambiguity and yields much more testable requirements.

Summary Takeaways

-          Plain Language can uncover high risk in your requirements.

-          Long sentences and passive language; these are your enemy.

-          Eliminate passive language and long sentences from your IT requirements to reduce risk.

In October 2010, Barack Obama signed the ‘Plain Language’ bill. It mandated that all federal communications adhere to plain language principals. The implementation deadline is October 2011. By that date, all fed communications must be plain language compliant… better get our skates on!

For those new to the phrase; ‘Plain Language’ seeks to simplify written language; the result is clear and unambiguous communication. The net effect is easier to review text.

Why would the US government, the president no less, mandate such an esoteric thing as plain language?

Let’s consider a case study. The State of Washington implemented a “Plain Talk” program across all state agencies some time ago. Read more here: http://www.usatoday.com/news/nation/2006-12-10-washington-plain-talk_x.htm

The results were pretty astounding:

After rewriting one letter, the Department of Revenue dramatically boosted its use-tax collections:

• Tripled number of businesses voluntarily paying use tax from 3% to 9%
• Goal: $1.2 million extra revenue in first year. Actual: $2 million extra collected.
• Cost: 1 cent for every dollar collected.

The benefits of ‘plain language’ apply equally to both Bid development and IT Delivery. Poorly written, overly complex documents lead to direct cost; including misunderstandings and extra cycles.

The reality for RFP responses is that applying plain language scans increases Bid Response quality. For IT programs, it results in decreased rework.

To this end; we have been building out our Plain Language analysis capabilities in VisibleThread. Already we scan for bad language flagging buzz words and uncommon/risky terms. The new Plain Analysis capabilities bring us further. I thought I would offer a sneak preview.

To do so, I took a collection of actual IRS documents from here: http://www.irs.gov/app/picklist/list/formsPublications.html .The following view shows the actual stats for each of these documents.

It took me 5 minutes to upload the 9 IRS docs above (all .pdf) and the results are intriguing. In one case, a doc with over 28,000 words (p17 highlighted in green) has significantly better readability and less passive language than a much smaller doc of 2000 words (highlighted in red). Wow!

The stats include: ‘Readability Index’, ‘Passive Language Density’, ‘Average sentence length’ & ‘number long sentences’. If you conduct a plain language audit, these are core checks. This view is powerful. It provides an instant comparison point across all your response volumes pinpointing issues.

The following shows an actual technical volume analysed by VisibleThread. The result is pretty illuminating. Tech Volume part 2 (highlighted in red) has; a lower readability score, a significantly higher passive language density (16% in part 2 vs 6% in part 1), and a much higher average sentence length ( 19 words in part 2 versus just 9 words in part 1).

Guess we better sort out Tech Volume part 2!

What simple steps can you take to work in plain language principals?

1. Eliminate Long sentences: Aside from being candidates for simplification, long sentences often mask several different requirements. They are thus hard to track. Splitting up the sentence is a first step.
2. Ensuring a verb driven approach: Active (verb preceding noun) rather than passive (noun preceding verb) language makes for far more understandable statements.
3. Remove redundant verbs.

If a Bid Proposal manager can finely tune their language to be concise and understandable, it drives a positive reaction for reviewers. If an IT professional adopts these principals, it lowers program execution risk.

We are really excited about the possibilities of plain language as it applies to Bid Development and IT Delivery. Stay tuned for more on this. Call or mail our sales team if you’d like a demo of your docs run through the VT plain language ‘hopper’. We’re very happy to offer sneak peeks. In case you’re wondering, we expect this capability to go live 2^nd half of July.

Extras:

If you are interested in learning more, including ‘before’ and ‘after’ examples check out these links:

-          Fed Plain Language Resources: http://www.plainlanguage.gov/index.cfm

-          Examples: http://www.plainlanguage.gov/examples/government/index.cfm

-          Washington State Case Study: http://www.usatoday.com/news/nation/2006-12-10-washington-plain-talk_x.htm

-          SEC Plain Language guidelines http://www.sec.gov/news/extra/handbook.htm

The VisibleThread team was 3 strong; Jeff from sales, Evan from marketing and myself. The show provided Proposal Professionals a great opportunity to find out what industry colleagues are up to as well as network. Some quick stats from ourselves:

• Our booth at the show attracted over 80 people, most of whom took time to view a brief (or not so brief) demo. Typical demo scenario in the picture to the right.
• We had about 100 people attend the ‘No Bad Language’ happy hour. That was a lot of fun. Picture below.
• We had 102 surveys completed assessing the state of proposal development. We will publish the results in the coming weeks, suffice it to say they are very interesting indeed.

As ‘chief demo giver’, I was completely hoarse after day 2. Thanks to Raymond Span for the ‘sage’ infused lozenges, a life saver. Sage, as it turns out, is an excellent cure for throat ailments!

We talked to a lot of folks who were interested in automating the proposal development life-cycle. One takeaway from my own perspective was the very real possibility that automation can shave cycles from mundane tasks and ensure a better response.

As I talked to people a particular challenge I noted was around win themes. How to ensure consistent win theme language & message across response volumes/docs? Interestingly, when we sliced and diced the survey results, this was further re-enforced. One specific question asked about biggest challenges in the life-cycle. Fully 42% of respondents said win themes are the most critical part of a winning bid. Team Collaboration and review was a distant 2nd place at 33%.

Not too surprisingly therefore, the VisibleThread capability that got people most excited was tied between ‘scanning response docs for Bad Language’ and ‘win Theme Validation and Discovery’. Both capabilities can be seen in our 3 minute demos at: http://www.visiblethread.com/products/demo/

From taking the pulse of attendees, the conference tracks were generally pretty well received. A few a little spotty, but you’ll have that at any show. Overall it was a well organised and run conference. It was a fun week for us and we were thrilled to meet such an excellent group of people and make so many new acquaintances from attendees as well as fellow exhibitors.

Already looking forward to next year, Best, Fergal.

Our customer; Doug, is currently leading a major bid response. He is applying VisibleThread in a very innovative way to help inform ‘win theme’ coverage analysis and was kind enough to share. It was quite a revelation and I wanted to blog on it.

So where to begin?

Well, let’s first consider ‘win themes’. As anyone involved in fed bids or commercial proposal development will appreciate, there are two core ideas at play when trying to win a contract, first; ensuring you are compliant with RFP requirements, second; making certain the core pain points of the offerer’s RFP are covered.

The first element; ‘compliance’, I always think of as a kind of a ‘dot the i’s cross the t’s’ type of process. In many cases, this is quite a mechanical, often tedious management process. VisibleThread does a pretty good job of facilitating compliance by identifying obligation language in the RFP or SOW, using automated scanning for ‘will’, ‘must’, ‘shall’ etc. It can generate an Excel output in a few minutes based of a fresh RFP, forming a really good starting point for a compliance matrix.

The second aspect; making certain the core pain points of the RFP are covered, is most frequently addressed by establishing win themes at varying levels in the response. This is less mechanical and dependent on the skill of the Bid/Proposal Manager. Basically, a Bid Manager will want to reinforce certain themes. Examples include; low-cost, low risk, history of achievement, innovation, or performance-based accountability.

Shipley and others have a lot to say on this topic. I really like how Larry Newman in his ‘Compliance vs. Responsiveness’ podcast characterizes these two aspects. Really well worth while listening to, if you have a couple of minutes. I think his definition of responsiveness is nicely aligned with effective ‘win themes’. Are we really addressing the true underlying needs of the issuing authority?

But I digress. Back to win theme coverage. Let me share the steps that I was shown.

Step 1: Firstly, the response volumes were added to a VisibleThread project as shown to the right.

Step 2: Now, the ‘Discovery’ view at project level displayed the documents side by side, with frequency of noun occurrence across the docs.

To explain the below view: the first column enumerates the nouns found in all documents, the second the frequency of occurrence of the nouns.

Each subsequent column represents each of the added documents, with a green indicator telling us that the particular noun is present. Within VisibleThread, as you hover over a green ball, you get the specific number present in that document.

In this case, we can see that ‘Cost’ is not mentioned at all in our 4th doc ‘122151-BOG…’. Now, let’s assume that one of our themes involves an emphasis on establishing a very ‘cost’ effective approach. Clearly, lack of reference to cost in the doc suggests, we have some detective work to do. Possibly one of the collaborators has accidentally removed references?

Step 3: Now the cool bit. Doug took the above and clicked on the ‘Export’ button highlighted in blue. This gave him a file that opened in Excel.

He then highlighted the cells containing frequency data and applied color scales using conditional formatting capabilities that come with Excel. This gave him an instant and very concise ‘heat map’ of the distribution of key theme words, red indicating heavy density, blue indicating light density.

Now, not only were gaps evident ala the normal VisibleThread view, but lack of sufficient distribution in other documents were highlighted too. Looks like ‘cost’ was not only absent in 1 document, it was too lightly covered in 2 other docs, with only 2 and 4 references respectively. Now that was concerning!

By adding a discussion comment in VisibleThread, Doug could then flag this for the team to take immediate action. As an aside, ‘In Context Discussion Threads’ were introduced in release 2.4 of VisibleThread and flagging commentary around possible issues is one of the ideal usage scenarios.

Bottom-line, the data that was provided by VisibleThread allowed really detailed coverage analysis in Excel. Now there’s a great example of product leverage!

Takeaways:

1. Using ‘discovery’ in VisibleThread can help determine win theme coverage.

2. Exporting the VisibleTherad data to Excel (available on all VisibleThread screens) can allow deeper analysis insights.

PS: Thanks go to Doug (you know who you are) for showing me this cool usage scenario.

In the first post here, I outlined how document management systems such as Sharepoint are really only as good as the structures set up to organise and store the data. Indeed, even assuming good folder structure, there is still the question of what lies within the documents. I finished the first post posing this question:

“What remains hidden is the content within the documents. Is it really any good?”

In other words, does the below ‘sea level’ doc content have acceptable quality? So, let’s now consider how ‘discovery’ and ‘concept mining’ can help in determining the answer.

Discovery

When reviewing a document, we tend to read it start to end. Under tight time constraints, you might opt to read the first few pages and skim the remainder. This approach to ‘discovery’ is error prone when dealing with voluminous documents in short time frames. You can’t assume every reviewer has equal capability or necessary domain expertise. Other options for document discovery are possible. Let’s consider one approach.

If we identify particular terms in their textual context along with frequency, you get a surprisingly rich sense of what a document’s themes are. Academics call this notion ‘concordance’. To give a concrete example, in the king James bible, ‘Lord’ occurs 7829 times, ‘God’ 4443 times. It might be fair to infer a theme here.

In 2008, the NY Times used concordance in a very visually rich way. Check out the link here for more.

They published an analysis of the state of the union address between 2000 and 2007. Interestingly, ‘IRAQ’ started at zero in 2000, tipping past 20 references in 2003 and onwards. Visual concordance is useful to identify textual themes.

‘Discovery’ using VisibleThread is similarly rich. In the case of VisibleThread, we are interested in analyzing sets of MS Office (Word, Excel) or pdf documents. Other than that, it has much the same intent as the NY Times work.

VisibleThread presents a breakdown of most frequent nouns correlated with section heading and textual extract. This reveals intent and themes within collections of documents. The below shows how this works for an arbitrary SOW (Statement of Work) issued by the department of homeland security.

In the screenshot, the list on the left shows the most frequently occurring nouns in the document.

The color coding allows us to correlate the noun to specific section headings in the SOW document. The section headings display to the right of the list in a structure view.

To the far right, a heat map shows term distribution. The specific text extracts containing the identified term are called out in the bottom pane.

We can see pretty clearly from the example, that this is an IT oriented SOW, with ‘IP’ showing 16 occurrences. Further, all 16 instances are confined to the ‘Specific Tasks’ section of the document.

More interestingly sometimes can be the absence of terms, revealing completeness issues. In the example above, we see only 1 instance of CIDR. Now that is worth tracking down! We may well have lack of clarity around the intent or meaning of CIDR as indicated by an ‘orphan’ or ‘one-off’ reference such as this. Sorting by low to high frequency can often reveal such gaps and incompleteness. This might apply to both the response doc or the issued doc. In the latter case, a natural opportunity to feed back into and help ‘shape’ the RFP might arise based on these insights.

So, on the one hand, intent is high-lighted with high frequency; on the other, lack of quality (or completeness) is flagged with low-frequency

Concept Mining and Concept Lists

Let’s take this a step further. Up to now, we have discovered document content by analysing a document or collection of documents. This is based on automatically calculated metrics for noun frequency and distribution. Consider the reverse. What if we had a collection of expected themes expressed as a ‘concept list’, could we spot deficiency in a document?

If we are authoring a response to an RFP for example, we may want to ‘push’ certain language in certain sections. As a Bid Manager today, I would package these concepts into ‘win themes’. Can we ‘inject’ these into the document and flag absence? As it turns out, it’s quite feasible. Let’s see how.

We can create a set of ‘win statements’ as a VisibleThread’ concept list. The concept list collects language that we want to reinforce. Think of it as collections of theme language packaged into groups. Each group corresponding to a specific theme. If we ensure a liberal sprinkling of theme language in the right sections of the response, this will increase the probability of a better score. How does it work?

We scan the document content looking for the theme language, cross-cutting by section. Quite similar to what we did for discovery.

In the example, we see a number of win themes represented and the ‘hits’ occurring within a response document.

One such theme is ‘Theme 5: Delivering improved customer service’ comprising terms speaking to that theme.

We can clearly see a lack of appropriate distribution of key theme phrases; ‘empowering’, ‘informed’ and ‘decisions’. Gray colorization indicates the absence of hits. So, we see the sections where theme language re-enforcement is required.

As you can see, dipping below ‘sea level’ can be very revealing. Once you cross-cut by section heading, analysis and metrics can be tremendously powerful. Document Content Analysis includes these techniques of discovery and concept mining. I hope you can see how this is far removed from Content Management. Really we are diving inside the document content to assess quality.

These are not the only techniques to assess quality below ‘sea-level’, others exist; structure analysis, word count analysis, passive language analysis to name a few. All have a role to play in determining content quality. I’ll aim to cover some others in future posts.

In the meantime, I hope the above examples might help you create better quality documents.