During 2011, we used VisibleThread to scan more than 120 separate bids (yes, we’ve been counting). The scanned bids included RFPs (Request for Proposal), Task Orders and commercial RFIs (Request for Information). We scanned bid responses from some of the largest integrators around the beltway, and smaller 8A companies too.

I spoke on Fed News Radio in March about the results we were seeing. I spoke again in October at the APMP NCA (Association of Proposal Management Professionals - National Capital Area) chapter conference with updated statistics. You should grab the full slide deck here which covers examples and metrics on our findings.

Despite this, we continued to see technically competent, well-priced bids, shot in the foot due to poor language.

So in this post, I wanted to share some more real examples that will help highlight this issue. At the end of this post, you’ll see a full cliché checklist. Feel free to copy this for use on your own bids. I also wanted to outline why I think this is a difficult issue and what we can do about it. I address this towards the end of this post.

Some Examples

Let me give a couple of simple examples from one Task Order.

Example 1: The following text appeared in section 1 of a technical volume, immediately following the table of contents.

“Our team is diverse with expertise in IT support unmatched by other small business consortiums.”

Introductory sections are read first. From the reviewer’s point of view, these summary areas set the tone for the rest of the response. You are basically saying upfront; I am responding to your need by using fluffy and marketing language. I will not bore you with actual facts to back up my claims.

So in this example, we claim that our company XXX is ‘unmatched’ in some capacity.

Put yourself in the reviewer’s position. If you saw this, what would you think? It is either not credible or lacks sufficient evidence. When VisibleThread encounters a term such as unmatched, it offers this guidance in the review report. I hope you see why it makes sense.

Likely a trite claim & ‘market-ese’, remove it, otherwise state the evidence and drop the ‘unmatched’ word. It simply comes across as not being credible.

As a side note, you may wonder why expertise is also flagged above in maroon color. VisibleThread flags this as a liability issue, as it is hard to defend. From a legal standpoint, it is easy to attack should an issue arise post bid win.

Example 2: In the same Task Order, the next paragraph contained:

“We demonstrate our diversity as a team by illuminating the fact that over seventy percent of Team XXX members are socio-economically disadvantaged companies.”

This long sentence needs to go on a diet. The first half of it is pure waste; the second half masks the real information. Try this instead:

“We are diverse because seventy percent of Team XXX members are socio-economically disadvantaged companies.”

This reduces the statement from 23 to 14 words. It emphasizes the metric. If this were a genuine color team review, I would go further and suggest this rewrite:

We are diverse because:

-          seventy percent of Team XXX members are socio-economically disadvantaged companies.

-          (2^nd point that supports the claim)

-          (3^rd point that supports the claim)

Lists drive concise responses. Secondly, lists are really easy to review. Our studies in website content analysis show that lists are way more effective way to communicate. Thirdly, lists put written text on a diet. Any fear of increasing page count is more than offset by a lowered word count.

I should point out that lists have an interesting side effect; they force you to group similar concepts and reduce repetition. This is a bigger subject than just this point. I’ll pen a separate post on this topic in the next while.

All told, lists are a good thing. Don’t be afraid to put them into your summary sections and emphasize metrics & numbers. You’ll be amazed at how better the result is.

Takeaways – why is it so hard?

Amid all the scans, what stood out was a continued problem with wordy and trite language in bids. I asked myself, why is it so hard?

The issue I think comes down to these points:

1. Time Constraint: Who the heck has time to properly scrub a doc when trying to meet incredibly tight delivery deadlines?
2. Review Process too late: If your process only allows for language review at red team review phase, then it’s invariably too late to fix amid all the other conflicting challenges.
3. Editing Content is hard: In many organisations especially tecnical heavy ones, ‘fixing the language’ is considered almost easy and an afterthought. This viewpoint needs to change. Editing content out is a skilled task and takes time.
4. Lack of Automation: If you try to scrub docs manually using search in word, it’s way too manual. You need some form of automated approach, be it MS Word macros or tooling designed for the job like VisibleThread.

Poor langauge kills bids, if reviews are left too close to the end, it is extremely hard to fix. So, we need to monitor the response using automation tools and eliminate language issues much earlier in the development cycle.

The full cliché Check List

So, I did promise I would share our list of ‘baddie’ words. Below you will find the full list we check for. It continues to expand.

If you’d like to review your own proposal docs online and see how they rank, contact us or sign up for a 30 day free evaluation of VisibleThread for Docs at: http://www.visiblethread.com/products/evaluate/ Our software can save you a ton of time and dramatically improve submission quality.

let’s call time on the cliché in 2012!

So, last week, on October 3^rd we scanned 31 federal agency websites with VisibleThread. The scans revealed some agencies doing really well, others remain challenged.

We plan to publish the full results soon. I wanted to share in this blog post some initial findings. These are items that caught our eye as we reviewed the scans. We hope these will be helpful for US federal agencies looking to comply with the Plain Writing act of 2010.

Why the Act?

Let’s remind ourselves why congressman Braley sponsored this act and why president Obama signed it.

Reason 1 - Improve Customer Engagement: Clarity in documents and websites means citizens are more likely to achieve their goals. Whether you are filing a tax return or trying to understanding labor rights; clear instructions mean we have a higher chance of completing an action.

Reason 2 - Reduce Cost: If the Feds can make it easier for people to engage, it means lower cost. It means lower cost for government due to less follow-up calls, letters and less time spent chasing. It cuts down on possible litigation due to misunderstood terms and conditions.

Plain Language is the mechanism that drives clarity for websites and documents.

The scanned results

Here are just two of the websites that we scanned:

Pension Benefit Guaranty Corporation (www.pbgc.gov) & Federal Highway Administration (www.fhwa.dot.gov). The metrics:

The Pension Benefit Guaranty Corporation scored a little over 20% when measured for long sentences. Long sentences exceed 25 words. In the context of web communications, this score is not good. The Fed Highway Administration wasn’t far behind at close to 18%. Equally, passive density was too high for the sites. Note that we did limit the scan to the first 100 pages of content, in both cases.

Hidden in the weeds - badness

One of the specific areas that caught our attention was the FAQ (Frequently Asked Questions) section of the pension site at: http://www.pbgc.gov/about/faq.html

Aside from the usage of an acronym ‘FAQ’ which is likely unfamiliar to the target demographic (i.e. aged persons), we found many issues with the content itself.

Now, you should note that the FAQ pages are reasonably well laid out and clear. So, the web design guys have done pretty well. The real issues lie in the content under the expandable answers. Under here, lurks some really bad stuff that we picked up in the scan. It’s the proverbial ‘under the rug’ area of any website.

For instance, at: http://www.pbgc.gov/about/faq/pg/general-faqs-about-pbgc.html , in answer to the very first question: ‘What is the Pension Benefit Guaranty Corporation (PBGC)?’ we get this:

A: PBGC is a federal agency created by the Employee Retirement Income Security Act of 1974 (ERISA) to protect pension benefits in private-sector traditional pension plans known as defined benefit plans. If your plan ends (this is called “plan termination”) without sufficient money to pay all benefits, PBGC’s insurance program will pay you the benefit provided by your pension plan up to the limits set by law. (Most people receive the full benefit they had earned before the plan terminated.) Our financing comes from insurance premiums paid by companies whose plans we protect, from our investments, from the assets of pension plans that we take over as trustee, and from recoveries from the companies formerly responsible for the plans, but not from taxes. Your plan is insured even if your employer fails to pay the required premiums.

Each long sentence is highlighted in light blue. Maroon highlights passive language.

So, let’s assume we understand the purpose of the agency, the next question ups the ante. In answer to ‘What can PBGC tell me about my pension?’ we have:

A: If you’re a participant in a pension plan that PBGC insures but that has not yet been taken over (“trusteed”), PBGC generally will have no information specifically about you, your plan, or your benefits under that plan. PBGC gathers this information only after PBGC trustees the plan. This means you’ll find only general information below about our insurance programs and how they operate, including our guarantees. If PBGC has trusteed your pension plan, we’ll have only limited information about your plan and your benefits until we’ve had time to obtain and review your plan’s records-typically after several months.

The rest of this area of the website has equally lengthy and complex language. Clearly, not achieving the aim of the Plain Writing act.

Long Sentences and Passive Voice, why important?

VisibleThread analyses websites and documents for a number of metrics. While other metrics (complex density, hidden verbs, readability) are useful; long and passive sentences are the ‘low hanging fruit’ of the Plain Language world. If all you do today, is just cut your long sentences down and write in active voice, clarity will immediately improve.

To re-iterate, two of the easiest ways you can reduce complexity & introduce clarity are:

1. Split long sentences and
2. Convert passive into active voice.

If you want more on this, read this excellent piece from the University of Wisconsin on passive voice. Then check out how putting wordy sentences on a diet helps too.

Easy Steps to simplify your website content

What simple steps can you take to improve website quality?

1. Scan for and eliminate Long sentences: Aside from being harder to interpret, long sentences often mask several different information concepts. They are thus harder to understand. If you split the sentence, it is an excellent first step to achieve clarity. For websites, where readers are actually visually skimming, long sentences are especially inappropriate.

1. Rewrite passive with a verb driven approach: Active (verb preceding noun) rather than passive (subject preceding verb) language makes for clearer requirements. It forces an explicit reference to the ‘actor’. The actor is the person or entity responsible for performing the action. This reduces ambiguity and improves likelihood your customers will complete an action.

Summary Takeaways

-          One year on, some agencies are still falling short of complying with the Plain Writing act.

-          We scanned several agency websites with VisibleThread and found some poor examples.

-          If you eliminate long sentences & passive language, clarity will improve.

In this post, I wanted to look at the IT space to see how we can improve documented requirements.

As background, people like Tom and Kai Gilb argue for improving IT requirements quality by rewriting them in a notation (Planguage) that is measurable and testable. This is really great if your team is able and willing to adopt that approach. In most business organisations however, it is a bridge too far. This is because BAs (Business Analysts) are not conditioned to translating informal English into mathematical statements.

Here I argue for a less radical and more digestible alternative; running a basic plain language check. This leads to dramatically improved requirements quality. More importantly, it is very workable in pretty much all organisations.

How does Plain Language analysis help?

For starters, let’s set some context on Plain Language. Plain Language (sometimes called Plain Writing) is all about clear, concise communication. If you want to see how and why the US government is sponsoring plain language, check here.

Let me first give a non-IT related before and after example from http://www.plainlanguage.gov/examples/before_after/index.cfm :

“When the process of freeing a vehicle that has been stuck results in ruts or holes, the operator will fill the rut or hole created by such activity before removing the vehicle from the immediate area.”

vs

“If you make a hole while freeing a stuck vehicle, you must fill the hole before you drive away.”

This example shows two things; (1) a long complicated sentence and (2) passive language. You can see the passive highlighted in maroon. Both sentences attempt to communicate the exact same thing. You be the judge of which is clearer.

Are you agile or waterfall? Does it matter?

You may be capturing your initial requirements using the ‘big fat doc’ approach or you may capture them as stories in a 100% agile environment. Either way, if you apply Plain Language checks, quality will improve.

Since we work with a good number of corporates who still use BRDs (Business Requirements Definition), I will show examples and analysis from BRDs but all of the principles of plain language apply equally to requirements captured as part of an agile approach.

The value of identifying long Sentences

Long sentences often mask multiple requirements and can often contain multiple meanings & ambiguity. We categorise them as sentences in excess of 25 words.

Here is an example of a long sentence in a recent BRD for a financial institution:

“Prices received electronically daily from the Fund Accountant via FTP and email, except for the Growth Stock Fund, which prices monthly and for the bi-monthly price receipt for Fixed Income  funds”

So, here are the immediate issues:

1. Multiple Requirements: There are actually 3 rules embedded in this sentence. A rule pertaining to ‘general pricing’, a rule pertaining to the ‘Growth Stock Fund’ and rules for ‘Fixed Income Funds’
2. Ambiguity: Is bi-monthly, once every two months or twice a month?
3. Ambiguity: Is daily business day only?
4. Ambiguity: Are Growth Stock Funds and Fixed Income funds also received electronically? Or is the ‘except’ clause inclusive of the receipt mechanism?

So scanning for long sentences tends to reveal multiple different requirements and exceptional scenarios. It often uncovers significant degrees of looseness. All of this leads to delivery risk and poor project estimatation.

What is Passive Language and why it matters?

It is common to find that people are unsure what passive language is. Let me give a simple example first:

“Quality is monitored” vs. “We monitor quality”

Notice that the first version contains ‘is monitored’. It does not specify the actor, i.e. who or what is responsible for monitoring. The formal definition of passive language is a statement where the noun precedes the verb. In our example, ‘quality’ is the noun, with ‘is monitored’ coming second, masking the active form ‘to monitor’.

Passive language also allows you omit specifying the person or entity responsible for performing the verb.

For IT systems, where completeness of specification is important, that omission can lead to guesswork and misinterpretation by developers. If you convert passive into active language, it will often reveal missing actors and incomplete specifications.

Omitting the actor also means less testable requirements. Agile stories and use cases by their nature emphasize the active form, which is why they are very effective capture mechanisms. On the other hand, BAs can tend to write general business & technical requirements in the passive voice.

Here is just one example from a real IT program spec in a BRD (Business Requirements Definition):

“Those transactions/fields that the user is authorized to input will be submitted to ZHG on the mainframe.”

Immediate issues:

1. Missing actor. Who or what authorizes?
2. Who or what submits to ZHG (renamed from the real system)?
3. We need an exact set of conditions that allow each step to occur. This statement actually contains two distinct rules. It would be better to split them as two separate statements (or steps) and link them in a flow or story.

Further, the following set of questions pop out:

4. Are transactions and fields using different permissions?
5. What are the transactions?
6. What are the fields?

While points 4 thru 6 are not explicitly associated with passive analysis, this kind of looseness is often spotted in the vicinity where passive language is present.

When we scanned the entire doc, it showed that 18.75% of all sentences in the BRD contained passive language.

So reviewing just for passive language often uncovers significant degrees of looseness and risk and the amount of passive language present increases risk.

Simple Steps to reduce Risk

What simple steps can you take to improve quality?

1. Scan for and eliminate Long sentences: Aside from being harder to interpret, long sentences often mask several different requirements. They are thus hard to track and estimate for. Splitting up the sentence is a first step.

1. Rewrite passive with a verb driven approach: Active (verb preceding noun) rather than passive (noun preceding verb) language makes for clearer requirements. It forces an explicit reference to the ‘actor’. The actor is the person or entity responsible for performing the action. From an IT standpoint, this reduces ambiguity and yields much more testable requirements.

Summary Takeaways

-          Plain Language can uncover high risk in your requirements.

-          Long sentences and passive language; these are your enemy.

-          Eliminate passive language and long sentences from your IT requirements to reduce risk.

In October 2010, Barack Obama signed the ‘Plain Language’ bill. It mandated that all federal communications adhere to plain language principals. The implementation deadline is October 2011. By that date, all fed communications must be plain language compliant… better get our skates on!

For those new to the phrase; ‘Plain Language’ seeks to simplify written language; the result is clear and unambiguous communication. The net effect is easier to review text.

Why would the US government, the president no less, mandate such an esoteric thing as plain language? Let’s consider a case study. The State of Washington implemented a “Plain Talk” program across all state agencies some time ago. Read more here: http://www.usatoday.com/news/nation/2006-12-10-washington-plain-talk_x.htm

The results were pretty astounding:

After rewriting one letter, the Department of Revenue dramatically boosted its use-tax collections:

• Tripled number of businesses voluntarily paying use tax from 3% to 9%
• Goal: $1.2 million extra revenue in first year. Actual: $2 million extra collected.
• Cost: 1 cent for every dollar collected.

The benefits of ‘plain language’ apply equally to both Bid development and IT Delivery. Poorly written, overly complex documents lead to direct cost; including misunderstandings and extra cycles.

The reality for RFP responses is that applying plain language scans increases Bid Response quality. For IT programs, it results in decreased rework.

To this end; we have been building out our Plain Language analysis capabilities in VisibleThread. Already we scan for bad language flagging buzz words and uncommon/risky terms. The new Plain Analysis capabilities bring us further. I thought I would offer a sneak preview.

To do so, I took a collection of actual IRS documents from here: http://www.irs.gov/app/picklist/list/formsPublications.html .The following view shows the actual stats for each of these documents.

It took me 5 minutes to upload the 9 IRS docs above (all .pdf) and the results are intriguing. In one case, a doc with over 28,000 words (p17 highlighted in green) has significantly better readability and less passive language than a much smaller doc of 2000 words (highlighted in red). Wow!

The stats include: ‘Readability Index’, ‘Passive Language Density’, ‘Average sentence length’ & ‘number long sentences’. If you conduct a plain language audit, these are core checks. This view is powerful. It provides an instant comparison point across all your response volumes pinpointing issues.

The following shows an actual technical volume analysed by VisibleThread. The result is pretty illuminating. Tech Volume part 2 (highlighted in red) has; a lower readability score, a significantly higher passive language density (16% in part 2 vs 6% in part 1), and a much higher average sentence length ( 19 words in part 2 versus just 9 words in part 1).

Guess we better sort out Tech Volume part 2!

What simple steps can you take to work in plain language principals?

1. Eliminate Long sentences: Aside from being candidates for simplification, long sentences often mask several different requirements. They are thus hard to track. Splitting up the sentence is a first step.
2. Ensuring a verb driven approach: Active (verb preceding noun) rather than passive (noun preceding verb) language makes for far more understandable statements.
3. Remove redundant verbs.

If a Bid Proposal manager can finely tune their language to be concise and understandable, it drives a positive reaction for reviewers. If an IT professional adopts these principals, it lowers program execution risk.

We are really excited about the possibilities of plain language as it applies to Bid Development and IT Delivery. Stay tuned for more on this. Call or mail our sales team if you’d like a demo of your docs run through the VT plain language ‘hopper’. We’re very happy to offer sneak peeks. In case you’re wondering, we expect this capability to go live 2^nd half of July.

Extras:

If you are interested in learning more, including ‘before’ and ‘after’ examples check out these links:

-          Fed Plain Language Resources: http://www.plainlanguage.gov/index.cfm

-          Examples: http://www.plainlanguage.gov/examples/government/index.cfm

-          Washington State Case Study: http://www.usatoday.com/news/nation/2006-12-10-washington-plain-talk_x.htm

-          SEC Plain Language guidelines http://www.sec.gov/news/extra/handbook.htm

The VisibleThread team was 3 strong; Jeff from sales, Evan from marketing and myself. The show provided Proposal Professionals a great opportunity to find out what industry colleagues are up to as well as network. Some quick stats from ourselves:

• Our booth at the show attracted over 80 people, most of whom took time to view a brief (or not so brief) demo. Typical demo scenario in the picture to the right.
• We had about 100 people attend the ‘No Bad Language’ happy hour. That was a lot of fun. Picture below.
• We had 102 surveys completed assessing the state of proposal development. We will publish the results in the coming weeks, suffice it to say they are very interesting indeed.

As ‘chief demo giver’, I was completely hoarse after day 2. Thanks to Raymond Span for the ‘sage’ infused lozenges, a life saver. Sage, as it turns out, is an excellent cure for throat ailments!

We talked to a lot of folks who were interested in automating the proposal development life-cycle. One takeaway from my own perspective was the very real possibility that automation can shave cycles from mundane tasks and ensure a better response.

As I talked to people a particular challenge I noted was around win themes. How to ensure consistent win theme language & message across response volumes/docs? Interestingly, when we sliced and diced the survey results, this was further re-enforced. One specific question asked about biggest challenges in the life-cycle. Fully 42% of respondents said win themes are the most critical part of a winning bid. Team Collaboration and review was a distant 2nd place at 33%.

Not too surprisingly therefore, the VisibleThread capability that got people most excited was tied between ‘scanning response docs for Bad Language’ and ‘win Theme Validation and Discovery’. Both capabilities can be seen in our 3 minute demos at: http://www.visiblethread.com/products/demo/

From taking the pulse of attendees, the conference tracks were generally pretty well received. A few a little spotty, but you’ll have that at any show. Overall it was a well organised and run conference. It was a fun week for us and we were thrilled to meet such an excellent group of people and make so many new acquaintances from attendees as well as fellow exhibitors.

Already looking forward to next year, Best, Fergal.

Our customer; Doug, is currently leading a major bid response. He is applying VisibleThread in a very innovative way to help inform ‘win theme’ coverage analysis and was kind enough to share. It was quite a revelation and I wanted to blog on it.

So where to begin?

Well, let’s first consider ‘win themes’. As anyone involved in fed bids or commercial proposal development will appreciate, there are two core ideas at play when trying to win a contract, first; ensuring you are compliant with RFP requirements, second; making certain the core pain points of the offerer’s RFP are covered.

The first element; ‘compliance’, I always think of as a kind of a ‘dot the i’s cross the t’s’ type of process. In many cases, this is quite a mechanical, often tedious management process. VisibleThread does a pretty good job of facilitating compliance by identifying obligation language in the RFP or SOW, using automated scanning for ‘will’, ‘must’, ‘shall’ etc. It can generate an Excel output in a few minutes based of a fresh RFP, forming a really good starting point for a compliance matrix.

The second aspect; making certain the core pain points of the RFP are covered, is most frequently addressed by establishing win themes at varying levels in the response. This is less mechanical and dependent on the skill of the Bid/Proposal Manager. Basically, a Bid Manager will want to reinforce certain themes. Examples include; low-cost, low risk, history of achievement, innovation, or performance-based accountability.

Shipley and others have a lot to say on this topic. I really like how Larry Newman in his ‘Compliance vs. Responsiveness’ podcast characterizes these two aspects. Really well worth while listening to, if you have a couple of minutes. I think his definition of responsiveness is nicely aligned with effective ‘win themes’. Are we really addressing the true underlying needs of the issuing authority?

But I digress. Back to win theme coverage. Let me share the steps that I was shown.

Step 1: Firstly, the response volumes were added to a VisibleThread project as shown to the right.

Step 2: Now, the ‘Discovery’ view at project level displayed the documents side by side, with frequency of noun occurrence across the docs.

To explain the below view: the first column enumerates the nouns found in all documents, the second the frequency of occurrence of the nouns.

Each subsequent column represents each of the added documents, with a green indicator telling us that the particular noun is present. Within VisibleThread, as you hover over a green ball, you get the specific number present in that document.

In this case, we can see that ‘Cost’ is not mentioned at all in our 4th doc ‘122151-BOG…’. Now, let’s assume that one of our themes involves an emphasis on establishing a very ‘cost’ effective approach. Clearly, lack of reference to cost in the doc suggests, we have some detective work to do. Possibly one of the collaborators has accidentally removed references?

Step 3: Now the cool bit. Doug took the above and clicked on the ‘Export’ button highlighted in blue. This gave him a file that opened in Excel.

He then highlighted the cells containing frequency data and applied color scales using conditional formatting capabilities that come with Excel. This gave him an instant and very concise ‘heat map’ of the distribution of key theme words, red indicating heavy density, blue indicating light density.

Now, not only were gaps evident ala the normal VisibleThread view, but lack of sufficient distribution in other documents were highlighted too. Looks like ‘cost’ was not only absent in 1 document, it was too lightly covered in 2 other docs, with only 2 and 4 references respectively. Now that was concerning!

By adding a discussion comment in VisibleThread, Doug could then flag this for the team to take immediate action. As an aside, ‘In Context Discussion Threads’ were introduced in release 2.4 of VisibleThread and flagging commentary around possible issues is one of the ideal usage scenarios.

Bottom-line, the data that was provided by VisibleThread allowed really detailed coverage analysis in Excel. Now there’s a great example of product leverage!

Takeaways:

1. Using ‘discovery’ in VisibleThread can help determine win theme coverage.

2. Exporting the VisibleTherad data to Excel (available on all VisibleThread screens) can allow deeper analysis insights.

PS: Thanks go to Doug (you know who you are) for showing me this cool usage scenario.

In the first post here, I outlined how document management systems such as Sharepoint are really only as good as the structures set up to organise and store the data. Indeed, even assuming good folder structure, there is still the question of what lies within the documents. I finished the first post posing this question:

“What remains hidden is the content within the documents. Is it really any good?”

In other words, does the below ‘sea level’ doc content have acceptable quality? So, let’s now consider how ‘discovery’ and ‘concept mining’ can help in determining the answer.

Discovery

When reviewing a document, we tend to read it start to end. Under tight time constraints, you might opt to read the first few pages and skim the remainder. This approach to ‘discovery’ is error prone when dealing with voluminous documents in short time frames. You can’t assume every reviewer has equal capability or necessary domain expertise. Other options for document discovery are possible. Let’s consider one approach.

If we identify particular terms in their textual context along with frequency, you get a surprisingly rich sense of what a document’s themes are. Academics call this notion ‘concordance’. To give a concrete example, in the king James bible, ‘Lord’ occurs 7829 times, ‘God’ 4443 times. It might be fair to infer a theme here.

In 2008, the NY Times used concordance in a very visually rich way. Check out the link here for more.

They published an analysis of the state of the union address between 2000 and 2007. Interestingly, ‘IRAQ’ started at zero in 2000, tipping past 20 references in 2003 and onwards. Visual concordance is useful to identify textual themes.

‘Discovery’ using VisibleThread is similarly rich. In the case of VisibleThread, we are interested in analyzing sets of MS Office (Word, Excel) or pdf documents. Other than that, it has much the same intent as the NY Times work.

VisibleThread presents a breakdown of most frequent nouns correlated with section heading and textual extract. This reveals intent and themes within collections of documents. The below shows how this works for an arbitrary SOW (Statement of Work) issued by the department of homeland security.

In the screenshot, the list on the left shows the most frequently occurring nouns in the document.

The color coding allows us to correlate the noun to specific section headings in the SOW document. The section headings display to the right of the list in a structure view.

To the far right, a heat map shows term distribution. The specific text extracts containing the identified term are called out in the bottom pane.

We can see pretty clearly from the example, that this is an IT oriented SOW, with ‘IP’ showing 16 occurrences. Further, all 16 instances are confined to the ‘Specific Tasks’ section of the document.

More interestingly sometimes can be the absence of terms, revealing completeness issues. In the example above, we see only 1 instance of CIDR. Now that is worth tracking down! We may well have lack of clarity around the intent or meaning of CIDR as indicated by an ‘orphan’ or ‘one-off’ reference such as this. Sorting by low to high frequency can often reveal such gaps and incompleteness. This might apply to both the response doc or the issued doc. In the latter case, a natural opportunity to feed back into and help ‘shape’ the RFP might arise based on these insights.

So, on the one hand, intent is high-lighted with high frequency; on the other, lack of quality (or completeness) is flagged with low-frequency

Concept Mining and Concept Lists

Let’s take this a step further. Up to now, we have discovered document content by analysing a document or collection of documents. This is based on automatically calculated metrics for noun frequency and distribution. Consider the reverse. What if we had a collection of expected themes expressed as a ‘concept list’, could we spot deficiency in a document?

If we are authoring a response to an RFP for example, we may want to ‘push’ certain language in certain sections. As a Bid Manager today, I would package these concepts into ‘win themes’. Can we ‘inject’ these into the document and flag absence? As it turns out, it’s quite feasible. Let’s see how.

We can create a set of ‘win statements’ as a VisibleThread’ concept list. The concept list collects language that we want to reinforce. Think of it as collections of theme language packaged into groups. Each group corresponding to a specific theme. If we ensure a liberal sprinkling of theme language in the right sections of the response, this will increase the probability of a better score. How does it work?

We scan the document content looking for the theme language, cross-cutting by section. Quite similar to what we did for discovery.

In the example, we see a number of win themes represented and the ‘hits’ occurring within a response document.

One such theme is ‘Theme 5: Delivering improved customer service’ comprising terms speaking to that theme.

We can clearly see a lack of appropriate distribution of key theme phrases; ‘empowering’, ‘informed’ and ‘decisions’. Gray colorization indicates the absence of hits. So, we see the sections where theme language re-enforcement is required.

As you can see, dipping below ‘sea level’ can be very revealing. Once you cross-cut by section heading, analysis and metrics can be tremendously powerful. Document Content Analysis includes these techniques of discovery and concept mining. I hope you can see how this is far removed from Content Management. Really we are diving inside the document content to assess quality.

These are not the only techniques to assess quality below ‘sea-level’, others exist; structure analysis, word count analysis, passive language analysis to name a few. All have a role to play in determining content quality. I’ll aim to cover some others in future posts.

In the meantime, I hope the above examples might help you create better quality documents.

We work with corporate entities both in the IT and Bid Compliance space. With few exceptions, these organisations currently use, or are looking to use Microsoft Sharepoint. VisibleThread does use its own repository based on subversion or equally can integrate with Sharepoint, so we work with many Sharepoint customers.

In any case, we’ve seen a number of organisations deploy Sharepoint in ‘vanilla’ form or with low levels of structural enforcement. They often take the basic free copy of Sharepoint called Windows Sharepoint Services and deploy it with fairly minimal configuration.

This can tend to lead to a ‘wild west’ scenario, where pretty much anything goes. The almost certain indicator that you are there is when you ask a colleague for a document and the response is: “it’s out on Sharepoint”. You have no clue as to where it might be; what site? What folder? You may just throw your hands in the air, sigh in exasperation and end up trawling through your e-mail threads or hard drive for the document. Well, at least you found a version of the document, right!

So, if we don’t apply logical categorisation structures and signpost them well, a content management system on its own will add little value, the vast majority of people will bypass recommended procedures. You really can’t blame them. Your data hygiene however, will quickly go down the toilet. Visibility, efficiency and consistency suffer when we cannot be sure where content exists. This is not the fault of the document management system rather it is a human problem.

Let’s take myself as an example. Looking at our own internal collateral for VisibleThread, we see these files on our marketing file share:

This might look OK but in fact the documents are really quite inconsistent:

• ‘VT’ versus ‘VisibleThread’ (e.g. ‘VT-Bid-Proposal-Brochure-Dec-2010.pdf’ as against ‘VisibleThread - Feature-List.pdf’)
• date based version convention as against number versioning: v1, v2
• and so on, can you spot the other anomalies?

So, as much as I might consider our organisation as fairly structured with ‘reasonable’ naming conventions, I am in fact falling down on ensuring consistency even when the documents are under my control and in a relatively small setting.

Scale up, with multiple people in larger organisations, possibly geographically dispersed and you quickly arrive at a major ‘discovery’ headache.

So now, let’s assume you’ve had the consultants in or have spent time configuring the Sharepoint deployment internally. Now you will have a reasonable library/folder structure, conventions will be in place for versioning, everyone knows where every document is neatly stored. Great, job done!

Wait, not so fast, how do we actually know that the documents are any good?

• Can we easily understand the intent of documents?
• How do we know we have adequate coverage of key concepts?
• What are the dependencies between documents?
• In a bid proposal scenario, do we have the right document heading structure?
• Do we have enough content relating to ‘win themes’ for bids?

What remains hidden is the content within the documents. Is it really any good? Will it deliver a great IT system or will we win the Bid?

The next post will consider this further, focussing on ‘discovery’ and ‘concept mining’ showing how document content analysis works and how it is complimentary to document management ensuring top class documents. Stay tuned.

Let’s draw back for a moment. Regardless of process approach, be it traditional or agile, poor treatment of NFRs is one of the highest risk areas in IT delivery. For anyone who has been involved in even a half-way serious type of project, lack of adequate NFR consideration will royally bite you in the bum. In my experience, it is probably the single biggest cause of lost sleep, likely I suspect accounting for the highest rate of ulcers for project & program managers. This applies to stakeholders at all levels, from project managers to executives. Executives as they realise in horror that the $70 million integration project due to go into production in 2 months time is behind and is not scaling according to plan and the business is looking for a release date or that the production system is down periodically for long spells due to scaling issues. Many stressful roads lead back to NFRs and if the NFR horse has bolted on your project, get ready for some serious stress. That said, let me get back to the point of this post, NFRs as stories.

So, I’d like to challenge one of the sacred cows for some in the agile community; namely that ‘everything is a story’, including NFRs.

A number of blog posts argue for representing NFRs as stories, including this one. Not everyone in the agile community agrees. The excellent Scott Selhurst blog is a notable example of extremely balanced thinking in that regard. Tom and kai Gilb have a lot to say on the matter here irrespective of whether you’re in traditional or agile settings or a mix.

Let’s get a definition or two out there, so we’re on the same page. Firstly what is an NFR? I will take the definition I first spotted in the ‘HP Fusion’ process oh about 18 or so years ago, that is we have basically two types of NFRs; Qualities and Constraints. Qualities are generally very clearly measurable and represent things like performance, uptime, data load etc, the classic ‘ilities’. Qualities will tend to be associated with strands of functionality or with the system as a whole. Constraints tend to be exactly that, statements of constraint, for instance ‘we must support IE 6’ (at VisibleThread this particular one pains me considering the variations on service pack OS etc. but as we do have customers on IE 6 we need to satisfy this constraint). An example of a second constraint is: ‘Must have a legal disclaimer on every page of the web interface’ or ‘we must code in Java on JDK version x.y’

You may not agree with the qualities or constraint delineation or have differing views but that’s fine, work with me, whatever you choose to call them, most people agree that both styles of NFR exist, regardless of your definition or terminology.

Let me offer a few reasons for why agile analysts & team members should consider avoiding representing NFRs as stories.

• Communication: NFRs often have a heavier impact than stories on core design foundations. Ask any tech architect or design lead for war stories on challenged projects they know and you are highly likely to be able to attribute the issue to lack of ability to satisfy one or more NFR as it’s root cause. Therefore the act of calling out NFRs as something that is not a story is of immense value from a communication standpoint. As an architect seeing a clear delineation of NFRs (particularly the qualities) from stories helps identify incompleteness.
• Cross-cutting scope: NFRs tend to come in two broad ‘namespaces’, those that apply universally across the project and those that may be associated with 1 or more strands of functionality, (a story in Agile-land). Handling both types of NFRs as stories does not easily allow us to map 1 NFR to multiple stories
• Elicitation: NFRs are by their nature, the most difficult class of requirements to elicit. Having explicit categorization and expectation that not only will they exist, but that they are measurable and verifiable forces serious questions to be addressed upfront in iteration-0. By forcing a clarification of measures early in the process around NFRs such as security or scalability, key design and architecture inputs may adjust trajectory
• Quality type NFRs are assertions with properties: NFRs are not functional in nature, stories are a functional artifact. Whilst it can be a useful device to use a story persona as a way to drive elicitation of NFRs, in normal complex systems where you need to put in multiple additional associated attributes; boundary values, max, min, mean, load etc., a tabular format tends to be more comfortable for people to read & document.
• Testing: Having 1 NFR, particularly Quality oriented NFRs, represented in tabular fashion with each row outlining testable measures for instance, leads to a better ability to conduct test planning. It also means that we can incrementally knock off specific rows as part of particular sprints, yet have the more general NFR in place so it’s pervasive and front of mind. Even the best intentioned team members may forget the NFR obligation unless it’s kept front of mind.
• NFR Lifecycle: Non-trivial NFRs in many cases can affect multiple sprints and have a life-cycle that lasts far longer than a 2 week sprint. Artificially closing them and re-opening them to suit sprint management is not following a principle I try to stick with, that of ‘common sense’.

Finally, the agile manifesto really says nothing about NFRs having to be represented as stories. That particular idea is really a vestige of specific agile methods. The true spirit of agile is as much about common sense and ‘fit for purpose’ as anything else. So, don’t be too afraid to stand up and say no to NFRs as stories.

Of course, if you find that NFRs as stories work for your context, excellent. It’s just that in my experience, it’s really pushing a square peg into a round hole, when we could just as easily fit a square peg into a square hole!

Scott, in common with many agilists is an advocate of iterative delivery. I’m there!

He also likes to refer to a ‘special’ type of agile iteration, at the inception of a project; ‘iteration 0’. I like this term. As a tech. guy back in the day, I also liked 0-based indexing in programming languages such as ‘c’ but that’s a different story.

Scott and other proponents of iteration 0 are articulating in ‘agilese’ something that has been done for years in regular projects, basically the Reqs Analysis phase, that is:

• In any project, regardless of size or process approach you always have to have an upfront phase to outline at a high level the project goals, business needs and tech constraints/considerations etc.
• The outputs of this upfront cycle do not produce working deployable code or slivers of systems.
• This set of activities apply equally to both approaches; waterfall and iterative.

Regardless of how you dress it up, iteration 0 (and any prior agile phases) are effectively scoping phases in ‘traditional’ language.

They are all about establishing an initial baseline of ‘just enough’ requirements, as well as other ‘preparation’ type activities such as articulating and putting in place infrastructure requirements (eg: test harness, resource allocation etc.). All extremely familiar to anyone who’s managed any kind of serious project in a traditional phased manner.

The core difference and it is a profound one, is that waterfall approaches attempt to articulate all requirements up front during that phase, expecting them not to change, whereas iterative or agile approaches stress the need to have enough to proceed and understand that the requirements will likely change & be adjusted as new information surfaces. Therefore the time for iteration 0 is short in agile projects, relative to its traditional waterfall counterpart; the requirements analysis phase.

But back to my point, for ‘non-agilists’, understanding that iteration-0 is effectively an early analysis/inception phase will I think, come as some comfort. Documenting the requirements, func. & non-func up front to a ‘reasonable’ level makes much sense. Leveraging a lightweight BRD type doc complete with lightweight use cases is a pretty decent option for larger environments striving to go more agile.

In any case, whether we call it ‘iteration 0’ or ‘requirements analysis’ makes no odds to me, so long as the result is a reasonable sense of what needs to get done in the project, i.e. requirements both func and non-func to a certain acceptable level.

On the flip side of this, if there is an important message that traditionalists should take on board from the agilists regarding this phase, it is the ‘just enough’ mantra. Making that judgement requires real business analysis & project management skill. It is not something that a 5-day Agile SCRUM course will teach. Fortunately your existing experience will be a far better determinant of success in this regard.