Relative Importance of Factors

Can someone please explain how price can be the third most important factor in descending order of importance but price is equal to all non-price factors in overall importance?

When there are three factors, including price, then price cannot be the third most important factor and yet be equal in importance to the other two factors combined.

Factor A = 3 points

Factor B = 2 points

Factor C = 1 point.

How can one point be equal to five points? I don't get it.

Can someone please explain how price can be the third most important factor in descending order of importance but price is equal to all non-price factors in overall importance? I can't seem to envision where, if design and performance capability are both more important than price, then price = both of the more important factors...

Industry has repeatedly told our agency that when we state that price = all the other factors combined, then price must be the most important individual factor.

In a recent protest (Clark/Caddell Joint Venture, B-402055, January 7, 2010), the protest decision stated the following: "Under the RFP, proposals were to be evaluated for ?best value? on the basis of the following evaluation factors listed in descending order of importance: ?design technical,? performance capability, and price. RFP at 4 and 5. The two non-price evaluation factors when combined, were equal to price.

See FAR 15.304 -- "Evaluation Factors and Significant Subfactors" for the requirement to describe relative importance of each factor and ALSO state the relative importance of price with respect to all evaluation factors other than price, when combined.

"...(d) All factors and significant subfactors that will affect contract award and their relative importance shall be stated clearly in the solicitation (10 U.S.C. 2305(a)(2)(A)(i) and 41 U.S.C. 253a((1)(A)) (see 15.204-5( c)). The rating method need not be disclosed in the solicitation. The general approach for evaluating past performance information shall be described.

(e) The solicitation shall also state, at a minimum, whether all evaluation factors other than cost or price, when combined, are --

(1) Significantly more important than cost or price;

(2) Approximately equal to cost or price; or

(3) Significantly less important than cost or price (10 U.S.C. 2305(a)(3)(A)(iii) and 41 U.S.C. 253a?(1)( C))."

Some people erroneously argue that "price or cost" is not a "factor". However, "price or cost" is discussed as a factor under 15.304 ( c). The folks who argue this with me are confused because they only evaluate technical factors, not price. They assume that since price is evaluated differently than the technical factors and by different people that it isn't a "factor".

15.304 ( c) "The evaluation factors and significant subfactors that apply to an acquisition and their relative importance are within the broad discretion of agency acquisition officials, subject to the following requirements:

(1) Price or cost to the Government shall be evaluated in every source selection (10 U.S.C. 2305(a)(3)(A) (ii) and 41 U.S.C. 253a( c)(1)() (also see Part 36 for architect-engineer contracts);

(2) The quality of the product or service shall be addressed in every source selection through consideration of one or more non-cost evaluation factors such as past performance, compliance with solicitation requirements, technical excellence, management capability, personnel qualifications, and prior experience (10 U.S.C. 2305(a)(3)(A)(i) and 41 U.S.C. 253a( c)(1)(A)); and

(3)

(i) Except as set forth in paragraph ( c)(3)(iii) of this section, past performance shall be evaluated in all source selections for negotiated competitive acquisitions expected to exceed the simplified acquisition threshold.

(ii) For solicitations involving bundling that offer a significant opportunity for subcontracting, the contracting officer must include a factor to evaluate past performance indicating the extent to which the offeror attained applicable goals for small business participation under contracts that required subcontracting plans (15 U.S.C. 637(d)(4)(G)(ii)).

(iii) Past performance need not be evaluated if the contracting officer documents the reason past performance is not an appropriate evaluation factor for the acquisition.

(4) The extent of participation of small disadvantaged business concerns in performance of the contract shall be evaluated in unrestricted acquisitions expected to exceed $550,000 ($1,000,000 for construction) subject to certain limitations (see 19.201 and 19.1202).

(5) For solicitations involving bundling that offer a significant opportunity for subcontracting, the contracting officer must include proposed small business subcontracting participation in the subcontracting plan as an evaluation factor (15 U.S.C. 637(d)(4)(G)(i))..."

(P.S., I disabled moticons but they still appeared in my preview post. I had to add a space...)

It is a bit difficult to reply without seeing the wording in the specific Section M to which you seem to refer. Let me offer the following with regard to the ultimate importance of price in a source selection: Regardless of the stated relative importance of price in the solicitation, its importance increases as the assessment of the non-price factors approaches equality.

See this quote from DIT-MCO International Corporation, B-311403, June 18, 2008:

?Although the RFP provided that the non-price evaluation factors were significantly more important than price, it also provided that, as proposals became more equal under the non-price factors, the importance of price would increase. RFP at 60. In this regard, in a negotiated procurement with a best value evaluation methodology, where selection officials reasonably regard proposals as being essentially equal technically, price properly may become the determining factor in making award, notwithstanding that the solicitation assigned price less importance?.

The paradox can be solved using negative values:

A >= B

B >= C

C >= A + B

A = -1

B = -2

C = -3

The paradox can be solved using negative values:

A >= B

B >= C

C >= A + B

A = -1

B = -2

C = -3

You haven't solved anything with respect to the relative importance of evaluation factors in source selection, but you've confirmed something that I've been thinking for a while.

It is a bit difficult to reply without seeing the wording in the specific Section M to which you seem to refer. Let me offer the following with regard to the ultimate importance of price in a source selection: Regardless of the stated relative importance of price in the solicitation, its importance increases as the assessment of the non-price factors approaches equality.

See this quote from DIT-MCO International Corporation, B-311403, June 18, 2008:

?Although the RFP provided that the non-price evaluation factors were significantly more important than price, it also provided that, as proposals became more equal under the non-price factors, the importance of price would increase. RFP at 60. In this regard, in a negotiated procurement with a best value evaluation methodology, where selection officials reasonably regard proposals as being essentially equal technically, price properly may become the determining factor in making award, notwithstanding that the solicitation assigned price less importance?.

napolik, yes - of course, as the non-price factors move toward equality, then price will tend to become the discriminator in the trade-off decision. That's not what I'm referring to here.

I agree with Vern's first post. Although Army can't use point scoring for technical or price, Vern's illustration makes the same "point" (no pun intended).

I've seen this in 2 recent GAO Decisions and am amazed that GAO didn't notice the contradiction. If such as "august" group didn't pick up on this, I figured that I am somehow missing something.

I'm beginning to think more and more that many people don't seem to have a clue how the trade-off process really should work. We as the Program Executive Ofiice for Army MILCON program execution, developed a mandatory source selection model for all Army design-build MILCON projects, in concert with a team of contracting officers and lawyers, a few years ago. We initially and purposely made price the fourth most important factor out of five factors in an attempt to make design quality, performance capability and contract duration the most important competitive factors within the budget. We also stated that price was significantly less important than the non-price factors when combined, but that price had to be both fair and reasonable and be within the contract cost limitation. We also said that additional quality would have to justify paying additional cost beyond a fully conforming proposal, etc. We said that price was more important than the fifth factor, the extent of SB/SDB subcontracting participation. This factor had mandatory requirements and annouced goals anyway. In essence we were saying that we didn't care about more extensive SB/SDB involvement than the goals if it would increase the cost beyond the budget. Thus, if all higher rated factors were essentially equal, the extent of SB/SDB participtation would become the discriminating factor.

The contracting community complained because they said they didn't know how to apply this in a trade-off comparison between proposals. Furthermore, the feedback was that they didn't know how to do a trade-off and selection where price wasn't either the most important factor or "equal to the non-price factors". Note that being the most important factor or being equal to all the other factors seems to be saying the same thing.

So, to mollify the community, our headquarters said to make price the least important factor, but still within the budget, price significantly less than non-price, etc. The reasoning was th it "would be easier to evaluate", so to speak. I've been recently told that some of our regions have started using language like that described in the GAO protest. I wonder, what do they mean? Do they even know what they are saying? I don't.

The best approach is to make all factors equally important. No one knows what that means.

I've seen this in 2 recent GAO Decisions and am amazed that GAO didn't notice the contradiction. If such as "august" group didn't pick up on this, I figured that I am somehow missing something.

I've been recently told that some of our regions have started using language like that described in the GAO protest. I wonder, what do they mean? Do they even know what they are saying? I don't.

Joel,

Until you wrote this, I just assumed the Army made a mistake and meant to say the non-price factors when combined, were significantly greater than price, and either GAO missed that or chose to ignore it. Now I see you're saying it's not a mistake. I wonder how they responded to questions from industry on this.

The requirement contained in 10 USC 2305(a)(3)(A)(iii) (and its Title 41 counterpart), at least as it has been implemented in FAR 15.303(e) has always bothered me. First, in an LPTA, there is no relation between price and non-price factors. Second, the statute already requires the agency to "clearly establish the relative importance assigned to the evaluation factors and subfactors, including the quality of the product or services to be provided (including technical capability, management capability, prior experience, and past performance of the offeror)." If you've done that right, then you've described the relative importance of price and non-price factors.

I note that the statute requires an agency to "disclose" to offerors the relative importance of price and non-price factors. The FAR went one further and required agencies to "state, at a minimum, whether all evaluation factors other than cost or price, when combined, are?

(1) Significantly more important than cost or price;

(2) Approximately equal to cost or price; or

(3) Significantly less important than cost or price"

Too many contracting specialists/officers take all this to mean that they are required to make the statement required by FAR 15.304(e) and a separate description regarding the relative importance assigned to evaluation factors and subfactors, and I don't believe that is the case. Too often, such practices yield conflicting and confused descriptions of how we intend to evaluate proposals. I spend more time editing the description of evaluation criteria than I do any other part of a solicitation.

Joel,

Until you wrote this, I just assumed the Army made a mistake and meant to say the non-price factors when combined, were significantly greater than price, and either GAO missed that or chose to ignore it. Now I see you're saying it's not a mistake. I wonder how they responded to questions from industry on this.

Formerfed, I didn't provide the chronology, but the PEO responded to industry as I explained above. We made price significantly less important than the technical factors. The Army wanted to reduce building cost by 15% and to obtain full authorized scope within the programmed amount (PA). We had been averqaging about 85% of dcope within the PA. Achieving full scope in itself would effectively result in about 18% reduction.

Obviously, when faced with cost cutting manadate, the Installations were concerned about maintaining quality of construction.

I won't go into great detail on how we reduced costs. (Using performance based design-build, standardized source selection critria and methods, standardized design criteria, using Centers of Standardization for each major facility type, maximizing use of commercial design and construction standards, standardizing and streamlining design-build contract execution, etc.).

We identified the contract cost limitation, included an aggressive upper limit on contract duration and made quality and further time reductions the most important factors. So, if we could get full scope within the PA with a much shorter schedule than pre-MT, we didn't need to emphasize further cost emphasis.

Overall, the program is a success. Last year, our Army workload was about 6 times greater than normal program and we awarded over 99 percent of the program within the overall MCA and BRAC budget.

The example I cited in the GA0 is a deviation from the published MILCON Transformation approach. In addition, it is seemingly in contradiction with itself.

The requirement contained in 10 USC 2305(a)(3)(A)(iii) (and its Title 41 counterpart), at least as it has been implemented in FAR 15.303(e) has always bothered me. First, in an LPTA, there is no relation between price and non-price factors. Second, the statute already requires the agency to "clearly establish the relative importance assigned to the evaluation factors and subfactors, including the quality of the product or services to be provided (including technical capability, management capability, prior experience, and past performance of the offeror)." If you've done that right, then you've described the relative importance of price and non-price factors.

I note that the statute requires an agency to "disclose" to offerors the relative importance of price and non-price factors. The FAR went one further and required agencies to "state, at a minimum, whether all evaluation factors other than cost or price, when combined, are—

(1) Significantly more important than cost or price;

(2) Approximately equal to cost or price; or

(3) Significantly less important than cost or price"

Too many contracting specialists/officers take all this to mean that they are required to make the statement required by FAR 15.304(e) and a separate description regarding the relative importance assigned to evaluation factors and subfactors, and I don't believe that is the case. Too often, such practices yield conflicting and confused descriptions of how we intend to evaluate proposals. I spend more time editing the description of evaluation criteria than I do any other part of a solicitation.

I disagree with you entirely.

10 USC 2305(a)(2) says:

In addition to the specifications described in paragraph (1), a solicitation for sealed bids or competitive proposals (other than for a procurement for commercial items using special simplified procedures or a purchase for an amount not greater than the simplified acquisition threshold) shall at a minimum include--

(A) a statement of--

(i) all significant factors and significant subfactors which the head of the agency reasonably expects to consider in evaluating sealed bids (including price) or competitive proposals (including cost or price, cost-related or price-related factors and subfactors, and noncost-related or nonprice-related factors and subfactors); and

(ii) the relative importance assigned to each of those factors and subfactors... .

10 USC 2305(a)(3)(A) then says:

In prescribing the evaluation factors to be included in each solicitation for competitive proposals, the head of an agency--

(iii) shall disclose to offerors whether all evaluation factors other than cost or price, when combined, are--

(I) significantly more important than cost or price;

(II) approximately equal in importance to cost or price; or

(III) significantly less important than cost or price.

You must read a statute as a whole.

FAR 15.304(d) clearly states:

All factors and significant subfactors that will effect contract award and their relative importance shall be stated clearly in the solicitation.

FAR 15.304(e) then says:

The solicitation shall also state, at a minimum... .

You must also read a regulation as a whole.

An RFP must state the relative importance of individual factors and significant subfactors and then make the summary statement required by FAR 15.304(e).

Vern, you are correct, and I was guilty of some fuzzy thinking this morning. A separate summary is required, and I've never advised a contracting officer or specialist to remove the summary.

I think the reason I chimed in this morning was because the requirement just bothers me. As noted in my post (as well as by the postings on this thread), the summary requirement seems to cause a lot of confusion. Frankly, I don't think it adds a whole lot of value (less is more nine times out of ten). And again, I ask, why in the world is that summary required for an LPTA?

...And again, I ask, why in the world is that summary required for an LPTA?

Parkerr, I do not have access to the FAR or the Federal statutory Code tonite - sorry, Vern.

However, the answer to your question might be a lack of coordination back in the rewrite of FAR Part 15 in the Fall of 2006. Prior to the rewrite, the LPTA method was discussed under a separate subparagraph from the "Best Value" FAR coverage. As part of the acquisition reform rewrite process, the FAR Council pulled LPTA under the "Best Value Continuum" with the Tradeoff Process at one end of the "Continuum" and the LPTA process at the other. I may be wrong, but don't have the present FAR or 1996 version available (I am TDY) for comparison or to check the current paragraph organization or numbering.

Going back to your original post in this thread, I do remember recently checking my FAR comparison book between the 1996 and 1997 language regarding having to separately identify the relative order of importance of each factor in addition to stating the overall importance of price and non-price. The 1996 version more clearly expressed this requirement. I don't know why they revised that specific language. as I compared the old and new language, I wondered at the time why they edited out the more clear wording. Maybe it was just part of "acquisition streamlining" to save print and what they thought were extra words.

"Under the RFP, proposals were to be evaluated for ?best value? on the basis of the following evaluation factors listed in descending order of importance: ?design technical,? performance capability, and price. RFP at 4 and 5. The two non-price evaluation factors when combined, were equal to price.?

If the above evaluation approach is accurate - what is it telling us to do?

Does the following make sense?

If we look at a notional evaluation consisting of four proposals, with two possible scores ?good? and ?average?:

Evaluation Factor Bid 1 Bid 2 Bid 3 Bid 4

Design Technical Good Good Average Average

Performance Capability Good Average Good Average

Price

In this evaluation Bid 1 is the winning bid from a non-price standpoint. Bidders 2, 3 and 4 would have to offer a lower price than Bidder 1 to make-up for their lower score. The question then becomes how much lower? Perhaps ?the two non-price evaluation factors when combined, were equal to price? helps us establish a means to calibrate this dollar threshold.

Evaluation Factor Bid 1 Bid 2 Bid 3 Bid 4

Price

More Important 5 % 10 % 15 %

Equal Important 10 % 15 % 20 %

Less Important 15 % 20 % 25 %

Therefore, if price is of a lower evaluation importance than technical the question becomes ?how much lower in price must the bid be to overcome the lower technical score?

But, if price is of a higher evaluation importance than technical then the question is reversed and becomes ?how much better technically must a bid be to overcome the lowest price?

Evaluation Factor Bid 1 Bid 2 Bid 3 Bid 4

Price Lowest 2nd Lowest 3rd Lowest 4th Lowest

Design Technical

Performance Capability

Maybe, the two questions yield different results: ?how much lower in price must the bid be to overcome the lower technical score? is not equal to ?how much better technically must a bid be to overcome the lowest price?.

It would be interesting to see if two acquisitions, RFPs and proposals received are identical in every aspect but the only difference is ?in descending order of importance? is changed to ?in ascending order of importance? in the evaluation approach would the same Bid win?

Sorry, the formatting got lost in the tables.

Sorry, the formatting got lost in the tables.

Don't get too deep in trying to figure this out. Last night, I spoke with a senior KO in the office that conducted the Source selection, who indicated that the criteria are probably a mistake and who seemed to agree with me that you can't make price both the least important factor and the most important factor at the same time. This wasn't the responsible KO.

I still don't quite understand why GAO didn't question the contradiction. I suppose it is because this wasn't a relative issue in the Protest. I know some of the folks in one of the protesting firms. I suspect they don't have a clue either.

It seems to me that if you want to make price equal to all other factors combined, then one can diminish the non-price evaluation factors to subfactors under a capability factor. This allows the capability factor to be equal to the price factor, with the subfactors listed in a descending order of importance. I know of no other way to make the math work.

My point is that you can make price equal to the other two factors, when combined. You can make price significantly less important than the other two factors when combined.

But you can't do both at the same time. In the case above, the overall stated comparison between price and non-price factors conflict with the relative importance of the three individual factors. If price is the 3rd most important factor, it can't be equal to the other 2 more factors in combination.

The cited project is design build. The two non-price factors are performance capabilty and the proposed design. The latter is distinct from "capability".

Thanks for the imput, all. I began this thread to inquire whether I was overlooking something. The responses indicate that I was on the right track.

I met the brand new lead KO for the office which was involved in the protest this afternoon. She confirmed that price can't be both the least important factor and equal to all non-price factors in combination.

Since nobody else discussed why GAO didn't comment on the apparent conflict in the RFP, I'm giving GAO the benefit of doubt. it wasn't relevant to the Protest. So, I assume that there is no need to address it, which would open the door to further basis for protest. GAO will probably avoid providing fodder...

Just ran across this old thread tonight. It is almost five years later. Based upon my continued experience in teaching Design-Build acquisition, it appears that many in the acquisition community in the organizations that attend our classes are still clueless about what they are saying when they list the relative importance of each factor and when they describe the relative importance between price or cost and combined, non-price factors.

Good thread to read.

My gripe with the order of importance is related to the fact that when Technical Factors A, B, and C have differing "importance" then how does that impact all aspects of the resultant tradeoff analysis? During the tradeoff analysis/decision, you are generally selecting the awardee(s) based on trading off (1) the perceived benefits stemming from assessed strengths, (2) the perceived disadvantages stemming from assessed weaknesses/deficiencies, and (3) the risks assessed by risks. Of course the Source Selection Official reviews strengths/weaknesses, etc., for validity, etc.

Usually during the evaluation process, I don't think evaluators are considering the order of importance.

One question I have is: when your solicitation has varying levels of importance for technical factors, does that mean that the strengths/benefits assessed for one factor have a lower baseline for the qualification of a strength than strength/benefits assessed under a less important factor? I think the answer is no since the context of the strengths/benefits are important (intelligent decision making), but it might be worth a discussion. I can see how my answer could be wrong since a protest could come about and point to misleading on the eval. criteria.

Based on the bolded above, why not always just state that all technical factors are of equal importance and then let the assessed benefits, disadvantages, and risks duel it out and determine which proposal(s) are technically superior to other proposals? Or is the reason that we state differing levels of importance for technical factors is to highlight to Offerors which factors the Government most values potential benefits from (and to also highlight the factors where weaknesses/deficiencies have more potential for a negative impact)?

I still like stating that technical factors are of equal importance from a business point of view, in most cases.

If anybody has good samples of trade off analysis (redacted of course) that you wouldn’t mind sharing, please PM me.

My gripe with the order of importance is related to the fact that when Technical Factors A, B, and C have differing "importance" then how does that impact all aspects of the resultant tradeoff analysis?

Here is how it works, using a simple example, eschewing vague terms like "strengths" and "weaknesses," and assuming no deficiencies, which would make a proposal unacceptable.

Suppose that you are evaluating "Technical" based on three subfactors, listed below in descending order of importance.

Rate of acceleration

Maneuverability

Top speed

Each subfactor is measured in its own terms, so in order to facilitate rating and ranking you convert each subfactor measurement to a rating on a common 0 to 100 point scale, 100 being best. Thus, the greater the acceleration, the higher the top speed, and the better the manuverability, the higher the rating on the 0 to 100 point scale.

In order to facilitate ranking you assign numerical weights to each factor, as follows:

Rate of acceleration = 50 percent of total technical score

Maneuverability = 40 percent of total technical score

Top speed = 10 percent of total technical score

Now suppose that you have two offerors, whose raw and weighted ratings are as follows:

Offeror A

Rate of acceleration = 80 raw points x .50 = equals 40 weighted points

Maneuverability = 95 raw points x .40 = 38 weighted points

Top speed = 60 raw points x .10 = 6 weighted points

Total weighted rating = 84 points

Offeror B

Rate of acceleration = 90 raw points x .50 = 45 weighted points

Maneuverability = 100 raw points x .40 = 40 weighted points

Top speed = 70 raw points x .10 = 7 weighted points

Total weighted rating = 92 points

Now you know that Offeror B is technically best and that Offeror A is second best. Now you can do technical/price tradeoffs. If Offeror B has a higher price than Offeror A, the question will be whether the differences in rate of acceleration, maneuverability, and top speed are worth the difference in price.

Offerors can use the information about factor ranking and weighting to make tradeoffs during their proposal development process: (1) within Technical (If we can't give both more top speed and more maneuverability, which do we sacrifice for the other?); and (2) between what they offer technically and what they ask for in price.

You can do the same kind of thing using adjectival ratings, too, but not as easily and neatly.

The more complicated you make it, the more complicated it gets.

Thanks Vern for the quick response. In my Government experience, across 3 agencies, I've never seen or used numerical weights and everything is adjectivial. The scoring approach makes sense if a factor is said to be more or less important than other factors.

I think that the adjectival rating approach can also make sense if a factor is said to be more or less important than other factors.

The proposers need to known what is most important to the goverrnment and the order of importance of the stated factors and subfactors if resources, budget or other considerations restrain what can be provided.

Sure, it can be tricky where one firm concentrates on the most important factor and another firm offers significant value in more than one other factor that, when looked at in the whole, "outweigh" the strengths or value offered in only the most important factor. However, that could occur using the numerical rating system, too.

I think that the adjectival rating approach can also make sense if a factor is said to be more or less important than other factors.

The proposers need to known what is most important to the goverrnment and the order of importance of the stated factors and subfactors if resources, budget or other considerations restrain what can be provided.

Sure, it can be tricky where one firm concentrates on the most important factor and another firm offers significant value in more than one other factor that, when looked at in the whole, "outweigh" the strengths or value offered in only the most important factor. However, that could occur using the numerical rating system, too.

My Formation of Government Contracts book, 3rd edition (outdated, I know), page 932, basically states that the overall tradeoff of the merits/dismerits of proposals trumps the stated relative of importance. I tried to trace back the court decisions that were referenced but could only find one online....

cs123 - I've been with a few agencies, and every one of them stress not using numerical ratings. Apparently sometime in the past there was a protest or something for an acquisition that used numerical ratings, and (as the tale goes) a CO was asked something like, "What is the difference between a 7 and an 8?" Or a 6 and an 8... And the CO could not provide a well thought-out answer, and thus push for non-numerical ratings (whether colors or words).

As has been described in other threads, there are a couple of major problems with numerical scoring systems.

1. many acquisition teams and selection officials have relied on the scores rather than on the underlying basis for the ratings, which is what is really important and

2. Many teams/officials tend to think that the scores denote some type of precision, as Desparado pointed out.

Another problem that I have witnessed was that rating officials would tend to back into the rating by first assigning a score, then develop a narrative to support their pre-determined rating or just ignore that requirement altogether.

The Army banned numerical weights and numerical scoring systems in 2004. The lead for the committee that revised the AFARS to ban numbers taught a class with me shortly after that and confirmed that the problems that pointed ought were behind the ban. The Air Force was already using colors that corresponded to adjectival ratings rather than numbers. I don't know what Navy was doing.

joel,

What you describe seems to be problems with people misusing numerical scoring systems--not the systems themselves.

Joel:

The problems that you describe are user problems, not system problems. If you put poorly educated and trained people to work in source selection you get stupid behavior and poor decisions.

I am sick of reading comments about problems with numerical scoring/weighting in source selection. If you take a college course in decision analysis you will learn numerical methods. There is a reason for that. They work better at what they are supposed to do, which is to simplify complex information and make it more easily digestible. Only the government would forbid the use of a perfectly workable and practically superior system because it cannot figure out how to train its people. Adjectival systems are fine if you have only two or three criteria, say: experience, past performance, and price. Otherwise, a more powerful system better facilitates analysis and integration.

Our workforce is intelligent, but it's pathetic in terms of its professional education and training. To people who think that adjectival systems are better than numerical systems, I say: You're ignorant. You don't understand the purpose of scoring systems. You don't understand what they're supposed to do and how they're supposed to work. Read a bleeping book, dummy. Don't wait for DAU or FAI to teach you. And don't wait for your boss to explain, because if he or she knew they would have explained it to you by now.

To people who want to do better than just get by and who want to be real professionals, I say: Go to Amazon.com and search in books for "decision analysis." Buy one. Read it. Think. Here are some of my favorites:

Skinner, Introduction to Decision Analysis, 3d ed.

Goodwin & Wright, Decision Analysis for Management Judgement, 5th ed. (very, very good)

Keeney & Raiffa, Decisions with Multiple Objectives: Preferences and Value Tradeoffs

Edwards & Von Winterfeldt, Decision Analysis and Behavioral Research (a magnum opus of the highest order, but only for those willing to think deeply)

Keeney, Value-Focused Thinking: A Path to Creative Decisionmaking

If you have a college education, you should be able to understand those, although you may actually have to think hard from time to time.

To everyone reading this, I say: If you're not willing to spend your money and do the hard work of reading and thinking, then go do what you can be good at -- micro-purchases. You'll have a long happy career, and you'll hardly have to think at all. If you want to be a source selection guru, then read, think, learn, and advance. To hell with DAU, FAI, and policy pukes who say numbers are bad.

Vern and Don, I don't necessarily disagree with you. I didn't personally have problems with numerical scoring system.

However, I observed much ignorance and mis-use of scoring systems. The Army directed all of its organizations to cease use of numerical scoring systems or assigning numerical weights to distinguish relative importance of factors and sub factors in April 2004 . We were also told not to debate or question the Army level decision.

I have recently observed some State Highway Departments using mechanical basis of award selection processes such as "lowest $/point" for their design-build highway projects. Those goofy systems were used by some Corps offices and were discarded in the late 1980's.

Dumb professional behavior must be changed through training or firing. Personally, I prefer training. People are smart enough, but too many agencies are letting amateurs do what is supposed to be done by pros. They just don't know how to do their jobs. Read this decision:

/legacy/a/b71eb2e7cea17180.pdf

Massive incompetence. Prof. Nash is writing about it in the January 2015 issue of The Nash & Cibinic Report. We were both shocked. This was a big acquisition. First the air tanker source selection and now this. It appears that some parts of the Air Force just can't do source selections. Whoever was in charge of that one should be relieved or reassigned. It took them longer than U.S. involvement in WWII, from December 7, 1941 to V-J Day on September 2, 1945, just to lose a protest decision. One day shy of four years from release of the RFP to the GAO decision. Four years, five if you count the draft RFP, and with what result? The GAO has recommended that they do a new proposal evaluation. Read that decision. Adjectival scoring didn't save those people.

How can something like that happen?