Why Are We Evaluating Proposals & Approaches

Start by saying what you mean by "approach" and "understanding".

What are those things?

Matthew Fleharty said:

@formerfed @FrankJon & any others:

I’m genuinely curious and want to continue the discussion, but not further derail Vern’s thread so:

What’s your best argument(s) for why the government should evaluate “approaches” or “understanding”?

In the context of professional services, which is generally what my branch purchases, I virtually always attempt to dissuade my COs and CSs, and their customers, from requesting written "approaches" as to how the contractor will execute or manage the work. I've read many of the same materials as you and find the arguments against requesting approaches in most instances extremely compelling. There's also the FAR, which in multiple spots (e.g., 12.205(a)) makes clear that a written "approach" often isn't appropriate.

However, my success rate in convincing others to change their habits is pretty low. Most COs, CSs, and customer orgs in my agency were taught that this is "the way" to do acquisition, and feel uncomfortable going without it. Breaking that muscle memory is tough. In most cases, we merely agree to reduce the maximum length of these proposals. A step in the right direction, I suppose.

All of that stated, I wouldn't go so far as to say that such a request is always without merit, as you seem to suggest. There may be acquisitions for which it's necessary to ensure that an offeror understands or intends to utilize a fundamentally sound approach to the work. In such cases, the details may be too numerous, or the scope too great, for less burdensome means such as demonstrations or oral presentations. Even if the CO disagrees with such a plan, the SSA may insist on it. And even when the CO is the SSA, I would make it happen if my customer insisted on it. My rule of thumb: When it comes to compliance decisions, I insist; when it comes to business decisions, I advise.

Finally, if the government reserves the ability in the solicitation to incorporate aspects of the approach into the final contract, I think it's reasonable to assume that the offeror will take greater care than it otherwise might have to ensure the approach is realistic.

I'm realizing I didn't directly respond to your request for "best arguments." I'll wait to see what updates you make in response to Vern's reply and give this some additional thought.

I wrote the attached in 1994 as a guest article for The Nash & Cibinic Report.

STREAMLINING SOURCE SELECTION BY IMPROVING THE QUALITY OF EVALUATION FACTORS.pdf

The commentary that follows was written by Professor Nash.

Vern Edwards said:

I wrote the attached in 1994 as a guest article for The Nash & Cibinic Report.

STREAMLINING SOURCE SELECTION BY IMPROVING THE QUALITY OF EVALUATION FACTORS.pdf

The commentary that follows was written by Professor Nash.

This is the article I've shared most with other 1102s. Unfortunately, most remain skeptical that it's usually appropriate to refrain from requesting a technical proposal.

For example, this past October, I asked my office to read this. We then got together -- about 15 of us from the SPE and HCA on down -- and discussed it. I asked for a show of hands on how many agreed with your premise that a technical proposal is usually unnecessary. One person raised his hand.

My impression is that 1102s are most concerned about taking blame from the customer if performance goes awry and we didn't request a technical proposal prior to making award.

I agree that “understanding” isn’t meaningful. I also believe that “approach” isn’t always necessary. But in some situations, evaluators need to examine how an offeror will perform. Some R&D projects is one example where offeror means of achieving results can vary widely. Performance based acquisitions using a Statement of Objectives (SOO) is another example. A SOO can allow for a wide range of competing solutions where you end up with a “competition of ideas.” Evaluators need to know details of how an offeror will perform in order to consider the respective merits. IT systems development often falls into need to see offeror approach’s as well. Issues like how much of the work will be met with existing software, how much results from reconfiguration, what needs custom developed, where are the specialized experts coming from, what methodology will be used, and what security will be employed?

Sure, one can argue that this information from proposal approaches isn’t really necessary. But when the acquisition is supporting critical missions and you need to select one company with one chance to successfully perform, getting approaches is just one way to help with decision making. If nothing more, it’s tradition.

Vern Edwards said:

Start by saying what you mean by "approach" and "understanding".

What are those things?

Thanks for the prompt Vern - it does make sense to get on the same page first.

In this context, "approach" means "information in a proposal describing how a contractor may do something." I'll not that in most cases this information is not an offer/promise.

In this context, for "understanding" I like the framework of Bloom's Taxonomy so understanding means "ability to demonstrate comprehension of something by explaining ideas or concepts."

FrankJon said:

I asked for a show of hands on how many agreed with your premise that a technical proposal is usually unnecessary. One person raised his hand.

My impression is that 1102s are most concerned about taking blame from the customer if performance goes awry and we didn't request a technical proposal prior to making award.

The term "technical proposal" is empty of meaning until we know what is "technical" and what "proposal" means. You asked for a show of hands about a concept that was needed to be clarified and fleshed out.

FrankJon said:

This is the article I've shared most with other 1102s. Unfortunately, most remain skeptical that it's usually appropriate to refrain from requesting a technical proposal.

Of course they do. Most don't think things through.

In the month after my article, The Nash & Cibinic Report included a postscript that reported the following responses:

We received the following letter from Bryan Wilkinson, Director, Compliance Guidelines, Teledyne, Inc., commenting on Vern Edwards' guest column at 8 N&CR ¶ 56:

Vernon J. Edwards' column in your October 1994 issue was so clear and sensible that I doubt it will have any impact on the procurement process.

When I was doing my doctoral work in psychology and statistics 45 years ago, the validity and reliability of ratings and rankings were a major concern, both in psychology and statistics. There was much literature published back then on the problem. There are some techniques to improve the process. Factor analysis of the rating elements is one that comes to mind. The conclusion reached by many, including myself, was that ratings or rankings will come out the way the rater wants them to, no matter the rating scheme.

When I got into the Government contracting area, I was astounded and “horrified” at the evaluation schemes used to award Government contracts. The extent of the overruns and product inadequacies should make it obvious that the selection process does not work properly.

Mr. Edwards has it right (from both a statistical and psychological standpoint) when he says use the offerors' records of experience, reputation for past performance, the qualifications of its key people, and price or cost and fee. (Though the first three factors are probably highly correlated.) These appear to provide rating factors which will result in valid and reliable ratings.

As a child of the Depression, I hate to see my tax dollars go for essay contests.

We also received the following letter from Steven Kelman, Administrator of the Office of Federal Procurement Policy:

I read with great interest the Vern Edwards article in the October issue of the report regarding streamlining evaluation factors. I strongly agree with Vern that most of our source selection process takes on the character of an essay writing contest. I also strongly agree with Vern that evaluation criteria should be centered around price/cost and past performance. I am very pleased Vern took the initiative to write this article for you, and I look forward to its arousing great interest in discussions within the procurement community--and to watching some agencies try out his suggested approach.

Vern Edwards said:

In the month after my article, The Nash & Cibinic Report included a postscript that reported the following responses:

Yes, I've come across this article before. High praise. I just wish the lesson had stuck to any meaningful extent.

Can we all agree that it is impossible to know in the present how a contractor will perform in the future?

Don Mansfield said:

Can we all agree that it is impossible to know in the present how a contractor will perform in the future?

I presume you mean how well they will perform. Since you used the word "know," I have no choice but to agree.

No, I mean how they will perform the work. The actual "approach" they will use.

In other words: process, procedure, method, technique.

I think the answer to your question depends on what we're buying. If we're buying a product (an item of supply), they might have a well-developed standard manufacturing process, which means we can know. I think we can know in the case of standard services.

Don Mansfield said:

Can we all agree that it is impossible to know in the present how a contractor will perform in the future?

We anticipate. I buy a can of beans at the market and anticipate what the beans will be like when I get it home. And I used some kind of metrics to buy the can - brand name, past experience, and even if the choices are so limited that I take what I anticipate will be good beans anyways. My thinking when the choice is limited might be a can of beans with so and so brand on it might actually come from the same manufacturing plant that not so and so's bean come from. Not much different in government contracting in my view.

So the question is by another example is what do I need to anticipate how a contractor will perform. I am quite possibly the outlier as I think I would want to know a contractors approach to building my house and their understanding of building houses to anticipate said contractors success in performing the effort of building my house.

The problem is that bureaucratic BS of FAR Part 15 is so intertwined with making a decision of anticipation it has clogged has the decision process.

Matthew Fleharty said:

In this context, "approach" means "information in a proposal describing how a contractor may do something." I'll not that in most cases this information is not an offer/promise.

formerfed said:

But in some situations, evaluators need to examine how an offeror will perform.

The evaluators are human beings and we humans have always had a saying: "My word is my bond." The evaluators misinterpret the written word as a bond - an offer/promise. Talk about your ancient concepts. Oaths are sacred.

Don Mansfield said:

Can we all agree that it is impossible to know in the present how a contractor will perform in the future?

Yes*, but that didn't stop all of humanity from entering into contracts. And you know how they made it work? Hard consequences for oath breaking. A king from ancient antiquity once even allowed the sacrificial killing of members of his tribe because he made an oath with another tribe to serve him in exchange for his protection. When he failed to ensure that protection, the king valued his bond more than his kinsmen’s lives even. A modern-day equivalent is one's guarantee of livelihood, vice an actual life. If we solicit a proposal framework that separates promises from paraphernalia, and explicitly places the hard consequence of partial payment on promise-breaking, we are applying ancient wisdom to our work. This theory is applied in an RFP that requires offers to be filled in for contract insertion, to be sure.

* I answer with 100% certainty in fixed-price contracts, but with less in flexibly priced contracts due to the post-award influence of invoice-by-invoice cost allowability determinations. In particular, terms of the contract would allow an offer's enforcement here.

This may well be the goofiest discussion of contracting I have ever read. 😂

Let me try again.

Can we know in the present what an offeror's approach will be in the future as if it were a fact?

Don Mansfield said:

Let me try again.

Can we know in the present what an offeror's approach will be in the future as if it were a fact?

No. Even if the approach were contractually stipulated, the contractor might breach the contract.

Don Mansfield said:

Let me try again.

Can we know in the present what an offeror's approach will be in the future as if it were a fact?

No.

Ok, good. If that's true, then can we agree that statements like this don't make sense?

formerfed said:

Evaluators need to know details of how an offeror will perform in order to consider the respective merits.

Except my comment is taken out of context.

In many instances where offerors proposes a variety of solutions, how can you evaluate without details?

formerfed said:

Except my comment is taken out of context.

In many instances where offerors proposes a variety of solutions, how can you evaluate without details?

It's the "need to know" that I'm taking exception to. You cannot "know" something about the future.

formerfed said:

In many instances where offerors proposes a variety of solutions, how can you evaluate without details?

Without details of what?

We can evaluate details of the offer and the offeror.

Is it necessary or even beneficial to evaluate details of their approach? If so, what's the strongest argument(s) for this assertion?

Matthew Fleharty said:

Without details of what?

We can evaluate details of the offer and the offeror.

Is it necessary or even beneficial to evaluate details of their approach? If so, what's the strongest argument(s) for this assertion?

I agree evaluation of details of approach isn’t needed. I was referring to details of the offer. Sorry for the confusion - bad choice of wording on my part.

Don Mansfield said:

It's the "need to know" that I'm taking exception to. You cannot "know" something about the future.

My mistake.  Agree

formerfed said:

My mistake.  Agree

So what can we know that is useful in an offeror's description of how they will perform a contract?

Don Mansfield said:

So what can we know that is useful in an offeror's description of how they will perform a contract?

The means they will comply with requirements without details of how. If an offeror has similar experience and solid past performance, we don’t need a step-by-step description of planned performance.

What I find most striking and saddening is that it seems clear that most workforce professionals don't think things through for themselves. They don't ask themselves questions about what the words in the regulations mean.

Take the word price. Look at how it's defined in the current FAR Part 15. Look at how it is defined in the overhauled Part 15. Now ask yourself: What is a price? What do you know about the meaning(s) of the word? Ask yourself whether the word price is suitable for the dollar figure that is written in government contracts.

In order for the acquisition process to be improved, the workforce must first improve itself. We need inquiring minds.

formerfed said:

The means they will comply with requirements without details of how. If an offeror has similar experience and solid past performance, we don’t need a step-by-step description of planned performance.

Bingo!!!

Matthew Fleharty said:

Is it necessary or even beneficial to evaluate details of their approach? If so, what's the strongest argument(s) for this assertion?

It may be necessary or beneficial in certain instances in which the agency wants to ascertain that the offeror understands what a technically sound approach is.

I have an active example, but the work is very specific and I’m reluctant to discuss publicly. The most I’ll share here is that past results do not guarantee future success. For this reason, my customer wants to do a close inspection under the hood to understand each offeror’s structure and practices. We reserve the right to incorporate any piece of the approach into the contract. (Personally, I think what my customer wants is probably overkill. But their rationale, if true, makes sense. So I defer to the judgment of the experts here despite personal doubts.)

Even if you can’t think of a specific example, isn’t it simple to imagine such a situation arising?

Can you help me to see why you believe it would never be appropriate to request a technical approach that the government doesn’t intend to incorporate into the contract?

Vern Edwards said:

What I find most striking and saddening is that it seems clear that most workforce professionals don't think things through for themselves. They don't ask themselves questions about what the words in the regulations mean.

Take the word price. Look at how it's defined in the current FAR Part 15. Look at how it is defined in the overhauled Part 15. Now ask yourself: What is a price? What do you know about the meaning(s) of the word? Ask yourself whether the word price is suitable for the dollar figure that is written in government contracts.

In order for the acquisition process to be improved, the workforce must first improve itself. We need inquiring minds.

Vern did you intend to post this in the other thread about value?

FrankJon said:

The most I’ll share here is that past results do not guarantee future success.

Neither do contracts.

I cannot think of any guarantee of success in a complex endeavor.

FrankJon said:

Vern did you intend to post this in the other thread about value?

It applies to both threads, whether the subject is value or approach or any other word in the FAR.

What does price mean? And I'm not referring to the FAR definition.

Vern Edwards said:

Ask yourself whether the word price is suitable for the dollar figure that is written in government contracts.

I contend that the majority of contracts that are firm-fixed-price at a lump sum unit of measure today could move to fixed unit-rate and we would see immediate savings and improvements in quality. But it’s not described or instructed enough in the FAR and not trained outside of construction, so it doesn’t come up in acquisition planning.

I started to say after a quick search price is what a buyer pays for an item and cost is what it takes for the seller to produce the item. Then I saw in Britannica dictionary something different. According to it “Price is generally used to refer to the amount of money that a seller is asking for something, and cost generally refers to the amount of money that buyer will spend rather than the price that a seller is asking for.”

So we can refer to contract amount as the cost government pays for a product or service.

Price seems simple, but it is a very complex idea. It's use in contracts is probably suitable only in the acquisition of commercial products, non-commercial commodity "items of supplyy, and commodified commercial and noncommercial services. It's use in contracts for construction and complex/relational work such as long-term support services and full-scale engineering development is terribly misleading, especially for members of Congress and new 1102s and technical personnel. Its use in acquisition legislation and FAR Subpart 15.4 has created a swamp of accumulated half-baked communication and policy.

To get an idea of the complexity of price read "The Definition of Price" by Frank A. Fetter, The American Economic Review, December 1912, and "What Is a Price" by Irena Asmundson in Back. To Basics, published on line by International Monetary Fund, Finance & Development, December 2013.

Those are only two of many such articles I have found online, downloaded, and read. Note the dates of the two articles.

Sometimes we use words freely that we don't really understand, thus deceiving ourselves, and thus hamstringing our policies and instructions.

And then there is cost. But we can talk about that idea another time.

On 11/6/2025 at 8:17 AM, Vern Edwards said:

Neither do contracts.

I cannot think of any guarantee of success in a complex endeavor.

This is correct. What I should have stated instead is that, with respect to my specific acquisition, past results may or may not be strongly correlated to contractor inputs. There may be a high amount of luck involved with past outcomes. For this reason, my customer insists on examining every aspect of an offeror's business before awarding a contract for this particular need.

To sum up my position on this topic, I wholeheartedly agree with the argument that requesting a written technical proposal or "approach" to the work is usually unnecessary and wasteful. But if @Matthew Fleharty is arguing that it's never appropriate to request such information unless the government intends to incorporate it into the contract, I disagree with this. I don't know how one could make such a blanket assertion across the myriad Government requirements that ever have been and ever will be. Moreover, even if this were substantively true, I disagree with the notion of "overruling" a customer who insists on requesting such information. I think such a decision rightfully falls within the "advisory" bucket of a CO's responsibilities, even when the CO is also the SSA.

On 11/6/2025 at 11:49 PM, Vern Edwards said:

Sometimes we use words freely that we don't really understand, thus deceiving ourselves, and thus hamstringing our policies and instructions.

I read a passage in Epictetus' Discourses this weekend in which he praises Plato for insisting that his students come to a common understanding of everyday words. Made me think of your recent posts, Vern. You would've been right at home in the agora....

FrankJon said:

You would've been right at home in the agora....

I don't know about that. The thing to know about Socrates was that he was a cunning questioner. If you had spoken and he then asked you a question about what you said, you were in trouble. I'm not skilled enough or patient enough to be cunning. Don Mansfield is the Socratic guy. The Athenians would have executed him along with Socrates.

But here is a book that you might enjoy reading: The Socratic Method: A Practitioner's Handbook, by Ward Farnsworth ⭐️⭐️⭐️⭐️⭐️ A really good book and a pleasant read. Especially good for contract negotiators.

Here's the problem with evaluating proposed "approaches":

The Marine Corps issued RFP M6700425R0009 on July 21 for "Logistics Integration Support". Proposals were due September 19.

The RFP is 65 pages long. It includes seven base-year CLINs. The "performance work statement" is an additional 51 pages. This was a typical FAR Part 15 competitive procurement.

The RFP required a "technical proposal" as follows:

Volume I: Technical – The Technical Proposal shall be specific, detailed, and complete so as to clearly demonstrate to the Government that the Offeror has a thorough comprehension of the solicitation requirements and the capabilities to perform these requirements throughout the performance period. The technical section must be sufficient to show how the Offeror proposes to comply with the Government's requirements and must include a full explanation of the methodology and procedures to be followed. The Offeror must also identify and evaluate any risks associated with the Offeror’s proposed approach and propose mitigation steps to minimize the Government’s risk in accepting the Offeror’s proposed solution. Your submission shall clearly demonstrate the following:

Section 1 – Technical Approach

a. Experience managing the remanufacture, overhaul and repair (ROR): The Offeror shall demonstrate they have a minimum of three years of experience managing the remanufacture, overhaul and repair (ROR) of repairable assemblies from the following four categories (Communications-electronics, Ordnance, Motor transport, and Engineer items). The offeror’s demonstrated experience must be across multiple geographically dispersed locations and on a scale similar to that required in the PWS. A similar scale is defined as a minimum volume of 7,000 repairs per year covering a minimum of 600 items/assemblies. The experience must demonstrate the offeror’s ability to act as a pass-through integrator (Third Party Logistics (3PL) provider) responsible for establishing and maintaining a vendor base capable of executing the ROR in accordance with technical manuals or OEM specifications within specified turnaround times (TAT) and with warranties offered to the customer for repairs.

b. Staffing Plan - The Offeror shall outline how staffing will be resourced and managed. The Offeror’s proposal shall provide an effective approach to coordinating its personnel and subcontractors to deliver the tasks stated in the PWS. The Offeror shall include a detailed description of the staffing approach used to provide high demand personnel, as well as the Offeror’s demonstrated ability to recruit, train, and retain sufficient numbers of personnel with required skill sets required to support the nine LIS SECREP customers located at six different geographic locations adhering to the specific equipment repair needs and security requirements identified in the PWS.

c. Management Plan - The Offeror’s proposal shall provide a management plan detailing their ability to deliver the services which meet the requirements of the PWS. The Offeror shall include a detailed description of the management plan approach to managing, teaming, staffing, communication, coordinating, resolving problems, and ensuring flexibility appropriate to the magnitude and complexity of repair work required by each LIS SECREP customer at each geographic location as detailed in the PWS.

d. Key Personnel - The Offeror shall provide the resume for all key personnel as identified in Section 4.5.5 of the PWS, which demonstrates the experience, skills, and qualifications that are necessary to perform the requirements in the PWS. The Offeror shall submit a current Letter of Intent (LOI) with all resumes of Key Personnel that are not currently employed by the Offeror or proposed subcontractor. The LOI must be signed by the individual key personnel AND the Offeror or proposed subcontractor. Personnel resumes and LOIs will not count against the page limit. Key personnel are critical to the successful performance of this contract. Personnel resumes shall be limited to two (2) pages and shall include, at a minimum, the following information: • Years of experience and education relevant to the definition of each key personnel as defined in PWS 4.5.5. If a degree is held, include the degree(s) received from and dates of attendance at accredited college(s) or university(ies). • Training • Unique or special qualifications • Tenure with the firm

e. Quality Control Plan - Offerors shall describe their approach towards providing quality control and quality assurance, including how they will ensure successful accomplishment of task order requirements and delivery of all contract/task order deliverables at a high-quality level and what steps they will take to resolve quality issues / unsatisfactory deliverables.

Section 2 – CO/CO IT Tracking System - Demonstrate that the contractor-owned, contractor-operated (CO/CO) IT tracking system meets the requirements of paragraph 5.13 of the PWS.

Section 3 – Experience Repairing & Handling CCI - Demonstrate experience repairing and handling unclassified controlled cryptographic items (CCI) per Section 5.14 of the PWS.

Section 4 – Draft Transition Plan - Explain the Offeror’s transition plan to phase into the role of the LIS SECREP integrator without a break in service in accordance with the solicitation requirements. The Offeror shall provide a Draft Transition Plan – Phase In in accordance with the PWS Section 4.9. Note: In addition, the Offeror shall explain how they plan to meet the Form DD254 requirement of a position with the required capability to obtain and maintain at least a Secret Security Clearance Eligibility to be granted access to IT-Level II sensitive information per DoDM 5200.01 and SECNAVINST 5510.30 C and have a “SECRET” level Facility Clearance Level (FCL) on Day 1 Ready of contract performance.

Section 5 – Mission Readiness - In order to ensure mission readiness, the Marine Corps (MC) customer(s) has to make a readiness decision i.e., NSN repair using organic services, NSN repair using the LIS SECREP process, or procure a replacement NSN. The decision is primarily driven by the length of time it will take to get an item back in serviceable condition in support of the warfighter. KTAT is a critical element to aiding the MC customers’ decision(s) regarding the execution of the mission and equipping the warfighter. Vendor/subcontractor capacity limitations could potentially have a negative impact on a KTAT. Therefore, it is critical to the success of the SECREP LIS repair process, that the proposed TAT/KTAT reflect capacity limitations or constraints of the ROR vendor/subcontractor.

a. Explain how the Offeror will ensure compliance with Contractual Turnaround Times (KTAT)s in the event there are vendor/subcontractor capacity limitations that compromise KTAT.

b. The Offeror shall provide a completed KTAT workbook in the attachment (solicitation Attachment J-10). The individual KTAT for each NSN shall be proposed by the Offeror in the KTAT workbook based on the number of days between when the contractor accepts an item for ROR and when the Government accepts an item back from ROR in accordance with PWS Sections 2.2 and 5.4. The KTAT workbook does not count towards the page limit for this volume.

The RFP limited the technical proposals to 100 pages. 🤣

That is a rather typical proposal preparation instruction, written by a clueless person or group of government personnel.

I don't know about you, but to me this kind of "approach" thing is absurd. There would be no chance for prospective offerors to sit down one-on-one with the Government team to go through the PWS and other terms in order ensure mutual understanding. Only the incumbent would know enough to provide well-developed responses in the time available for proposal preparation. It is highly unlikely that any of the proposals other than the incumbent's would represent a knowledgable "approach".

Anyway, that is why I think that kind of "competition" is ridiculous. The CO who put that RFP on the street either didn't care or didn't know better. I would not have let my team mates do that to themselves.

As CO, I would work with them to develop a sensible way to go about the job..

BTW, if what they really want is to get the incumbent back, then the only evaluation factors they would need are experience, past performance, and price. No need for a "technical approach." The same is true if they didn't want the incumbent back.

I could say more, explain more, but I've said all I'm going to say on this topic. My many writings over the years cover the ground more completely.

Vern Edwards said:

Volume I: Technical – The Technical Proposal shall be specific, detailed, and complete so as to clearly demonstrate to the Government that the Offeror has a thorough comprehension of the solicitation requirements and the capabilities to perform these requirements throughout the performance period

From FAR provision 52.215-1 that is included in the solicitation -

The first page of the proposal must show (emphasis added)–

(i) The solicitation number;

(ii) The name, address, and telephone and facsimile numbers of the offeror (and electronic address if available);

(iii) A statement specifying the extent of agreement with all terms, conditions, and provisions included in the solicitation and agreement to furnish any or all items upon which prices are offered at the price set opposite each item;

Thanks everyone who has contributed to the discussion. I’m working on an article that critiques the common practice of requesting approaches so I’ll have a more fulsome response for your thoughts/feedback soon.

While I rarely say always or never, I have not seen/encountered a situation in my professional career that would have benefited from evaluating approaches in competitive acquisitions - quite the contrary, I’ve only seen the practice make situations worse.

FrankJon said:

Moreover, even if this were substantively true, I disagree with the notion of "overruling" a customer who insists on requesting such information. I think such a decision rightfully falls within the "advisory" bucket of a CO's responsibilities, even when the CO is also the SSA.

I strongly disagree with this position - if most acquisition “professionals” lack competency, most customers are even more ill informed about how to properly structure a competitive acquisition’s decision making process. In almost all cases, they’re insistent on doing something because that’s what they did last time which certainly doesn’t make it right.

Vern once told me (or wrote) contracting officers don’t have “customers” we have “clients” and unlike the adage “the customer is always right,” part of doing professional work is being able to tell a client “no.”

Matthew Fleharty said:

Vern once told me (or wrote) contracting officers don’t have “customers” we have “clients” and unlike the adage “the customer is always right,” part of doing professional work is being able to tell a client “no.”

Well, rather than telling them "no," we should explain why what they want to do would be wasteful of their time and resources and then suggest something different, perhaps like oral presentations and Q&A sessions instead of written "technical proposals".

Most of a CO's clients want to conduct source selections the way they were taught to conduct them by contracting personnel. They need to be counseled, untaught, shown better ways, and persuaded rather than be ordered around. As often as not, if properly advised, their reactions will be, "Can we really do that? That would be great!"

On 11/5/2025 at 1:39 PM, Don Mansfield said:

You cannot "know" something about the future.

Euclid, Newton, and any number of mathematicians and astronomers might disagree with that assertion.

Vern Edwards said:

Well, rather than telling them "no," we should explain why what they want to do would be wasteful of their time and resources and then suggest something different, perhaps like oral presentations and Q&A sessions instead of written "technical proposals".

Most of a CO's clients want to conduct source selections the way they were taught to conduct them by contracting personnel. They need to be counseled, untaught, shown better ways, and persuaded rather than be ordered around. As often as not, if properly advised, their reactions will be, "Can we really do that? That would be great!"

Agreed

Vern Edwards said:

Euclid, Newton, and any number of mathematicians and astronomers might disagree with that assertion.

That’s disingenuous to the discussion. We were talking about holding a human decision maker to his promises, not holding an atom to its law. We are on matters of the mind, not matter in physics.

On 11/6/2025 at 11:49 PM, Vern Edwards said:

Its use in acquisition legislation and FAR Subpart 15.4 has created a swamp of accumulated half-baked communication and policy.

Can you expand on this statement?

WifWaf said:

Can you expand on this statement?

Sure. But first, tell me your definition of price.

WifWaf said:

We are on matters of the mind, not matter in physics.

Are you saying physics is not a matter of the mind? Really? Euclid and Newton are products of the mind. Have you the Elements? Pure logic. (Not physics.) Have you tried to read Newton's Principia? Do you own a copy? Have you studied the force diagrams? One of the greatest products of the mind in human history.

Anyway, my comment was addressed to Don, who will get it. Really, WifWaf, stick to procurement.

Now, how do you define price?

Vern Edwards said:

stick to procurement.

To that I will. Because:

WifWaf said:

We were talking about holding a human decision maker to his promises

From practice alone, my working definition of price flows mainly from the “meeting of the minds” necessary element of a contract in contract law. This is why in my noncommercial contracts I seek incorporation of offer elements to the maximum extent practicable. In addition, I dissuade COs from making a PCO-to-ACO handoff, as the foreknowledge and any verbal agreements are lost in transition.

Can we keep it practical like this?

WifWaf said:

This is why in my noncommercial contracts I seek incorporation of offer elements to the maximum extent practicable.

Both I and Professor Nash have warned against that practice.

In government competitive contract awards the "meeting of the minds" is purely a matter of form. In many if not most cases competitive awards are made without discussions. The minds have never met. They have signed up to who knows what and then argue about it during performance.

Now, you have told me what your definition of price "flows from", but you haven't told me your definition.

Vern Edwards said:

Euclid, Newton, and any number of mathematicians and astronomers might disagree with that assertion.

I had clarified that I was using "know" as in "know for a fact." I asked ChatGPT if one could know anything about the future as a matter of fact. Here's the response:

Excellent question — and very precise.

If we interpret “as a matter of fact” to mean in the same sense we know existing or past facts — as something that is unambiguously true, independently of belief or prediction — then the answer is no, we cannot presently know any future fact.

Here’s why, step by step:

---

1. Facts are truth-bearers about reality

A fact is a state of affairs that actually obtains in the world.
When we say, “It is a fact that the sun rose this morning,” the statement corresponds to something that has occurred.

Future-tense statements, however — “The sun will rise tomorrow” — describe what has not yet occurred.
Until the event exists in reality, there is no fact yet for the statement to correspond to.
So even if we have overwhelming evidence that it will happen, that’s still a prediction, not a fact.

---

2. Truth vs. justification

A belief about the future can be justified (based on evidence or lawlike regularities), but it cannot be verified as true until the future event occurs.
Knowledge, in the factual sense, requires both — justification and truth.
Before the event, you can have rational expectation, but not factual knowledge.

Example:

• On November 10, saying “The sun will rise on November 11” is justified.

• On November 11, after it happens, we can say “It is a fact that the sun rose.”

• But before it happens, the proposition isn’t yet a fact.

---

3. Determinism doesn’t help

Even in a fully deterministic universe (as Newton imagined), we still lack epistemic access to the future fact itself until it’s realized.
We might infer it with perfect accuracy, but what we have is deductive certainty within a model, not an observed fact about the world.

---

4. Probabilistic knowledge is not factual knowledge

Science gives us high-confidence forecasts (e.g., orbital mechanics, weather, atomic decay rates), but these describe probabilities of future states, not facts about them.
Facts are binary (true or false); probabilities are degrees of belief about truths not yet determined.

---

5. The logical asymmetry of time

Past and present facts are accessible; future facts are not, because they don’t yet exist.
That’s why philosophers distinguish between:

• Epistemic possibility (what could be known)

• Ontological status (what exists)

Future facts have no ontological status yet—there is nothing there to be known “as a matter of fact.”

---

Conclusion

As a matter of fact — in the literal, truth-corresponding sense —

We cannot know anything about the future, because future facts do not yet exist to be known.

What we can have are rational expectations, probabilistic forecasts, and law-based inferences that become knowledge once the future arrives and the facts are established.

Thanks for that Don. Good AI response. I also recommend the entry on a priori knowledge in the Stanford Encyclopedia of Philosophy.

On 11/10/2025 at 6:51 AM, Vern Edwards said:

Don Mansfield is the Socratic guy. The Athenians would have executed him along with Socrates.

That's pretty good company.

Don Mansfield said:

That's pretty good company.

Well, yeah, until they hand you the hemlock cup.

Matthew Fleharty said:

While I rarely say always or never, I have not seen/encountered a situation in my professional career that would have benefited from evaluating approaches in competitive acquisitions - quite the contrary, I’ve only seen the practice make situations worse.

I think you have to concede that it's possible that evaluating approaches in competitive acquisitions could be helpful in some situations. However, the fact that you, Vern, and I haven't encountered such a situation (or can think of one) should tell us something.

Don Mansfield said:

I think you have to concede that it's possible that evaluating approaches in competitive acquisitions could be helpful in some situations.

I think we need to agree on what is meant by "approach".

Early on in acquisition the word approach referred to a conceptual design of a product or system, and programs evaluated design approaches. What do we mean by it now? When you ask offerors to describe their proposed approaches to performing a service contract, what do we want to know?

Matthew Fleharty said:

While I rarely say always or never, I have not seen/encountered a situation in my professional career that would have benefited from evaluating approaches in competitive acquisitions - quite the contrary, I’ve only seen the practice make situations worse.

The routine practice in R&D contracting, particularly with studies, analysis, some basic and research, and the like is require technical approaches. Program/technical staff as well as contractors expect their use. In fact, FAR 35.007(c) mandates the submission:

(c) Solicitations shall require offerors to describe their technical and management approach, identify technical uncertainties, and make specific proposals for the resolution of any uncertainties. The solicitation should require offerors to include in the proposal any planned subcontracting of scientific or technical work (see 35.009).

I’m not saying it’s absolutely needed but it’s the norm and it is expected.

The question is: What do we mean by "approach"?

When we ask offerors to describe their proposed approach to this or that, what do we want them to include in the description?

Here's the definition of approach (noun) from the Chambers Dictionary:

noun

1. A drawing near

2. The stroke by which a player putsor attempts to put the ball ontothe green (also approach stroke, shot, etc; golf)

3. Access (archaic)

4. An avenue or means of access

5. The course or process of bringingan aircraft near to the airport, the stage prior to landing(aeronautics; also adjective)

6. Approximation

7. Attitude towards, way of dealing with

8. (usu in pl) advances towards personal relations

9. (in pl) trenches, etc by which besiegers strive to reach a fortress, or routes into any area of military importance

There are lots of definitions for use with proposals but most have a common theme. It’s the plan explaining how a project gets carried out to meet required objectives.

Edit: one important part of “approach” is inclusion of rational and justification for the chosen means of addressing the problem. Another important part is identification of uncertainties and ways to address those along with mitigation of risks. A plan of activities is usually sought after by program officials as well to better understand feasibility.

Vern Edwards said:

Now, you have told me what your definition of price "flows from", but you haven't told me your definition.

Price = an estimate of allowable costs + a construct of risks justifying profit. Price is a package presented quid pro quo by an offeror in exchange for an award.

I also remembered this from the Office of Naval Research from when the government was working on the OTR process. If submitting a proposal, ONR requires use of a technical proposal template which includes technical approach templates. Technical approach is just the norm for R&D.

https://www.onr.navy.mil/work-with-us/how-to-apply/submit-contract-proposal

formerfed said:

Technical approach is just the norm for R&D.

I don’t contest that this is the norm - that said, are our R&D projects generally successful (choose your measure(s))? That’s worth thinking about because, if they’re not, well we all know what they say the definition of insanity is…

Just now, Matthew Fleharty said:

I don’t contest that this is the norm - that said, are our R&D projects generally successful (choose your measure(s))?

I don’t know. But what R&D people insist upon is knowing in evaluation is what each offeror says they will do and how they will accomplish it. I don’t see that changing.

formerfed said:

[W]hat R&D people insist upon is knowing in evaluation is what each offeror says they will do and how they will accomplish it.

Instead of instructing offerors to "describe your proposed approach", why not instruct them to answer a set of specific questions?

Why shouldn't contracting officer tell technical personnel to develop a list of specific questions they want offerors to answer? Requiring them to develop a list of specific questions would force them to think through what they want to know. The CO could sit with them and edit the questions for clarity and specificity. The questions could then be ordered logically.

Instructing the offerors to answer the same specific questions in the same order would force them to write focused responses. The CO could set a maximum word count for each question, rather than an overall proposal page limit.

Every proposal would thus be structured in the same way, and every offeror's answers would be directly comparable to every other offeror's answers. Offerors could thus be evaluated and compared based on their specific responses, which would be fair and would facilitate efficient evaluation and ranking. It would also make it easier to pinpoint strengths, weaknesses, and deficiencies.

Of course, that approach would require literacy, articulacy, clear thinking, and competence on the part of all involved, so maybe it's too difficult for government work. But it would be innovative. And isn't that the buzzword of the day?

🤔

Vern Edwards said:

Instead of instructing offerors to "describe your proposed approach", why not instruct them to answer a set of specific questions?

That is an excellent idea. If the government anticipates a wide range of divergent solutions, the questions would need carefully crafted to obtain needed information from all offerors. A benefit is it maximizes chances of getting essential information from technical approaches without having to go back to offerors seeking elaboration, filling in gaps, and covering oversights.

I think it's a good idea, but I don't want to oversell it. We already have too much mindless mimicry in the contracting business.

The main idea is to put more thought into how we do what we do, instead of engaging in so much cutting and pasting.

Vern Edwards said:

I think it's a good idea, but I don't want to oversell it. We already have too much mindless mimicry in the contracting business.

The main idea is to put more thought into how we do what we do, instead of engaging in so much cutting and pasting.

Makes me wonder how long the idea when used would fall prey to cut and paste itself?

Vern Edwards said:

Instead of instructing offerors to "describe your proposed approach", why not instruct them to answer a set of specific questions?

Why shouldn't contracting officer tell technical personnel to develop a list of specific questions they want offerors to answer? Requiring them to develop a list of specific questions would force them to think through what they want to know. The CO could sit with them and edit the questions for clarity and specificity. The questions could then be ordered logically.

Instructing the offerors to answer the same specific questions in the same order would force them to write focused responses. The CO could set a maximum word count for each question, rather than an overall proposal page limit.

Every proposal would thus be structured in the same way, and every offeror's answers would be directly comparable to every other offeror's answers. Offerors could thus be evaluated and compared based on their specific responses, which would be fair and would facilitate efficient evaluation and ranking. It would also make it easier to pinpoint strengths, weaknesses, and deficiencies.

Of course, that approach would require literacy, articulacy, clear thinking, and competence on the part of all involved, so maybe it's too difficult for government work. But it would be innovative. And isn't that the buzzword of the day?

🤔

ARPA-H and DARPA have experimented asking offerors to answer the Heilmeier questions. I haven't been around either organization enough to know if has been successful.

KeithB18 said:

ARPA-H and DARPA have experimented asking offerors to answer the Heilmeier questions. I haven't been around either organization enough to know if has been successful.

The Heilmeier "catechism" is used to evaluate proposals to fund research projects. I don't know if it's been used in FAR Part 15 competitive negotiations ("source selections"). I doubt it.

For those who don't know about it, here are some links"

https://www.darpa.mil/about/heilmeier-catechism

https://www.depts.ttu.edu/research/ordc/Resources/heilmeier-catechism.php

https://userpages.cs.umbc.edu/finin/home/heilmeyerCatechism.html

https://userpages.cs.umbc.edu/finin/home/heilmeyerCatechism.html

On 11/11/2025 at 7:16 AM, Vern Edwards said:

Most of a CO's clients want to conduct source selections the way they were taught to conduct them by contracting personnel. They need to be counseled, untaught, shown better ways, and persuaded rather than be ordered around. As often as not, if properly advised, their reactions will be, "Can we really do that? That would be great!"

To be sure, the message and messenger matter. However, status quo bias in this area is stronger than you think. It's important to realize that, until January 2025, most customers at most agencies became accustomed to a carousel of 1102s coming and going. Most of these 1102s were unqualified and unwilling to learn. Each gave different advice and established different rules for the customers to follow, some of which was right and much of which was wrong.

Beyond this, it's part of human DNA to continue wanting to do what has "worked" to avoid embarrassing or costly errors. Humans are also fixated on results over processes, easily confusing a strong outcome for a strong process.

In this context, it's not hard to understand a customer's insistence on requesting written technical proposals, just as s/he's always done. As a manager and CO, once I encounter resistance, I do my best to find a middle ground. If I can help teach my employees and customers do acquisition a little bit better than they did before, I count that as a win.

FrankJon said:

However, status quo bias in this area is stronger than you think.

No, it's not stronger than I think.

On 11/11/2025 at 3:38 PM, WifWaf said:

Price = an estimate of allowable costs + a construct of risks justifying profit. Price is a package presented quid pro quo by an offeror in exchange for an award.

Would anyone like to challenge me on this? Allow me to show my work. I am looking to sharpen it up.

We know a cost is allowable only when the cost complies with all of the following requirements:

(1) Reasonableness.

(2) Allocability.

(3) Standards promulgated by the CAS Board, if applicable, otherwise, generally accepted accounting principles and practices appropriate to the circumstances.

(4) Terms of the contract.

(5) Any limitations set forth in FAR Subpart 31.2.

We know the negotiation of a contract type and a price are related and should be considered together with the issues of risk and uncertainty to the contractor and the Government. FAR 15.405(b).

At time of award, then, these five factors determining allowability plus all issues of risk and uncertainty known to be possible (by at least one party) produce a price. The "something for something" nature of a quid pro quo exchange that occurs, therefore, is only sustained so long as both parties' "somethings" do not substantially change in any of the five points. Likewise, so long as they do not change in any risk not inherently assumed by contract type. When one changes substantially, the affected party may be equitably adjusted with a new price, or else the price may no longer be fair and reasonable.

Think of the example of a cost accounting change. Here allocability has changed, therefore the government explores the magnitude of it and makes a materiality determination and seeks recovery of a cost impact. That’s also FAR 52.230-6 terms of the contract.

Vern Edwards said:

No, it's not stronger than I think.

❤️ 😂

Vern Edwards said:

No, it's not stronger than I think.

In my opinion this --

On 11/11/2025 at 7:16 AM, Vern Edwards said:

As often as not, if properly advised, their reactions will be, "Can we really do that? That would be great!"

--is an overly optimistic expectation for most agencies in most scenarios.

@FrankJon

Well, yeah, that statement was probably overly optimistic. 😂 😏 😞

On 11/11/2025 at 12:38 PM, WifWaf said:

Price = an estimate of allowable costs + a construct of risks justifying profit. Price is a package presented quid pro quo by an offeror in exchange for an award.

Although I could quibble with some of tht wording ("a construct of risks justifying profit"), and some people might quibble with price being an estimate, I think that even a so-called "firm-fixed price" is often nothing more than an estimate of what the government will have to pay for goods and services, e.g., construction and system full-scale development.

Price is a very complicated idea to which the current FAR Subpart 15.4 and the overhauled Subpart 15.4 do not do justice.

Great discussion in this thread and I agree 100% that in most, and I mean almost all, cases something like approach shouldn't be looked at. It's funny how reading a thread will open up old memories. A long time ago we needed to have some trees removed at a remote location at an outpost, none of us even knew it existed. We brought a three companies up at different times to look at the area. Of course in casual conversation we asked about the job. The first two said they would bring a truck in and cut then down. Pretty simple and how we would expect it to be done. The third company said they would do the same but they would use ground protection mats. None of us, including the requirement owner, had any idea what ground protection mats were. The company let us know rainy season was about to start and without those mats any truck would leave 2-3 foot ruts throughout the back. This was just some measly purchase order but that company let us know something we had no clue about.

Again, I don't think approach should be used, but I can't say it should "never" be used.

In the holiday spirit I will use a part of The Santa Clause (now that's a contract!)

1. Neil: "What about Santa's reindeer? Have you ever seen a reindeer fly?"

2. Charlie: "Yes."

3. Neil: "Well, I haven't."

4. Charlie: "Have you ever seen a million dollars?"

5. Neil: "No."

6. Charlie: "Just because you can't see something, doesn't mean it doesn't exist."

@Scrutor

What you have described is a failure of requirements analysis and market research. Asking for a proposed approach should not be a substitute for competent requirements analysis. But I think you already know that.

Vern Edwards said:

@Scrutor

What you have described is a failure of requirements analysis and market research.

Bingo - there is a difference between wanting to understand various approaches during market research and evaluating approaches (which are usually not promises) during source selection.

@Scrutor Had your story occurred during a source selection (not some measly purchase order) and the one offeror wrote that as their approach, received the contract because the evaluators liked what they read, and then started to perform without using the ground protection mats (let's say because using the mats would have cost more money and a dollar saved is a dollar earned for the company), what then?

Matthew Fleharty said:

@Scrutor Had your story occurred during a source selection (not some measly purchase order) and the one offeror wrote that as their approach, received the contract because the evaluators liked what they read, and then started to perform without using the ground protection mats (let's say because using the mats would have cost more money and a dollar saved is a dollar earned for the company), what then?

Matt, it would all depend if the language the contractor used was promissory in nature, and then if said promissory language was written into the contract. In general I highly doubt it would be written like that, so in general the gov would be SOL. I have very, and I mean very, little faith that most contracting people understand and can identify promises in a proposal even though they think that can. It's why right now at some work center a KO is explaining to a clueless requirement owner why the contractor only has ten people working when their proposal said fifteen.

Vern mentioned requirement analysis and market research. Those are key pieces. The lack of doing both is a major contributor to contract failure. So many acquisitions are based upon the government not focusing on the specific need and not understanding what industry can best provide.

One of my favorite questions to ask industry leaders when performing market research is “if you were in our position (the government), how would you proceed in satisfying the need?”

On 11/12/2025 at 7:51 AM, Vern Edwards said:

Instead of instructing offerors to "describe your proposed approach", why not instruct them to answer a set of specific questions?

Why shouldn't contracting officer tell technical personnel to develop a list of specific questions they want offerors to answer? Requiring them to develop a list of specific questions would force them to think through what they want to know. The CO could sit with them and edit the questions for clarity and specificity. The questions could then be ordered logically.

Instructing the offerors to answer the same specific questions in the same order would force them to write focused responses. The CO could set a maximum word count for each question, rather than an overall proposal page limit.

Every proposal would thus be structured in the same way, and every offeror's answers would be directly comparable to every other offeror's answers. Offerors could thus be evaluated and compared based on their specific responses, which would be fair and would facilitate efficient evaluation and ranking. It would also make it easier to pinpoint strengths, weaknesses, and deficiencies.

Of course, that approach would require literacy, articulacy, clear thinking, and competence on the part of all involved, so maybe it's too difficult for government work. But it would be innovative. And isn't that the buzzword of the day?

🤔

Just so we're on the same page, while this method has its benefits, the offerors' responses would still come within Matthew's definitions of "approach" and "understanding," would they not?

On 11/5/2025 at 10:37 AM, Matthew Fleharty said:

In this context, "approach" means "information in a proposal describing how a contractor may do something." I'll not that in most cases this information is not an offer/promise.

In this context, for "understanding" I like the framework of Bloom's Taxonomy so understanding means "ability to demonstrate comprehension of something by explaining ideas or concepts."

Matthew Fleharty said:

Bingo - there is a difference between wanting to understand various approaches during market research and evaluating approaches (which are usually not promises) during source selection.

And a difference of opinion on who has responsibility for the market research and evaluation of approaches. Whether that responsibility be implied, imperative or even elective. Long gone is the era of adequate resources that allow for a contracting professional to be actively involved in the acquisition process cradle to grave.

Just now, FrankJon said:

Just so we're on the same page, while this method has its benefits, the offerors' responses would still come within Matthew's definitions of "approach" and "understanding," would they not?

Assuming that question was addressed to me, my answer is that it would depend on the questions.

In many markets, the government cannot obtain robust market research from outside. Even when it does obtain it from outside the market, its result is program people describing requirements with which they have little recent experience. When requirements writers do not have great experience doing the work, they ask for approaches and understandings, so that they can make their evaluation decisions more binary (strength or weakness).

If you must gain research by actually entering the market, you might try this newly described FAR Part 15 source selection approach.

15.103-4 Phased acquisition.

(a) Phased acquisitions break down complex or high-risk acquisitions into distinct parts for contract award. Instead of a single and comprehensive proposal and evaluation process, phased acquisitions are structured to make separate awards at distinct phases of the acquisition. This approach can reduce source selection timeframes and promote flexibility by permitting adjustments to the requirement between phases.

(b) When using a phased approach, the RFP must—

(1) Establish the phases for the acquisition; and

(2) Describe the process that will be used to evaluate proposals and any criteria offerors must meet to progress to the next phase in the RFP.

WifWaf said:

In many markets, the government cannot obtain robust market research from outside. Even when it does obtain it from outside the market, its result is program people describing requirements with which they have little recent experience. When requirements writers do not have great experience doing the work, they ask for approaches and understandings, so that they can make their evaluation decisions more binary (strength or weakness).

• A much better market research approach in my opinion consists of these steps:

• Work with program officials to define requirements succinctly at a high functional level

• Research to identify those organizations that perform this function well - can be commercial sources known not interested in proposing and other government agencies as well

• Hold one on one discussions to identify how they perform the work; what are best practices; what makes for top performers; who are their customers; what are lessons learned; and how would they proceed next if they were in your government shoes

• Put together a draft and circulate for theirs and others sources input

You end up with a brief statement of requirements without all the unusual inefficiencies, meaningless steps, individual personal preferences, and waste. Do this at the very beginning and in parallel with other activities and it doesn’t slow down anything. In fact, it likely saves overall time.

WifWaf said:

When requirements writers do not have great experience doing the work, they ask for approaches and understandings, so that they can make their evaluation decisions more binary (strength or weakness).

@WifWaf Are they competent to evaluate "approaches" if they don't have experience? If they don't have experience, how would they know what to ask about an "approach"? How would they know which "approach" is better?

They may just be asking for "approaches" because the contracting officer has not shown them another, more effective, more efficient way to evaluate offerors and their offers.

This is where COs must use professional knowledge to take a leadership role.

I think we have exhausted this topic.

Read this:

ESSAY-WRITING CONTESTS How Did We Get Here.pdf