DoD Source Selection Procedures (March 31, 2016)

The new guidelines expressly admit that the purpose of VATEP is "taking some of the subjectivity out of the best value evaluation" (see para. 1.3.1.4).

The VATEP approach is an attempt to quantify or dollarize the tradeoff decision, and to take away subjective discretion from the selecting official -- FAR 15.308 specifically says the selection documentation "need not quantify the tradeoffs that led to the decision," and the GAO routinely reinforces this principle.  However, maybe some selecting officials in DoD don't want to make subjective best value decisions?  Or someone at a higher level doesn't trust them?  Regardless, use of VATEP seems to be optional.

Use of VATEP is optional so I don't buy the concern regarding distrust at the higher levels.  Furthermore, according to the guidance, the purpose is not merely to "[take] some of the subjectivity out of the best value evaluation" but also (according to the preceding sentence) to "[provide] the offeror information to determine if the additional cost of offering better past performance will put the offeror in a better position in the source selection" (1.3.1.4).  This method is just another tool in the toolbox - if the acquisition team decides that giving away subjective tradeoff flexibility is in the best interest of the Government so that offerors are more informed regarding the value of higher technical performance, I don't see how that is inherently bad.

You're right -- it's not inherently bad -- and in a disciplined acquisition with knowledgeable and empowered participants, the VATEP process might actually work.  But I do know that there are some decision-makers who do not want to make difficult and subjective decisions, and some reviewers who don't want decision-makers to make difficult and subjective decisions -- they are the wrong people for the VATEP process.  I hope VATEP is only used where it makes sense.

It looks like performance and price tradeoff with a pass/fail technical factor is no longer an option. This should be corrected.

Considering risk for all evaluations involving technical, with the exception of LPTA, doesn't always make sense and can complicate simple technical factors such as experience. This should be corrected.

I'm heavily considering leaving the government and returning to industry. I don't understand how we expect to retain top talent with all the dysfunction within the career field. It's maddening at times.

Maybe they'll ask the GAO to draft the next version - it might be more helpful.

Jamaal Valentine said:

It looks like performance and price tradeoff with a pass/fail technical factor is no longer an option. This should be corrected.

Jamaal Valentine, do you think you can describe and execute (the functional equivalent of) a performance price tradeoff with a pass/fail technical factor within the existing, current rules of FAR Part 15 and the DoD SSP? PepeTheFrog is not asking whether a risk-averse, internal reviewer who needs explicit, written direction to feel safe will approve it. Assume your entire chain of command has framed copies of FAR 1.102(d) on the wall.

Along these lines of discussion, PepeTheFrog thinks that VATEP (or its functional equivalent) was available before this revision of the DoD SSP.

Jamaal Valentine said:

It looks like performance and price tradeoff with a pass/fail technical factor is no longer an option. This should be corrected.

Considering risk for all evaluations involving technical, with the exception of LPTA, doesn't always make sense and can complicate simple technical factors such as experience. This should be corrected.

I've read through the updated DoD SSPs (admittedly only once, so I may have missed it), but I don't see any language prohibiting the use of performance and price tradeoff with a pass/fail technical factor.  Could you reference the section/paragraph that does?

In fact, I think the language under section 1.3  (pg. 2-3) still allows the use of the technique (emphasis added below):

Quote

This document describes source selection processes and some techniques that may be used to design competitive acquisition strategies suitable for the specific circumstances of the acquisition including: Value Adjusted Total Evaluated Price (VATEP) tradeoff source selection process with monetized adjustments included in the evaluated price for specific enhanced characteristics; tradeoff source selection process with subjective tradeoffs; and lowest price technically acceptable (LPTA) source selection process.  These are not the only source selection processes available on the best value continuum.  SSTs should carefully consider and use the approach that is most appropriate for their acquisition.

In other words, just because the DoD SSPs are silent on that particular technique, barring any language within the SSPs prohibiting the use of the technique it appears to be permissible.

I don't think they explicitly prohibited it per se. Rather, I think we would have to reconcile the shall statement in section 2.3.4.2.1, Technical Risk, (page 20).

I'm unable copy and paste the specific section from my phone, I get an HTML error on the post.

Admittedly, I've only read it once, but that was my first take from the passage. Additionally, I believe there are standard use evaluation tables, under the "one of two ways" methodologies when including technical factors other than LPTA (see page 24 - 26).

*Edited to add quote (emphasis added)

Quote

All evaluations that include a technical evaluation factor shall also consider risk,
separately or in conjunction with technical factors, with the exception of LPTA where the technical
proposal is evaluated only for acceptability based on stated criteria.  Risk can be evaluated in
one of two ways:

•    As a separate risk rating assigned at the technical factor or subfactor level (see paragraph
3.1.2.1).

•    As one aspect of the technical evaluation, inherent in the technical evaluation factor or
subfactor ratings (see paragraph 3.1.2.2).

I don't have my earlier edition of the DOD Source Selection procedures with me in Atlanta (and I don't feel like looking it up tonite on the Internet on the iPhone), but I'm pretty certain that it treated offerors with no record of past performance similarly to this latest version. In addition to what has been said above about not covering every possible situation, one should read the applicability provisions of the procedures.  Per paragraph 1.2, they are applicable to negotiated Part 15 procedures exceeding $10 million. They don't apply to commercial item acquisitions under Part 12 below $10 mill or to commercial item acquisitions above that threshold that don't mix Part 15.3 source selection procedures, acquisitions under Parts 8.4,13, 14 , 16.505, fair opportunities procedures and other exceptions.

Quote

Per paragraph 1.2, they are applicable to negotiated Part 15 procedures exceeding $10 million.

Read the memo again, Joel. See 1.2, Applicability and Waivers, first sentence:

Quote

These procedures are applicable to all acquisitions conducted as part of a major system acquisition program, as defined in Federal Acquisition Regulation (FAR) 2.101, and all competitively negotiated acquisitions with an estimated value greater than $10 million.

Emphasis added.

The plain language says that the procedures apply (a) to "all" acquisitions conducted "as part of" a major system acquisition program and (b) to negotiated acquisitions that are valued at $10 million or more. It doesn't say that they apply to all acquisitions conducted as part of a major system acquisition program that have an estimated value greater than $10 million.

A major acquisition program consists of many individual acquisitions, large and small, including $500,000 (and smaller) study contracts and experiments to $1 billion system development contracts.

I have read it.

It is a sad piece of work--a rambling 71 page mess that is needlessly wordy, often confusing, and sometimes incoherent. It shows that the authors don't understand fundamental and crucially important ideas. (Don't they read books?) And it fails to provide helpful guidance about topics for which guidance is is badly needed. (Don't they read protest decisions?) It's not just flawed. It's an embarrassment.

I wish I could say that it's a disappointment, but the sad truth is that while I had hoped for better, I didn't expect it. I've talked to and corresponded with others who are horrified, even outraged, but resigned.

When I think of some of the astonishingly good work American bureaucracy has produced in the past, first rate stuff, I can only feel deep regret over how far our bureaucracy has fallen. It's very sad. Why not the best?

What a chance missed.

Vern Edwards said:

Read the memo again, Joel. See 1.2, Applicability and Waivers, first sentence:

Emphasis added.

The plain language says that the procedures apply (a) to "all" acquisitions conducted "as part of" a major system acquisition program and (b) to negotiated acquisitions that are valued at $10 million or more. It doesn't say that they apply to all acquisitions conducted as part of a major system acquisition program that have an estimated value greater than $10 million.

A major acquisition program consists of many individual acquisitions, large and small, including $500,000 (and smaller) study contracts and experiments to $1 billion system development contracts.

In reading the plain language, it would seem that the exceptions in 1.2 still apply to acquisitions under major systems acquisitions.  

The various DOD Activities usually have further implementing instructions.

WARNING: The description and explanation of the VATEP method in Appendix B are complex and obscure. The method strikes me as complicated. I fear that its use will create a protest-rich environment if not done very, very carefully. Don't allow yourself to be seduced by any notion that it will be less likely to prompt a protest because it's an "objective" method.

joel hoffman said:

In reading the plain language, it would seem that the exceptions in 1.2 still apply to acquisitions under major systems acquisitions.  

The various DOD Activities usually have further implementing instructions.

I agree about the exceptions. As I read the first sentence of 1.2, there are two alternative applicability criteria for competitive negotiated acquisitions: (1) whether the acquisition is "part of" a major system program, regardless of dollar value and (2) whether the acquisition is worth more than $10 million, whether part of a major system program or not. The procedures do not apply to any acquisition that falls under any of the exceptions.

The decision logic might work as follows:

1. Do any of the exceptions listed in 1.2.1 apply? If yes, then the source selection procedures do not apply. If no, go to 2.

2. Is the acquisition "part of" a major system program? If yes, then the source selection procedures apply. If no, go to 3.

3. Is the acquisition worth more than $10 million? If yes, then the source selection procedures apply. If no, then the procedures do not apply.

There is provision for waivers.

That's what the memo says. They might have meant that the procedures apply only above $10 million. If that had been the case it would not have been necessary to mention major system programs.

I agree.

i tried to edit my last post to cite paragraph 1.2.1 for the exceptions but was unable to post the edit on iPad .

Quote

1.2.1 These procedures are applicable to all competitively negotiated procurements meeting the requirements in paragraph 1.2, except those using...

Vern Edwards said:

WARNING: The description and explanation of the VATEP method in Appendix B are complex and obscure. The method strikes me as exceedingly complex. I fear that its use will create a protest-rich environment if not done very, very carefully. Don't allow yourself to be seduced by any notion that it will be less likely to prompt a protest because it's an "objective" method.

PepeTheFrog agrees, and sees the same danger as reliance on numerical scoring (rather than underlying strengths and weaknesses from adjectival ratings). Like numerical scoring, VATEP provides a veneer of objectivity that might not hold up in protest scrutiny: "Subjective and unreasonable decision without adequate documentation?!? Just look at these fancy VATEP graphs!"

VATEP will also create demand for consultants, instructors, gurus, etc. to explain the mystical intricacies of VATEP to the crowds.

PepeTheFrog thinks that DoD and other large bureaucracies make similar mistakes, as exemplified by the DoD SSP. It seems like the DoD SSP provides too much guidance; it's overkill. It could have been more concise, and provided a longer leash for contracting professionals. But identifying, evaluating, and correcting mistakes and bad actors takes time and effort on the part of leadership and management. Instead, bureaucracies create blanket, reactive, constrictive, and burdensome "solutions" that apply to everyone, including those who can perform well in the absence of the "solution."

Oh Oh, Pepe the Frog, prepare to be "gigged". You mentioned numerical scoring systems...

"a veneer of objectivity" Sigh.  No one reads books.

joel hoffman said:

Oh Oh, Pepe the Frog, prepare to be "gigged". You mentioned numerical scoring systems...

PepeTheFrog just learned what "gigged" means and is hopping away in horror!

https://en.wikipedia.org/wiki/Gigging

Vern Edwards said:

"a veneer of objectivity" Sigh.  No one reads books.

Vern Edwards, did you not like the phrase, or did you not like its application to numerical scoring? Will you explain?

Hi Pepe:

Love the avatar and moniker.

The long-standing knock on numerical scoring is that it falsely indicates objectivity. Numerical scoring doesn't do that. Rather, some people who interpret numerical scores interpret them that way. When used by people in the know, numerical scoring is a powerful analytical tool that is far superior to adjectives and colors. I will quote from a very great book, written under contract to the U.S. Navy by two very great decision scientists, Detlof von Winderfeldt and the late Ward Edwards. The book is Decision Analysis and Behavioral Research (Cambridge University Press, 1986). The quote that follows is from page 20:

Quote

Numerical subjectivity

The SEU [subjectively expected utility] model [of decision-making] embodies a fundamental principle. Both the utilities and the probabilities that enter into it are numbers, but they are inherently subjective in the sense that you and your colleagues might disagree about them and would have no way of resolving any such disagreement. The fundamental principle might be called numerical subjectivity, the idea that subjective judgments are often most useful if expressed as numbers. For reasons we do not fully understand, numerical subjectivity can produce considerable discomfort and resistance among those not used to it. We suspect this is because people are taught in school that numbers are precise, know from experience that judgments are rarely precise, and so hesitate to express judgments in a way that carries an aura of spurious precision. Judgments indeed are seldom precise--but the precision of numbers is illusory. Almost all numbers that describe the physical world, as well as those that describe judgments, are imprecise to some degree. When it is important to do so, one can describe the extent of that imprecision by using more numbers. Very often, quite imprecise numbers can lead to firm and unequivocal conclusions. The advantage of numerical subjectivity is simply that expressing judgments in numerical form makes it easy to use arithmetic tools to aggregate them. The aggregation of various kinds of judgments is the essential step in every meaningful decision.

Emphasis added.

Back in the 1970s and early 1980s, the GAO encountered faulty use of numerical scoring by source selection personnel in a number of protest cases and complained that the method lent a false air of objectivity and precision to what is a matter of subjective judgment. But numbers didn't do that; poorly educated and trained human beings did that. Still, the GAO has not said that numerical scoring cannot or should not be used. See IAP World Services, Inc.; Jones Lang LaSalle Americas, Inc., B-411659.2, 2015 CPD ¶ 302:

Quote

IAP argues that the agency used the artificial precision of a weighted numerical score to gloss over what would otherwise be a complex weighing of proposals' strengths and weaknesses. IAP Protest at 17–19; IAP Comments (July 27, 2015) at 31. The record shows that although the SSA's best value tradeoff discussion makes multiple references to the offerors' numerical scores, the SSA was well aware of the strengths and weaknesses underlying the evaluations. See AR, Tab 33, Source Selection Statement, at 20 (“I conducted my deliberations by first looking at the overall evaluated scores for Mission Suitability and the findings that led to those scores....”) While numerical scoring cannot substitute for a nuanced evaluation of proposals, here, the selection official had a basis to conclude that the scores were an accurate representation of the relative strengths and weaknesses of the proposal.

There we have a case of an SSA who knew what he or she was doing. However, some agencies (e.g., DOD) have prohibited the use of numerical scoring rather than properly train their people in the use of that extremely effective tool. Ironically, DOD's contractors use numbers when conducting systems engineering tradeoff analyses, and no one thinks anything of it.

Sigh.

There was a time when I used to try to explain all this to contracting people, but ignorance is so deeply rooted among us that I no longer waste my time. I have learned that many people in our business simply will not read books and will not study and prepare themselves for the tasks ahead unless their boss sends them to some government-sponsored class or they can hop on Wifcon for a quick and easy answer. They have allowed their ignorance to deny them the use of a powerful tool for the conduct of complex source selections. In explaining now I am making an exception, because I like your style and because I sense a keen intelligence behind the humor.

I will not discuss this matter further with anyone.

Vern

Vern, I know you said you wouldn't discuss this matter further with anyone, but I'm interested whether or not the use of numerical scoring (back in the 1970s/1980s or even today) prompts protests during the solicitation phase with potential offerors arguing that the numerical scoring system chosen "stacks the deck" against them (or in favor of another offeror)?  I'm not asking the question to imply that a potential increase in protests during the solicitation phase would be a reason to forego the use of numerical scoring due to the fear of protests, I'm merely curious.  Also, thanks for the reference to scholarly material regarding numerical scoring/decision making - I hope you'll make an exception to your declaration not to discuss this topic anymore.

If you're not willing to discuss numerical scoring (or even if you are), would you be willing to discuss your vision for Source Selection Procedures that provide helpful guidance for the DoD contracting workforce?

A current challenge is centralizing the rules (physically where to find the rules to the game). Seems we create our own write-ups by failing to follow an agency procedure or policy more often than we fail to comply with statutes or regulations. DoD SSP will lead to agency FAR supplement changes, several more layers of informational guidance and mandatory procedures, etc. I wish a lot of the superfluous additions would include sunset provisions of mandatory review periods.

But let's not waste an opportunity. There should be a some public comment periods, for FAR supplement changes, and we can voice our opinions. I think DAU and leaders should allocate resources to teaching practitioners on the Federal Register process. A lot of people are unhappy with the rules, but never comment when given an opportunity. It's like complaining about politicians, but never voting.

Heck, did the DoD SSP go out for public comment? If not, should it have?

Matthew:

Quote

Vern, I know you said you wouldn't discuss this matter further with anyone, but I'm interested whether or not the use of numerical scoring (back in the 1970s/1980s or even today) prompts protests during the solicitation phase with potential offerors arguing that the numerical scoring system chosen "stacks the deck" against them (or in favor of another offeror)?

I don't recall having ever seen a protest against the use of a numerical scoring scheme during the solicitation phase. That doesn't mean that there has not been one, just that I don't recall having seen one. Protests involving numerical scoring have generally arisen in reaction to the source selection outcome. Complaints seem mainly to have been about the way scores were assigned or distorted tradeoffs, i.e., based solely on numbers and without consideration of actual strengths and weaknesses.

Jamaal Valentine said:

Heck, did the DoD SSP go out for public comment? If not, should it have?

I think the DOD source selection procedures are strictly internal. As such they need not be published n the Federal Register.

I was thinking the same, but based on some of Don's latest posts I'm unsure. Curious to know how they reconcile FAR 1.401(e),1.401(f), and 1.404 requirements.

I think a reasonable argument can be made that the DoD SSP has a significant effect beyond the internal operating procedures of the agency or has a significant cost or administrative impact on contractors or offerors.

Jamaal,

i'm with Vern.  The guidelines are strictly internal -- they impose nothing on offerors or contractors.  Apparently, others agree and the guide had not been put out for public comment.

Matthew Fleharty said:

If you're not willing to discuss numerical scoring (or even if you are), would you be willing to discuss your vision for Source Selection Procedures that provide helpful guidance for the DoD contracting workforce?

At the very least, off the top of my head, good source selection guidance would address:

• source selection as a project and source selection project management;
• the two objectives of the source selection process: (1) contractor selection and (2) contract formation;
• the three main functions of the solicitation;
• the nature, content, and two-part organization of proposals:
• solicitation quality management: control and assurance;
• the nature and selection of evaluation factors and the meaning and application of "relative importance";
• the two steps of the evaluation process;
• evaluation team organization, training, discipline, and management;
• the distinction between evaluation and rating (scoring);
• assessing proposals against factors;
• the purpose and types of proposal rating (scoring) systems;
• content, preparation, and presentation of evaluation documentation;
• the distinction between and uses of clarification and discussion;
• deciding whether to conduct discussions;
• establishing a "competitive range";
• pairwise proposal comparisons and proposal ranking;
• tradeoff analysis methods and decision logic;
• decision validation;
• content and presentation of decision documentation;
• the all-important distinction between decision explanation and decision justification;
• notice and debriefing management and conduct;
• protest venues, remedies, and procedures.

ji20874:

How could the guide not affect offerors, they are the other party to solicitations? I'm not insisting just throwing it around.

If all evaluations with technical shall evaluate risk isn't that a consideration offerors will have to account for in their offer? Proposal preparation is critically important to business, time consuming, and expensive.

Could way say offerors are impacted in regards to how they evaluate Government requirements/solicitations and prepare offers? Again, just throwing it around without much slow thinking.

The procedures are in DFARS PGI 215.3.

Jamaal, everything affects offerors...

Vern Edwards said:

Hi Pepe:

Love the avatar and moniker.

The long-standing knock on numerical scoring is that it falsely indicates objectivity. Numerical scoring doesn't do that. Rather, some people who interpret numerical scores interpret them that way. When used by people in the know, numerical scoring is a powerful analytical tool that is far superior to adjectives and colors. I will quote from a very great book, written under contract to the U.S. Navy by two very great decision scientists, Detlof von Winderfeldt and the late Ward Edwards. The book is Decision Analysis and Behavioral Research (Cambridge University Press, 1986). The quote that follows is from page 20:

Quote

Numerical subjectivity

The SEU [subjectively expected utility] model [of decision-making] embodies a fundamental principle. Both the utilities and the probabilities that enter into it are numbers, but they are inherently subjective in the sense that you and your colleagues might disagree about them and would have no way of resolving any such disagreement. The fundamental principle might be called numerical subjectivity, the idea that subjective judgments are often most useful if expressed as numbers. For reasons we do not fully understand, numerical subjectivity can produce considerable discomfort and resistance among those not used to it. We suspect this is because people are taught in school that numbers are precise, know from experience that judgments are rarely precise, and so hesitate to express judgments in a way that carries an aura of spurious precision. Judgments indeed are seldom precise--but the precision of numbers is illusory. Almost all numbers that describe the physical world, as well as those that describe judgments, are imprecise to some degree. When it is important to do so, one can describe the extent of that imprecision by using more numbers. Very offend, quite imprecise numbers can lead to firm and unequivocal conclusions. The advantage of numerical subjectivity is simply that expressing judgments in numerical form makes it easy to use arithmetic tools to aggregate them. The aggregation of various kinds of judgments is the essential step in every meaningful decision.

Emphasis added.

Back in the 1970s and early 1980s, the GAO encountered faulty use of numerical scoring by source selection personnel in a number of protest cases and complained that the method lent a false air of objectivity and precision to what is a matter of subjective judgment. But numbers didn't do that; poorly educated and trained human beings did that. The GAO has not said that numerical scoring cannot or should not be used. See IAP World Services, Inc.; Jones Lang LaSalle Americas, Inc., B-411659.2, 2015 CPD ¶ 302:

Quote

IAP argues that the agency used the artificial precision of a weighted numerical score to gloss over what would otherwise be a complex weighing of proposals' strengths and weaknesses. IAP Protest at 17–19; IAP Comments (July 27, 2015) at 31. The record shows that although the SSA's best value tradeoff discussion makes multiple references to the offerors' numerical scores, the SSA was well aware of the strengths and weaknesses underlying the evaluations. See AR, Tab 33, Source Selection Statement, at 20 (“I conducted my deliberations by first looking at the overall evaluated scores for Mission Suitability and the findings that led to those scores....”) While numerical scoring cannot substitute for a nuanced evaluation of proposals, here, the selection official had a basis to conclude that the scores were an accurate representation of the relative strengths and weaknesses of the proposal.

There we have a case of an SSA who knew what he or she was doing. However, some agencies (e.g., DOD) have prohibited the use of numerical scoring rather than properly train their people in the use of that extremely effective tool. Ironically, DOD's contractors use numbers when conducting systems engineering tradeoff analyses, and no one thinks anything of it.

Sigh.

There was a time when I used to try to explain all this to contracting people, but ignorance is so deeply rooted among us that I no longer waste my time. I have learned that many people in our business simply will not read books and will not study and prepare themselves for the tasks ahead unless their boss sends them to some government-sponsored class or they can hop on Wifcon for a quick and easy answer. They have allowed their ignorance to deny them the use of a powerful tool for the conduct of complex source selections. In explaining now I am making an exception, because I like your style and because I sense a keen intelligence behind the humor.

PepeTheFrog really appreciates this, Vern, and is thankful for your contributions to the intellectual and practical body of knowledge in contracting. Years ago, a Government contracts tadpole discovered your insights, analysis, and commentary and realized there was more to the profession than meets the eye. That young tadpole learned the Government contracts pond was quite deep, even intellectually stimulating, for those willing to take the plunge.

PepeTheFrog should have stated that the "veneer of objectivity" criticism applies only to those in contracting who misuse numerical scoring, not the concept or practice itself. PepeTheFrog was thinking cynically about what is rather than what should be. Unfortunately, there are not enough "people in the know" in federal contracting, which is a shame. Bring back the civil service exam, set "LPTA" to one standard deviation above the mean, and for cover, use the same attorneys who justify corporate screening exams, e.g. Procter & Gamble's "Reasoning Test."

PepeTheFrog suspects one reason DoD went to kindergarten colors was to emphasize ordinal instead of cardinal numbers, and to prevent the ability to aggregate the numbers, or manipulate them mathematically. Five colors, five ratings could just as well be five ordinal numbers: one, two, three, four, and five. Luckily, the Better Buying Power Mantras haven't inspired anyone to "do more (mathematics) with less (numbers)" by adding purple, blue, and blue, and comparing that "sum" to purple, purple, and blue...or have they?

PepeTheFrog will sit on a log and think for a long, long time about whether it is true that "subjective judgments are often most useful if expressed as numbers." There's a lot to contemplate. If anything useful and coherent bubbles up, it will be posted to the forum.

Pep, pep, Pepe! My imaginary contracting office just grew by one for a total of 8 (not including myself).

On 4/9/2016 at 4:15 AM, Jamaal Valentine said:

Pep, pep, Pepe! My imaginary contracting office just grew by one for a total of 8 (not including myself).

Thanks, Jamaal Valentine. PepeTheFrog is happy to hop on the Dream Team.

On 4/8/2016 at 4:26 PM, Jamaal Valentine said:

I was thinking the same, but based on some of Don's latest posts I'm unsure. Curious to know how they reconcile FAR 1.401(e),1.401(f), and 1.404 requirements.

I think a reasonable argument can be made that the DoD SSP has a significant effect beyond the internal operating procedures of the agency or has a significant cost or administrative impact on contractors or offerors.

I can only assume that someone in DOD consciously considered these sections before incorporating the Source Selection Procedures into DFARS PGI (see DFARS PGI 215.300.)  They must have concluded as Vern suggested, that the procedures are internal, and do not constitute a deviation.

Navy:

If so, do you agree with their conclusion that the procedures are internal, and do not constitute a deviation?

Jamaal,

The guide is addressed to internal agency personnel, and nothing in it is inconsistent with the FAR -- it is guidance on what internal agency personnel might do when working under FAR Subpart 15.3 -- it does not impose any burden on any contractor.

Does the DFARS PGI 215.300 (Source Selection Procedures) infringe on the wide latitude contracting officers are afforded by FAR 1.602-2? If yes, have the requirements in FAR 1.401, 1.301, and 1.404 been considered and/or complied with?

I've already offered an example, above, of how prospective offers can be affected by the procedures in the guide.

Just wondering how these things come to pass and if any of the above is applicable to publishing a PGI or similar internal policy.

Jamaal,

Are you insisting yet?  Or still just throwing it around?

By the logic of your questions, a contractor "can be affected" by an internal policy that established a lunch hour for agency employees between 11:30 and 12:30.  A contractor "can be affected" by an individual contracting officer decision to stop negotiations for today and resume tomorrow.  Would you want to apply the procedures for deviations and public comment to these?  I think you're pushing too hard.  The guide does not impose any burden on any contractor or do anything else described in FAR 1.301.  The standard for deviation or public comment is not whether a contractor "can be affected."

I don't like the guide -- for example, I don't like the mandating of 5 ratings (outstanding, good, satisfactory, marginal, and unacceptable) -- but I support the management prerogative of the agency head to issue guidance to agency employees.

Jamaal Valentine said:

Does the DFARS PGI 215.300 (Source Selection Procedures) infringe on the wide latitude contracting officers are afforded by FAR 1.602-2? If yes, have the requirements in FAR 1.401, 1.301, and 1.404 been considered and/or complied with?

I've already offered an example, above, of how prospective offers can be affected by the procedures in the guide.

Just wondering how these things come to pass and if any of the above is applicable to publishing a PGI or similar internal policy.

Infringement "on the wide latitude contracting officers are afforded by FAR 1.602-2" is not the standard for whether or not the requirements of FAR 1.401, 1.301, and 1.404 need to be considered and/or complied with.  For deviations we have a definition (which you reference at 1.401) and nowhere within the six standards for "deviation" is 1.602-2 mentioned.  1.602-2 is also absent from FAR 1.301, where the consideration is instead whether or not the SSPs are an agency acquisition regulation (1.301(a)(1)) that meets the standard for publication for public comment.

If you want to make an argument that DFARS PGI 215.300 meets one of those standards, let's discuss that instead of a standard that isn't present in any of the FAR references you state should have been considered and/or complied with.

Jamaal Valentine said:

Does the DFARS PGI 215.300 (Source Selection Procedures) infringe on the wide latitude contracting officers are afforded by FAR 1.602-2? If yes, have the requirements in FAR 1.401, 1.301, and 1.404 been considered and/or complied with?

Jamaal, do you really think that Contracting officers have such a "wide latitude" under FAR 1.602-2 that the organization that issued their warrants has no right to issue internal regulations governing how a contracting officer's authority is exercised?  Perhaps you skipped over 1.601 ( a )? A KO is far from being the final authority in an agency on how to contract for supplies and services.

So, if there is no applicability concerning a deviation to FAR 1.602-2 in the Source Selection Procedures, where do they otherwise deviate from the FAR or DFARS?

Matthew Fleharty said:

Infringement "on the wide latitude contracting officers are afforded by FAR 1.602-2" is not the standard for whether or not the requirements of FAR 1.401, 1.301, and 1.404 need to be considered and/or complied with.  For deviations we have a definition (which you reference at 1.401) and nowhere within the six standards for "deviation" is 1.602-2 mentioned.  1.602-2 is also absent from FAR 1.301, where the consideration is instead whether or not the SSPs are an agency acquisition regulation (1.301(a)(1)) that meets the standard for publication for public comment.

If you want to make an argument that DFARS PGI 215.300 meets one of those standards, let's discuss that instead of a standard that isn't present in any of the FAR references you state should have been considered and/or complied with.

There were two questions. You appear to have provided a response to one so I'll discuss that further if you outline - what are the standards in FAR 1.401(a),(e), and (f) referring to if not FAR, etc.?

Where else do the Source Selection Procedures deviate from the FAR or DFARS, since they don't deviate from 1.602-2?

joel hoffman said:

Jamaal, do you really think that Contracting officers have such a "wide latitude" under FAR 1.602-2 that the organization that issued their warrants has no right to issue internal regulations governing how a contracting officer's authority is exercised?  Perhaps you skipped over 1.601 ( a )? A KO is far from being the final authority in an agency on how to contract for supplies and services.

So, if there is no applicability concerning a deviation to FAR 1.602-2 in the Source Selection Procedures, where do they otherwise deviate from the FAR or DFARS?

Not at all what I think. For starters, I'm just kicking this around to see what other's think regardless of my true personal opinion. It's interesting to hear others opinions, interpretations, defense strategies, etc.

I think agencies issuing warrants have the authority to manage their agencies as they see fit - the question is what is the process.

You mention internal regulations that govern contracting officers, but FAR 1.3 provides the process for promulgating agency regulations that implement or supplement the FAR. Can this be done haphazardly on an ad-hoc basis?

The source selection procedures do not supplement the FAR in my opinion.

Jamaal,

The criteria for required publication, stated at 41 U.S.C. 1707, are whether the policy, regulation, procedure, or form:

1. has a significant effect beyond the internal operating procedures of the agency issuing the policy, regulation, procedure, or form; or
2. has a significant cost or administrative impact on contractors or offerors

What in the Source Selection Procedures do you think meets either (or both) of these criteria?

Jamaal Valentine said:

There were two questions. You appear to have provided a response to one so I'll discuss that further if you outline - what are the standards in FAR 1.401(a),(e), and (f) referring to if not FAR, etc.?

Asking another question does not make an argument.  I agree that the standards in FAR 1.401(a), (e), and (f) are referring to the FAR (after all they explicitly state as much), but that does not mean that FAR 1.602-2 is some sort of trump card that results in regulations or procedures (in this case the DoD SSPs) requiring deviation authorization or publication.  In regards to whether or not the DoD SSPs require(d) as much...:

• For (a) you need to provide an argument showing that the DoD SSPs are "inconsistent with the FAR."
• For (e) you need to provide an argument showing that the DoD SSPs authorize "lesser or greater limitations on the use of any solicitation provision, contract clause, policy, or procedure prescribed by the FAR."
• For (f) you need to provide an argument showing that issuance of the DoD SSPs are not "in accordance with 1.301(a)."  Here, Don Mansfield has provided the criteria in the above post for consideration.

Such arguments would warrant a meaningful discussion regarding the merits of how the DoD SSPs came to be (after all, whether or not the DoD SSPs even impede on the amount of latitude contracting officers have in exercising business judgment has yet to be shown, it has only been inferred by referencing 1.602-2).  Also, it may be worth nothing that 1.602-2 merely states "contracting officers should (emphasis added) be allowed wide latitude to exercise business judgment" versus shall.

Matthew: 

Neither does avoiding pointed questions or twisting words. I'm not saying you are wrong so don't take it personal. Just read my post, the FAR, and the SSP carefully and participate at your leisure. This is just a discussion. I'm not married to any thoughts I've posted on this subject.

I asked if the SSP infringed on the wide latitude contracting officers are afforded by FAR. A question, not an inference. If you had answered that question it would have changed the discussion.

For arguments sake, in your opinion, would the SSP be inconsistent with FAR (a deviation) if FAR permitted evaluating a technical factor without evaluating risk, but now contracting officers shall evaluate risk when evaluating technical factors as outlined in the SSP? If you believe that's a deviation what is the process that should be followed? I have not and am not saying the process was not followed.

Note:

Should means an expected course of action or policy that is to be followed unless inappropriate for a particular circumstance.

The SSP states that its policies supplement existing statutes and regulations.

Don:

Frankly, not at this time, but what's the fun in that? I perused the SSP and reserve the right to change my mind.  It does seem that the SSP governs the process between contracting agencies and prospective offers (FAR 1.301) and will have some affect on offerors. I do not claim to know the degree of impact. From my experience on the industry side - responding to solicitations is time consuming and expensive. The more requirements placed in a solicitation, the more time and money it takes.

I really just wanted to see what people think about the process of issuing agency regulations, and why they think it. I honestly would like to know more about the process and requirements for issuing a PGI or mandatory procedure.

Jamaal,

My mistake for assuming you were taking a position on the issue based on your line of questioning.  I'm certainly not taking anything personally or avoiding pointed questions (the question was quite open/broad) or twisting words (I quoted fairly heavily from the FAR and your posts); still I think this discussion would have been better served had you taken or stated a position one way or the other from the beginning because, as someone that does not think the DoD SSPs take away contracting officers' "wide latitude to exercise business judgment," I (or others) who felt similarly were in a position to try and prove a negative (which is a fallacy in reasoning).  Until someone makes a credible argument that they do, I don't believe there is much discussion to be had regarding that line of questioning.

Jamaal Valentine said:

For arguments sake, in your opinion, would the SSP be inconsistent with FAR (a deviation) if FAR permitted evaluating a technical factor without evaluating risk, but now contracting officers shall evaluate risk when evaluating technical factors as outlined in the SSP? If you believe that's a deviation what is the process that should be followed? I have not and am not saying the process was not followed.

Note that the SSP states that its policies supplement existing statutes and regulations.

Of the six criteria for "deviation," I think two are pertinent to your question: 1.401(a) (is it inconsistent with the FAR) and 1.401(e) (does it impose lesser or greater limitations on the use of any policy or procedure prescribed by the FAR).

For 1.401(a), inconsistency is the operative word and one definition is "not compatible with."  Your scenario lacks specificity, so it would depend on the hypothetical FAR language that permits evaluation of a technical factor without evaluating risk as to whether or not the DoD SSPs are inconsistent.  The likely case is that the FAR is silent and broad on what constitutes an appropriate evaluation factor (as it is today) and therefore, the additional specificity in the DoD SSPs wouldn't necessarily represent an inconsistency, but rather more specificity.

In which case, 1.401(e) becomes an interesting discussion.  Requiring the evaluation of risk when evaluating technical factors would likely be a requirement above merely evaluating the technical factor; however, it's impossible to answer this hypo without specific language because the FAR currently states under 15.304( c ) "The evaluation factors and significant subfactors that apply to an acquisition and their relative importance are within the broad discretion of agency acquisition officials, subject to the following requirements..."  In that case, I don't believe it imposes greater limitations because the FAR has already ceded that discretion to the agency officials.

Therefore, because the DoD does not meet either of the relevant two criteria for a "deviation" and because of 15.304( c ) I'd argue that it would not be a deviation needing to follow the process.  As for the process that should be followed, I know from reading your posts on Wifcon that you're smart enough to have already drilled down to the DFARS, AFFARS or whatever agency supplement applies to your organization and that you're likely asking as a matter of practice.  Unfortunately, I cannot help you there as I have not had to process a deviation before.  Happy hunting for someone who has!

Thanks for the thoughts and dialogue, Matthew.

Is the Value Adjusted Total Evaluated Price (VATEP) process essentially a weighted numerical rating?

VATEP is aimed at reducing subjectivity in the tradeoff process, but SSAs are still required to evaluate the benefits/advantages, if any, of trading one or more factors for more or less of another. Specifically, SSAs have to determine if the tradeoff is worth it ... isn't VATEP supposed to determine "worth" and "value" in advance so offerors can make informed proposal decisions?

If VATEP doesn't facilitate clear, objective, fixed "mechanical" evaluations and award decisions what real value is it to the offerors it's trying to provide proposal instructions and basis for award information to?

Just trying to think through benefits and drawbacks of VATEP in advance of being prompted to try it.

VATEP is an approach to quantify or monetize the tradeoffs in a selection decision.  FAR 15.308 clearly says it is not necessary to quantify the tradeoffs, and the GAO has consistently upheld this principle, but some people insist on doing it anyway.  VATEP is a structured way to do it.  VATEP is a way to remove subjectivity from the selection process and make it more objective -- instead of an SSA's subjective and independent judgment, a VATEP selection decision can be made by a clerk with a scoring worksheet.  Well, maybe I exaggerate a little.

VATEP is similar in some respects to the old dollars-per-point scheme that was used for a while in the 1970s and 1980s. 

VATEP could be suitable for acquisition of commercial item products with modular features and functions. For example, I am looking to buy a handheld GPS navigation device for use in geocaching. I have learned that they range in price from less than $100 to $1,000 or more, depending on features and functions. If I were conducting a government acquisition of such devices, I could establish a requirement based on certain essential ("minimum") features and functions, and then set a price on what I'd pay for each of several non-mandatory (optional) features and functions and then conduct objective tradeoffs on that basis. I think that might work just fine. It might also be suitable for some standardized commercial services.

VATEP is ill-suited for the acquisition of most noncommercial services.