In a December 21, 2009 editorial in the Suffolk News Herald, the city’s Director of Media and Communications gave the folks at the Suffolk News Herald a hand-slap. Ouch! I read the editorial and then decided to look at the Request for Proposals for the Obici House Renovation and Reuse. I wanted to answer the question: Was the actual RFP a fair tool?To answer my question, I studied the scoring tool included in the RFP to determine its effectiveness as a fair tool for evaluating proposals. Why is this important? The scoring tool is what is used to award the points for each proposal – it is the place where all of the work done by firms is finally scored for merit. The scoring tool is the most important part of the RFP. So here goes.
The scoring “matrix” for the Obici House Renovation and Reuse proposal was included with the RFP handout. You see, the purpose of an evaluation tool is to ensure a valid, reliable, and bias-free evaluation of proposals while awarding points to projects. A valid scoring tool simply means that the tool evaluates what you wanted to evaluate. I would expect to see actual statements from the proposal included in the tool. A reliable scoring tool means that if you put the proposals in front of two or more people for scoring, the scores will be similar. A bias-free tool means that the scoring will be fair – no matter what persons, organization, or entity turned in a proposal. Each firm has an equal chance from the scoring side of the evaluation.
The first thing I want to do is to address the top part of the evaluation matrix – which is not really a matrix. This is simply a scoring guide. I want to commend the city for including a scoring guide in the proposal packet. That is evidence of working toward a fair process in terms of scoring the proposals. But, the scoring guide isn’t yet a valid, reliable, or bias-free scoring tool. Let me show you why.
In order to get an Outstanding, a firm must achieve 8-10 points. There is nothing in this scoring tool that tells firm members what has to be achieved in order to receive those points. What does a firm have to have in the proposal in order to get an 8? What makes it a 9? What makes it a 10? How does someone who is scoring the proposals know to give a firm an 8, 9, or 10? What makes an 8 different from a 9?
In the 4-7 Points category, a firm meets expectations and is fully qualified and has adequate experience. Why wouldn’t that be an outstanding in terms of points? What would make a firm lose points here? What is different in a proposal that receives a 4 from one that receives a 7?
In the 1-3 points category, a firm will have minimal experience. What designates minimal experience? Also what does “less than expectations” mean? What does”less than desired level/insufficient documentation” mean? What gives a firm a score point of 1? A score point of 2? A score point of 3?
In the 0 Points category, what does “not documented” mean? The whole proposal is not documented? A part of the proposal or section is not documented?
These areas do not correspond on a one-to-one basis from the proposal – as they should. Each entity (e.g., city, firms submitting a proposal) should know EXACTLY each score point looks like in each area being evaluated. The scoring tool is not yet valid because there is no connection between the tool and the requirements in the RFP.
For example, look at number 1 below: Obici House Renovation, Restoration and Re-Use Plan and Property Management Program to include Architectural and Historical Preservation. What does a firm have to specifically do from the RFP requirements to get a score of 10? A 9? An 8? Since the information about how a firm achieves each score point is not there, the scoring system is flawed. A firm trying to use this tool to guide its work cannot tell what it specifically needs to do to achieve a level 8, 9, or 10 – or any other level for that matter. There is nothing that aligns the overall score system to the RFP requirements.
Another thing I’d like to address with this score system is that the way it is designed; everything counts the same. A firm can earn just as many points for having a complete proposal as it does for the actual proposal ideas. A scoring tool is also a communication tool – communicating to firms what is most important to pay attention to because it’s most important to the city. The scoring tool does not do this at all. Again, the scoring tool is not aligned to the requirements of the RFP. Even if the information is included elsewhere in the RFP, if the scoring tool is not adequate and aligned, it is a flawed tool.
A high-quality and effective scoring guide will include an actual matrix, which clearly shows how each point can be attained. A high-quality scoring tool will also align to whatever it is the city is specifically looking for in its proposals. Without this, the scoring tool is arbitrary and open to the scoring whims of the scorers. Additionally, the scoring tool will reflect the weight the city places on the items it is scoring.
Now to answer my own question – was the actual RFP itself fair to those who used it to develop their proposals? I’d have to say no for the following reasons: (1.) the RFP includes an inadequate scoring tool making it an unreliable tool that is open to bias, (2.) The scoring tool is not aligned with the requirements in the RFP making it not valid, and (3.) The scoring tool does not reflect the weight of items in the RFP further impacting its validity.
I can offer a simple and easy solution. In the next version of the RFP (or any other RFP), simply create a valid and reliable scoring tool that can fairly be used by the city and firms responding to RFPs.
14 comments:
Ms. Wahlstrom,
While I find your analysis of the Obici-House RFP Scoring Process of great interest and comprehensive in nature, you have missed the entire demand of the exercise by both the Mayor and City Manager. That is to put forth the appearance of a fair and meaningful RFP, while in fact doing nothing of such a nature, and to use such RFP only as a means to insure the requirements of the "Procurement Act" are meet and to use such as a shield of any real accountability or action for review by the public on the part of the City. As a secondary benefit to the City Staff, there is also the use of the "Procurement Act" to shield the submitted plan documents from the prying public-view under the "FOIA", since there is some obscure language that allows such under the ACT and as such, is commonly used by the City Staff to hide the results, their actions and the body of the submissions under the color of law. I have been in the city for many years and have worked with the City Staff on several projects and do not speak of such without some experience in this matter.
So one must always keep in mind that you must have a full and complete understanding of the expected results or outcomes desired, by such players as the Mayor, City Manager, and others. It was clear from the very beginning that there was no support at the Mayor/Manager levels to save this property, and in fact seems to be great interest and effort to tear it down for the benefit of another city lessee. Such deals are how it is done in the fine City we live in and does define the real actions and rational that underpins the "Scoring System" you have noted and defined. The system was only a shill, to refute any plan or any purpose submitted to reuse and rehabilitate the property. Why is this one might ask? It is because the answer to what was desired was already known, but there was a need to give political cover to those who had made such a highly charged political decisions. So to answer once again why the scoring system is found as you describe: because it provides the means and mechanism to gain the desired result of denying any plan that does not meet the primary goal of offering and supporting under the table an already anointed lessee. The real desire is the demolition of the site to insure that economic competition with a well connected lessee is not infringed, and to meet an already destined plan to allow an unpopular development that the Mayor and her supporters have defined many times in the past, that being the redevelop of the site with "Condos" and other uses.
Once one takes a look at this process clearly and under such real-view, an understanding that the outcome defines the process is the need and there is little to understand in that the process must be vague by design, so as to allow the process to be manipulated for the desired outcome. That outcome has nothing to d with: Fairness, Openness, or anything that might deal properly with the stated intent, since it is trumped by the preloaded demand already put forth!
So when defining a request for understanding of the processes of the City in such dealings, one must always look for predetermined outcome that defines the real needs found within the processes to get there.
With full Understanding - Thanks for your input and insight! My first step in analysis is to always look at key documentation involved in a process - thus I started with the RFP. If this is flawed, everything that follows will also be flawed - whether intentional or not. But you are right, the bigger question is whether or not additional parts of the process were flawed - and whether or not the outcome was really predetermined. What other predetermined outcomes have we had in our great city? Centerpoint? The water deal with Norfolk? What others?
Whoever you are, please keep weighing in with your insights about this topic and others.
Ms. Wahlstrom,
Little of consequence in the administration of Suffolk is not already “pre-determined” and front loaded before it is even put forth and for a known and desired outcome by those in power in our city. Since the merger of our City into what is now known as Suffolk today, there has been a long accepted political standard that when you get into power, you use that power for your own agenda, benefit, and purposes. One only has to take a look at how the Hilton Garden Inn deal was devised to validate this longstanding axiom and to understand how it all works in Suffolk to this very day. The deal started out as a true pipe dream to anchor some elitist's desire to rebuild the “Old Downtown” that was Suffolk, and to give credence to the talking points used to increase the resume of many who saw themselves as great “Urban-Planners”.
The deal started out as a scheme to attract, with enough public money, a nationally known anchor Hotel chain and then to buttress that with a large and flashy “Conference Center”, but the funding was always a problem as the city was run on a shoestring, so to say. Then some smart staffers came upon a plan to get the funding by floating bonds via the Suffolk IDA. The rational that it all was put forth under was, that the funding would be “Off the City Books” since the IDA was a separate political body from Suffolk, established by the Commonwealth some years ago, giving cover to the real spenders. So they now had a means to float the money to build with and started a campaign to define and attract a hoteliar of some stature that could be married to the process. They used almost nineteen million dollars of borrowing, three million dollars of loan supports, and untold amount of deferred tax money to put the deal together under the IDA, which fronted for the City. Then they added the icing to the cake, so to speak with a plan that at some time in the future, they would refinance the Hilton Garden Inn and Conference Center IDA Bonds, within some opportunity to get it under the General Obligation borrowing of the city, which is what they did. Our lovely Suffolk Treasurer was knee deep in this devious effort too, and should be always watched too! This plan accomplished two things that they desired very much: They got the borrowing under the normal City General Obligation lending eventually and they got the debt load off of the IDA (now the EDA) so that they could use the borrowing capacity they had developed under the EDA for another big ticket project, and all without any public hearings held for public comment. They also got the three million in loan reserves released for spending in one clean stroke too. Very clean and neat, related to their desired processes and a very good method to keep it out from under the possible discussion of those pesky public hearings and away from some of those vocal and well-educated commentators.
Ms. Wahlstrom,
PART II:
So do they have a plan for what they are doing at all times? Well the only answer to that is a resounding YES, and they have a process that works to keep it off the books, so to speak and out of the public discussion, too. The process was then going to be reused to build the Health and Human Services Building, but they hit a snag due to the heavy local complaint about where they intended to put it, since it was going to gain a huge benefit for a certain City Councilor, who also owns a nearby Funeral Home and a certain spouse of a Mayor who also owns property nearby the first chosen site on the western end of town, that would pushup the value of the real estate both owned nearby, said site. Call this one a “Plan within a Plan” and it almost got done, if not for the pesky public protest of the undesirable location by those low-income folk downtown that used the service, to now be located in such a building. So, it was decided that the building was to be relocated to an area that was only chosen to punish those who spoke out, and it was placed within the “Fairground Site” to use up land that the Council and Mayor really did not want to spend “their” (our) money on. They got to use up almost half of the site, without building any of the low-income homes that were promised for years and traded for political favors in the past with creatures like Charles Brown and Curtis Milteer, and by result created a three pronged political double cross.
So Little Lady, just keep looking and you will see much that supports the main theme in your analysis, that being that the outcome of anything of consequence is in fact predetermined and planed by the Mayor and her council, under the work of “their City Manager” who has little regard for the real people of Suffolk, unless it includes some benefit that can be pocketed for her, like another big off the books raise, via a “car allowance”. So the policy that you are only begining to gleen is: Suffolk suffers to allow those that can take, to take!
Seems like someone knows alot about Suffolk and how it all works. We all need to hear much more from this guy or gal! Then we need to hear how to get rid of the hacks that control our city so badly, too!
I think many of us know who this mysterious commentator is. It might be good if we also asked what were the circumstances that caused this person's alienation from a prior city involvement.
Incidentally, I wonder if it is a typo where the same commentator referred above to Ms. Wahlstrom as "Little Lady?"
In spite of this, it is a shame that "he" can't reconcile his differences with the possibility of his future reinstatement with and renewal of any past city involvement.
We need more people to step forward with such significamt insights into the inner workings of the city. The comments msde sbpvr about who this commenter is, has nothing to do with the validity of what he has to say. He has in a few pages of text, defined with great prose, what is in fact the inner workings of this entire question and situation!
I commend him or her and hope the information and explainations continue. The cronic abuse of position and power in this city is a pox on us all who live here in Suffolk and as such needs to be corrected. Just from what I have read so far from this man or woman of wisdom is of great value to us all. I do hope our new commenter has much more to say and hangs on and not vilified by those who crave darkness to the light of reason! We all need the facts on how the inside of Suffolk works and this is a good first step.
So write on "With Full Understanding" we are all ears, or rather, eyes and await your further insights and explainations of the workings of Suffolk. Perhaps we can even get this information to more readers so they too can make the connections that he has defined here for us.
GREAT JOB!
The only city involvement that he could be reinstated to would be "Grand Protector of Suffolk". This guy knows the facts and how to swat the mayor, manager and the idiots on council for how they run this city into the ground. Reconciliation really is about subversion, isn't it? ;)
Heavens no, the scoring was not fair of even honest. The young Assistant City Manager that was left to fix this thing for the Mayor was only doing what he was told, to kill this thing after giving it a show for the populace. The real thing here is that the Mayor, Par, Gardy do no want his done to protect thier chosen lessee on the golf course that wants to build a meeting building to rent out too! so that is the simplicity and answer to what hey wanted and will get! The scoring you want to review was just a front fot the answer that the young Assistant was told to get from the processes.
How true your comment is about the young staffer doing as he was told by the Mayor and Manager! Suffolk has a real problem shown by how this event worked out and how the mayor runs it all. Time to call for an accounting. How about you real complainers out there chiming in on this and sending a letter to the state attorney to see about this thing?
Put yourself into the shoes of that young staffer. Would you take the chance on lossing a $100,000 a year job, by bucking the boss and her bosses? There is no doubt about the decission process here, nor the planned outcome that was already demanded by the head mistress, our mayor! She will crack the whip if there is not blind obediance to the demands she places for a predetermined outcome. So fair scoring, you have got to be kidding me!
The revelation of "Obici - Homegate" occurs at a time when the accuracy of the intent of our political leaders is being seriously questioned by all. Over the last decade they have spent us all into a deep hole, yet every realistic model (there are many) predicted a significant decrease in the ability of the tax-payer to pay in that same time period. If the budget and spending models sold to us all cannot get it right for the past 10 years, why should we trust them at all?
"Housegate" reveals how predetermined suffolk political agendas shape outcomes rather than the other way around. It is high time to question the true agenda of the politicians and their staffers now on the hot seat and to bring skeptics back into the public debate on how to make local government better.
A million and half for the old main street courthouse but not a dime to help preserve the O-House? The courthouse is rumored to become the next clubhouse for Ms. White's Do-Nothing Tourism Office. Yet we stand idely by while the home of Suffolk's greatest benefactor is torn down or prostituted away to help a petty politician's agenda. It's about time concerned citizens protest on the lawn of the Mayor's home.
Cut costs by thousands, fire all in the Tourist Bureau. Think of all the money already wasted. The "profit" they claim for the city is no more than an uneducated wild guess. Talk about silliness. The city won't even release numbers on the Hilton, the Cultural Center, or Tourism.
Post a Comment