Felthamc.gif (4054 bytes)

Feltham Associates provide strategic, independent and experienced consultancy services for the procurement, selection and management of hospital clinical laboratory computer systems. The company has unrivalled specialist marketing and consultancy skills in all aspects of hospital laboratory service computerisation.

Laboratory Information System (LIS, LIMS) Selection © 2007
(updated December, 2001)
 This is an extended version of the three articles published in Biomedical Scientist between February and April 2001

WARNING:
This article summarises the work needed to procure or select a hospital laboratory information system. It is not a technical manual, it is not definitive, it does not try to cover everything, it does not give all the answers. It does, however, give common-sense information and advice based on our experience of many procurements up to November, 2009, but it should not be used as the sole or main source of procurement advice. Numerous links to NHS documents are provided (e.g. PRINCE2, POISE2, STEP, PFI, CIM, PROBE) and you should obtain all relevant statutory documentation and read it before embarking on a procurement, as well as seeking professional advice. Three articles based on this information were published in the Biomedical Scientist  Volume 45, Numbers 2, 3 and 4 in February, March and April 2001 respectively.  No liability for loss arising from following this information will be accepted.  The information provided here is covered by our LEGAL NOTICES: Copyright, Warranties & Disclaimers.

Contents (click on the hyperlinks to go directly to that section)

Introduction

Problems during the selection and implementation of a new laboratory computer system are openly discussed at great length in the literature, at professional meetings, during user group gatherings and in laboratory staff rooms!   It often appears to be a contest among users to see who can complain the loudest - about their system or their supplier!  While there are hints that the selection or procurement process may not have been handled well, few people appreciate how many problems result from a badly structured procurement.

This article takes a look at the process of procuring or selecting laboratory information systems with lifetime costs in excess of the EU threshold.   Smaller projects beneath the threshold do not require so much formal processes and advice should be sought from your local NHS Executive Regional Office. Project planning is discussed in detail since this activity is often handled superficially because of the desire to start looking at systems. In the worst case planning involves "doing what comes naturally". Unfortunately, what comes naturally for inexperienced organisations might not be the best course of action and is contrary to NHS guidance.

In the UK public IT system procurements are governed by European directives and UK legislation.  The NHS has promoted various techniques to help NHS Trusts and Health Authorities through the selection process and these are  described and discussed briefly below.  This article does not try to provide all the answers but it seeks to outline what is required and give pointers about best practice.  If further information or clarification is required then contact Feltham Associates Limited or your local NHS Executive Regional Office.

Procurement methodology (POISE2)

POISE2 (Procurement of Information Systems Effectively version 2) provides essential guidance and processes for the selection of NHS IM&T systems.   The POISE2 website has all the information including a downloadable PDF Manual, contact details to obtain advice or a printed manual etc. All the information given here is from the Manual with some explanatory material based on experience.

The POISE2 methodology covers four main stages: Plan, Prepare, Purchase and Perform, divided into 29 steps:

Stage Sub-stage Step Description of task
PLAN 1 Confirm the scope and status of the requirement
2 Conduct initial market assessment
3 Develop procurement strategy
4 Prepare the Project Initiation Document (PID)
5 Organise the project
PREPARE 6 Produce and approve the OBS
7 Prepare the evaluation plan
8 Produce and approve the OJEC notice
9 Adapt the STEP standards questionnaire
10 Produce the Information Memo (optional
PURCHASE Shortlist 11 Issue OJEC notice and screen responses
12 Prepare procurement models
13 Issue Information Memo and select potential suppliers (optional)
14 Issue OBS and standards questionnaire
15 Evaluate to a manageable short list
16 Prepare draft contract framework
Negotiate 17 Issue draft contract framework
18 Negotiate draft contracts
19 Evaluate
20 Develop final offer evaluation model
21 Agree draft contracts
Offer 22 Invite offers
23 Receive offers and clarify
24 Evaluate offers
Award 25 Award contract and conduct post-award administration
PERFORM 26 Accept the solution
27 Monitor and review
28 Conduct post-implementation evaluation
29 Plan for extension, re-competiton, termination

It is not our intention to reproduce the POISE2 documentation. Rather we offer pointers and experience from actual procurements.  You are recommended to obtain a copy of the POISE2 Manual and follow the guidance; it has both NHS Supplies and CSSA supplier support.  Most suppliers have experience of this methodology and appreciate well-planned and organised procurements rather than ones subject to constant delays, project team changes, unnecessary demands and ridiculous timescales (see Selection tasks below).

Project teams

Decision making has to extend from the top to bottom in a successful project structure. It involves departmental units, upper management, hospital executives and specialised committees established for the specific project; it can also involve specialist external consultants. A typical hospital approach to a laboratory system procurement should involve the following people

  • Trust Board of Directors - at the very least the Finance Director should be involved as the person who will eventually have to sign the cheque!
  • Administration - hospital and laboratory business managers to facilitate tasks
  • Project Board - consisting of Executive, Senior User, Technical representatives
  • The Medical Staff of the laboratory
  • Departmental Managers in the laboratory
  • Information Systems Department - useful for advice on standards, Trust strategy, other procurements in the hospital, networking and comms. issues
  • Project Team - consisting of project manager and discipline representatives who will carry out the procurement and report to the Project Board
  • External advice - specialist consultants and/or NHS Region/Supplies expertise (as necessary)
  • Other staff throughout all levels of the laboratory organisation as necessary

Describing the roles played by each of these elements is outside the scope of this article. Suffice it to say these roles should be agreed upon prior to the initiation of a project. The involvement of the Project Board is often poorly defined. This is evident by the fact that it is concerned with "tieing the pieces together" - at the highest level of planning in the first case, and during the actual decision making process in the second - and it is these connections that cause many users the greatest problem.

Selection tasks

Many approaches are taken to selecting laboratory information systems. They vary in complexity from elaborate computerised models with Gantt and Pert charts and unending meetings of all parties to the opposite extreme involving essentially no structured system at all. When asked why a particular approach was used, most participants have no idea. They certainly don't indicate that an approach was chosen because it was considered to be the best among a variety of methodologies considered.

Major surprises about what a system does or how it performs can usually be traced back to a rushed or otherwise inadequate procurement and evaluation.

Lack of attention to the process to be used prior to initiating work is not a trivial issue considering that many user complaints are related directly to how the system was chosen rather than the product itself. This problem is not easy to see at first since complaints are often levelled at "the system" or "the company." Consider some examples:

  • They took ages to start installation and now we're months behind schedule.
  • It's too slow.
  • When we installed the computer we thought we would be able to ... but I now we find we can't.
  • I don't like the new system because the reports are confusing and hard to read.
  • I like it but you can never get on the terminals; they're always busy.
  • We can't get the information we need out of the system.
  • Another user told us we would be able to ... but apparently we can't.
  • I don't like it because it takes a lot more time to ...
  • They never told us we would have to ...

Complaints of this nature should have been dealt with while the system was being selected. If there are insufficient terminals or response time is poor, the OBS and Draft Contract processes were probably faulty. Users who incorrectly thought a system would have certain capabilities did not get their information from the correct source - the proposal written by the company and the contract signed by both organisations.

With software that is essentially complete - i.e. little customisation will be performed - learning the capabilities and limitations of a system is relatively straightforward. However, it is immensely time consuming if done right but it is still possible to do. Major surprises about what a system does or how it performs after it is installed can usually be traced back to a rushed or otherwise inadequate procurement and evaluation.

The tasks described below (and in the POISE2 outline above) are not meant to be a perfect formula for success. They represent a tried and tested formula to do the job and they should be examined with the same degree of thoroughness as any other approach. The main reason these tasks are included is to show how a logical plan can be constructed that makes sense to users - under the right circumstances - and incorporates the all-important aspects of user participation, ownership and responsibility that are essential to select the right system for you.

Project planning (PRINCE2)

A good project plan is more than a list of tasks, completion dates and assigned responsibilities. Too often simple project schedules are drawn up by one or two people who would like the effort to be conducted in a certain fashion (usually quick and cheap) with little regard for the real problems they will encounter. This results in a plan that may be completely unrealistic and is therefore abandoned or significantly modified almost every week.

Key individuals in all affected departments must understand the plan and believe it is the best way to make the decision

On the other hand more sophisticated plans can suffer from too much complexity. A manager with a new "project planning program" e.g. Microsoft Project, can be a dangerous person! The human issues of selecting and installing a new system are often ignored as reams of paper are generated to explain how a computer will be selected. Ironically, a major goal of the new computer will be to reduce paper!  The NHS recommends IM&T procurement plans should follow PRINCE2 (PRojects In Controlled Environments version 2), the government's structured project planning methodology (details can be downloaded from NHSnet: http://nww.standards.nhsia.nhs.uk/prince2/index.htm). In practice most laboratory procurements use a simplified version which covers the salient points but reduces the bureaucracy and need for too many meetings. The CCTA PRINCE2 website can also be visited for an overview: http://www.ogc.gov.uk/prince/ or go to http://www.prince2.com.

There are two simple but critical criteria for a good plan: 1) key individuals in all affected departments must understand the approach outlined in the plan and believe it is the best way to make the decision. 2) functional managers must agree to provide the resources required of their unit to conduct the project in the time frame agreed.

To begin a system selection without such an understanding will almost guarantee the project cannot be executed as planned or the installation will be a disappointment to many users.

Why plan?

To develop a good plan one should start with example task lists obtained from a variety of sources. From these a set of tasks should be selected that make sense to the users, are in a logical order and appear to cover all the bases. No plan should be used just because "we've always done it this way" or "we read it in a book."

Doing something because an "expert" said it was right is a poor excuse for managing. Order of execution is just as important as the tasks themselves. Sending out an Output Based Specification to all respondents to the OJEC advert may result in 10 or more detailed proposals to review. Screening suppliers first based on major criteria and sending OBS's to a few finalists will drastically simplify the evaluation task and result in a better evaluation since effort can be concentrated on the most likely candidates to meet your specific business requirements.

After observing many selection projects, a number of questions come to mind.

  • Why does a well qualified supplier sometimes decline to bid - often to the dismay of the purchasing organisation?
  • What value are site visits when participants often cannot even remember which system they saw let alone many of its important features?
  • Why is it that procurements seem to go smoothly until proposals are received and it is time to make a decision?
  • Why do complaints arise about how a decision was made after the selection is finished?

After considering these and numerous other questions we outline below a list of tasks aimed at addressing such "process" questions; they tie in with the POISE2 steps already described above.

Since understanding what was available - not writing specifications for a system - is the primary concern, education forms the core of our advice. Part of the education can be formal through manuals, workbooks and structured techniques (e.g. PRINCE2, POISE2, STEP, PFI) while other elements need to be informal. These consist of supplier presentations, site visits, definition of business requirements and OBS questions all aimed at learning what systems can do and - just as importantly - cannot do!

The Project Manager

While POISE2 gives a well tried set of steps to follow, there is no specific detail for pathology computer systems.  Plans and timescales must be realistic if they are to have any chance of success.  Appointing a Project Manager is essential. In most circumstances this needs to be a full-time post for the procurement process - probably a year to 18 months.  Ideally such a person should be found from within the organisation - a senior manager, but their current workload may not permit them to provide the necessary focus and time.  External consultants in procurement are readily available, although specialists with sufficient knowledge of pathology are rare.   The resources required to appoint and manage an external specialist might amount to 3 or 4% of the budget for the system but if that means the right system has been selected to meet your business requirements, the project was on time, and to budget, then it will have been worthwhile.

One of the first tasks of a Project Manager is to collect together all of the guidance and documentation and to research other similar projects.  Usually there is information through the "grapevine" about other procurements or you can contact various suppliers to see with which other procurements they are involved.  Alternatively, if using a specialist consultant, they should have ready access to this information (Feltham Associates have maintained a database of openly tendered UK NHS procurements since 1990).

The Plan

An outline plan should be discussed at the outset of the project. This should cover overall timescales and include key events (i.e. POISE2 stages) such as release of OJEC advert, release of OBS, contract award date, go "live" date.  The period between contract award and go "live" date will be very dependent on the complexity of the project. A medium single DGH system should take no more than 9 months and could be ready in 6 months.  A large, multi-site, multi-hospital system could take 18 months and might be ready in 12 months.

Do not underestimate the time following the release of the OBS to the contract award date. During this period suppliers need to respond to the OBS, and you need to plan for demonstrations, site visits, draft contract negotiations and invitation to offer (ITO) stages. The EU directive stipulates 37 lapsed days after releasing the OJEC advert for suppliers to respond.

Once the market has been assessed and an outline plan agreed, a more detailed project plan called the Project Initiation Document or PID (PRINCE2 terminology) should be produced. This is the route-map for the procurement and the key dates must be agreed with the Project Team and Project Board.  It is well worth asking for specialist help and advice over this planning period as it is better to be realistic (even pessimistic) rather than be too optimistic and then have to account for project slippage with frustrated suppliers and users.

System research - market assessment

An initial market assessment should be conducted right at the start of a project to help focus the Project Team onto the definition of the key business requirements and functions. This information will help to eliminate vendors that are inappropriate for the organisation during the formal procedures e.g. vendors new to the market with limited experience, or the system does not meet the basic business requirements, vendors with a poor track record in support.

Published lists of LIMS suppliers, healthcare computing exhibitions and seminars, advertisements in healthcare computing or laboratory specialist magazines, websites such as Feltham Associates, as well as the experience of other hospitals can form the basis for the initial assessment.

This task should conclude with a list of vendors who are likely to bid for your business along with a brief description of each and any sales material available about the company and system.

Initial site visits

It is worth considering making two or three initial site visits prior to starting the project in earnest.  Following your market reasearch you will know about the leading suppliers and it is not hard to find contact details of their sites; the salesmen will probably be delighted to let you have this information although beware if they try to overtly "guide" you to a particular site - it is probably not stretching their system and so is not causing the company any aggravation!.  It is best to try and choose potential sites to visit which are comparable to your own.  Contrary to what many people think, the purpose of an initial site visit is not to learn what a system does. Only the supplier can tell you that - preferably in writing in a contract. 

While the average user you meet during a site visit has some information about the capabilities of the system, he/she may be completely wrong in important aspects. They know how they use the system. They may not know about capabilities it has which their organisation chose not to install. They do not know about new functions the supplier is now marketing which don't happen to be in the version they are using. They may not know that a particular feature of the system which they like very much was custom developed for their organisation and is not available for other sites.

The main purpose of an initial site visit is to determine several users' reactions to the systems and to see how they operate in a roughly comparable environment. This will provide you with information which you may need to consider putting into your list of requirements. Secondly, the visit is to find out how a supplier performs in practice not through a salesman's "rose coloured spectacles"!  A third purpose is to discuss issues of installation and operation which will be valuable later on. In these areas, users are the experts not the suppliers. Many of these issues will be common to all suppliers so it should not be considered a waste of time to visit an installation of a supplier that is subsequently eliminated.

Do not make your selection - even though unofficial - at this stage!  You have a long road to journey down and this is only the beginning. Many events can happen to cause you to change your mind e.g. supplier deciding not to bid, an unknown supplier having a better matched system, not sufficient budget to purchase your "favourite".  Naturally you will have seen features you like, probably different ones in different systems. Make a note of these and in your Project Team discussions decide how important they are and what weighting to give them in your evaluation criteria.

Preliminary justification (OBC)

The question of when and how to justify a computer system plagues every organisation. One cause of this difficulty is that justification is seen as a "one shot" process with a straight forward yes or no answer. Many people simply want to know whether a computer system can be justified and they would like to know the answer before they waste money evaluating systems.

The problem is that much of the information needed to do a thorough justification is not available until a significant portion of the evaluation is complete. This can be seen when the "justification question" is rephrased in a more meaningful fashion.

"Is it in the best interest of the hospital to invest this amount of money on this product at this time?"

It is obvious from such a statement that a detailed justification cannot be accomplished before proposals are received. That's when system costs will be known. Justification at the outset of a project should be stated differently:-

"Based on our limited knowledge of the subject matter does the likelihood we can eventually justify a computer appear high enough to warrant researching the subject matter in depth?"

While it would be nice to completely justify a purchase before doing any work, this is an unrealistic goal. The best an organisation can do at the beginning is to justify spending resources on an investigation. This might appear to be a half-hearted approach since there is no guarantee a system will be purchased even after all the research is done. It is, however, no more half-hearted than discovering at the conclusion of the evaluation that the original "justification" was faulty and no system is installed as a consequence.

A "preliminary justification" has the benefit of being honest. The official NHS name for this process is the Outline Business Case (OBC). A Trust Executive will have a great deal of respect for a laboratory with the following statement. "We won't know whether or not we will be able to afford a system until we learn more about the capabilities and costs of the available products. We do, however, believe the prospects are good enough that we should investigate the possibilities in more detail."

Private Finance Initiative (PFI)

The Private Finance Initiative (PFI) is intended to encourage the development of closer relationships between the public and private sectors. Under PFI the private sector funds the capital cost of the system and its subsequent operation and maintenance, raising finance from whatever sources are judged appropriate.

The objective of PFI procurement is to provide high quality public services that represent value for money for the taxpayer. PFI is one model of Public Private Partnerships in the NHS; other forms may develop over time. PFI is a key policy for improving the quality and cost-effectiveness of public services. It enlists the skills and expertise of the private sector in providing public services and facilities. It is not simply about the financing of capital investments, but about exploiting the full range of private sector management, commercial and creative skills.

Any PFI scheme must demonstrate value for money (VFM) for expenditure by the public sector. This can be achieved if the private sector assumes risks which would otherwise have been borne by the public sector where they are more cost effectively managed by the private sector, and by efficiency savings.

The NHS PFI process (which runs alongside POISE2) is as follows:

  • establish the strategic context, assess the options and, for major schemes, make the case for change in a Strategic Outline Case and get approval;
  • identify and develop a preferred option through an investment appraisal, make the case in an Outline Business Case, and get approval;
  • prepare for procurement by turning the approved option into a detailed specification of outputs, outcomes and desired allocation of risks;
  • advertise the project in the Official Journal of the European Communities (OJEC), identify potential providers and the best privately financed solution;
  • select a preferred bidder with whom negotiations can be completed,
  • involving stakeholders (eg staff and trade unions) in the assessment of proposals;
  • complete the definitive investment appraisal and Full Business Case to obtain approval;
  • finalise, award and implement the contract; and
  • evaluate and monitor the project.

Procuring bodies in the NHS should seek the appropriate professional advice before undertaking any procurement.

Outline Business Case (OBC)

An Outline Business Case (OBC) is required to lay down the reasons behind the request for the investment. Later in the procurement process a Full Business Case (FBC) will be required to finalise the selection procedure. The objectives of the OBC are to:

  • show the Trust has thought about why it needs the investment;
  • generate a full list of options for meeting the objectives of the investment and a short list sifted on the basis of agreed criteria;
  • assess the costs and benefits of each short-listed option;
  • identify the objectives of the investment and their link to the Trust's strategy and overall business objectives which should be linked to the overall purchasing strategy;
  • identify the benefits expected as a result of the planned investment;
  • identify any constraints on the means of achieving the objectives of the investment;
  • prepare a risk analysis, and an assessment of the impact of the investment on the Trust's prices;
  • identify and develop the preferred option for the proposed scheme;
  • show that this has been achieved through a rigorous investment appraisal process;
  • show that it is in response to a robust case for change and is in line with national and local strategies and priorities; and
  • demonstrate that key stakeholders have been involved in formulating, and are committed to, the proposals.

It is important for the OBC to be developed in close collaboration with commissioning HAs or PCGs, key clinical and other staff affected by the proposals, and the Regional Office. It should adhere closely to the guidance in the Capital Investment Manual (download from http://www.doh.gov.uk/nhsexipu/resource/itinvest/l10.htm), particularly stages 1 and 2 of the Business Case Guide, and PFI guidance (download from ).

The costs, benefits and risks associated with this option should be rigorously assessed over the full life of the project using:

  • an economic appraisal (or value for money analysis);
  • an assessment of the non-financial benefits and factors (eg using a scoring and weighting analysis); and
  • a cost benefit analysis that brings together the economic and non-financial factors.

The preferred option will normally be the one which meets the project objectives and delivers the greatest ratio of benefits to costs.

Team selection

Most project teams are too large. They also suffer from a lack of structure. A core project team of 4-8 people should be adequate to conduct most evaluations. As mentioned earlier, it is important to separate technical evaluation from management guidance and approval. If this is done properly, the project team's responsibility will be to evaluate the systems in depth and make recommendations to management at each major decision point (project checkpoint).

Consequently, most of the work can be handled by the technical experts and manager(s) of the departments with proper coordination with others throughout the organisation. A few people can be designated as "ad hoc" members and brought in to the process when their specific input or expertise is required. Contributions from other staff members will be solicited by the team at numerous points in the process to aid "ownership".  The Project Manager should discuss the possible team membership with senior managers, and confirm the agreed membership with the Project Board.   It is important that the Project Team are representative of the departments affected by the new system. They should ideally be expert users in the existing system (if as is usual, a replacement system is being procured) and have an interest in the future of the organisation.  It is not necessary for senior management to think they have a right to be on the Project Team. Their input will be esential during the procurement but they are usually not the most appropriate people to be members of the Project Team.

In large projects where more than one hospital organisation is involved, e.g. multi-site procurements, there may need to be a Project Steering Group which acts as a funnel for individual "coal face" teams in each hospital.

Once a team is selected, it is vital that all members understand the importance of their contribution. If a team is too large, it is easy for one person to miss a meeting. A small dedicated team where each member has a well-understood assignment is a much more effective approach and unavailability from meetings, site visits or demonstrations could affect the overall timescales and a proper objective evaluation.   A real commitment from team members will be essential throughout the procurement stage.  Team members will need to negotiate with their line managers to agree about making the time available for the project.

Sales presentations

Supplier visits must be structured by the user. Otherwise, each one discusses different aspects of a system and no comparison can be made. Certainly each representative should be encouraged to point out unique features of his or her product, but the overall format should be similar. While it is common practice to have products demonstrated - through terminal connection or PC software - their are drawbacks to demonstrations at this early stage.

Reading standard material provided by a vendor before the representative comes in for a presentation will make each meeting much more productive. Some understanding of terminology and distinguishing features of a system prior to meeting with the vendor will add significantly to the value of the presentation.

First and foremost, it is best to review systems in layers - from the top down. A major part of an introductory presentation is to learn about the company, their clients and their approach to the business. If too much time is spent on details of terminal operation, data entry, report generation, etc., the highest layer will be omitted. Details can be examined later when the team is better prepared and fewer products are involved.

There is always a trade-off between the number of companies under consideration and the depth to which each product can be investigated. The number of companies involved at the beginning precludes a detailed investigation of each one. As the field is narrowed so decisions must be made on an increasingly detailed level, terminal demonstrations of selected systems can provide some of that detail.

The second problem with early demonstrations is that few details will be remembered for the duration of a project. Information that is important about exactly how a system operates will be lost by the time an in-depth comparison is conducted.  In contrast, if a supplier has a exceptionally, special feature which captures the imagination of the bulk of the users (e.g. telemedicine, voice recognition, WAP links, 3-D graphical workload statistics etc.) then this can detract from the objectivity of future demonstrations.

Preliminary proposals

Even though a final configuration cannot be determined until later in the project, it is possible to obtain indicative pricing information on a tentative configuration. Most companies can produce a "budgetary proposal" with a small effort and skeletal information about transaction volumes. It is very important that the user suggest the number of terminals, analysers, printers, workloads etc. so that all companies will propose roughly comparable systems.

While some - particularly first time users - will find it difficult to guess at a configuration, it is better than letting the supplier guess. The supplier is generally biased towards underconfiguring at this stage. They do not want to look high priced so if they believe the user will need 50-60 devices, they will propose 50 - or less sometimes.   Sometimes suppliers will ask permission for one of their technical specialists to do a "walk through" round the laboratory. It can be useful as their experience can sometimes give you information which you might otherwise have overlooked.

The results of this budgetary proposal should be used to determine whether or not the preliminary justification was on track. If prices, are much higher than suggested earlier, it would be legitimate to ask whether the original analysis was still valid.

In guessing at the initial configuration, it is better to err on the high side rather than to underconfigure. No-one has ever been reprimanded for saying "we have decided to spend less money by removing some of the terminals from the original configuration". On the other hand, how many people have ever said they made a mistake and bought too many terminals?

Concerning software, no decisions should be made at this stage about whether or not a particular optional package will be used. All software modules including interfaces should be priced and included in a preliminary proposal.

Evaluation criteria

A major flaw in many system selection projects is that no formal evaluation system is developed until very late in the game. In particular, many organisations do not develop the criteria and methodology for evaluation until after proposals are received. It is at this point they realise how difficult it is to make a decision where many people with different needs are involved.

Pricing should be excluded from the evaluation until a ranking based on system capability and corporate strength has been completed.

It is unlikely that one system will  be judged best by all participants so a method of compromise must be developed. This method usually involves an assignment of weights and scores to various system features which are then totalled to come up with the final vendor ranking.

The problem with doing this after proposals are received is that the OBS probably did not request the appropriate information in the best format for evaluation. How could it, when the evaluation system was not prepared at the time the OBS was written?

While it is not possible to describe a complete evaluation system in this document, a few critical points will be mentioned.

  • A hierarchical approach works best when assigning weights. This allows weights to be changed in various components without changing them throughout the entire system.
  • Not all participants should evaluate all elements. Individuals on the Project Team were selected because of their expertise in specific areas. They and they alone should judge the system in the areas were they have the most experience.
  • Pricing should be excluded from the evaluation until a ranking based on system capability and corporate strength has been completed. After this ranking is done and comparable proposals have been received, it is possible to make a decision concerning the difference in capability vs. the difference in price among all the vendors.
  • Mandatory requirements should be kept to a minimum for two reasons. First, a large number of mandatory requirements will more than likely eliminate every vendor since they only have to miss one to be disqualified - if they are indeed mandatory requirements. Also, mandatory requirements are often dropped when the price to meet them is determined. So, in fact, they weren't mandatory at all. They are like all other "requirements" - desirable if they are affordable!

In practice a summary, in the form of a spreadsheet, can be very useful with every requirement numbered sequentially in the OBS and the spreadsheet.  Weightings can be applied and then either suppliers can be asked to complete the spreadsheet or the Project Team mark it after reading the OBS. The advantage of asking the supplier to complete the spreadhseet OBS summary is that it is their decision whether they comply or not with each requirement and not some subjective assessment by a Project Team member.   A suitable scoring system could be 0 = non-compliance, 1 = partial compliance, and 2 = full compliance.  By amalgamating all the supplier responses into a single spreadsheet an initial ranking can be achieved extremely easily.  Clearly if compromises are required for some of the partially compliant features to separate closely ranked suppliers then the detail about the response will be found in the OBS.

Advertising your project (OJEC)

The procurement of laboratory information systems is subject to European public procurement rules which apply to all NHS information management and technology (IM&T) procurements.

The rules consist of relevant legislation which must be adhered to when carrying out the POISE2 process. Since the rules apply to IM&T purchases, the project team members should work within the procedures and principles that are established. Unfortunately the rules were designed for ordinary purchases of standard commodities and not the complexities of information systems specifically.

The rules set out procedures for the award of contracts above certain threshold values throughout the European Union (EU). Their purpose is to open up the public procurement market and to ensure the free movement of goods and services within the European Union thereby increasing opportunities for competitive suppliers, contractors and service-providers. In most cases they require competition. They go with the grain of the Government’s policy that procurement decisions should be based on value for money through competition.

Generally, the ‘corner-stone’ rules of the legislation are contained within the Treaty of Rome. Specifically:-

  • Article 30 prohibits restrictions on the free movement of goods within the community
  • Article 59 prohibits restriction on the freedom to provide services within the community
  • Article 85 prohibits agreements which have as their objective prevention, restriction or distortion of competition. This is particularly important when considering contract length, since a contract considered to be too long may be open to challenge under this article. Broadly if the contract length is set with a view to obtaining best value for money and the contract is subject to periodic re-competition, there should be no difficulty.

The EU’s collective aim is to assist in the creation of a single market across Europe, with buyers making their requirements known widely across state boundaries and choosing suppliers in an open, easily understood way. Since the UK is a member state of the EU, the authority’s buying decisions must serve not only local interests but also those of the state.

Breaches of the directives and/or the Treaty of Rome may be investigated by the European Commission. If the Commission believes that a breach of the rules has occurred, it may seek suspension of a contract. If a contract has been awarded, the Commission does not have the power to overturn it, but the European Court of Justice does, and may exercise that power if the breach is considered serious enough.

The rules apply to all written contracts for goods for health authorities, trusts, or agents working on their behalf, if the total value is more than the current threshold.

Contracts for lease or rental are treated as if they were purchases, regardless of which party finally owns the equipment. For a fixed term rental or lease, the value is the total of the payments. For an indefinite rental or lease, the value is 48 times the monthly payment.

In estimating the value, the authority should include all services which are associated with the procurement, such as costs for installation, licensing operating system software, conversion of an existing application etc. VAT should be disregarded . Under the Services and Supply Directives it is forbidden to split up the total projected cost of procurement in order to avoid the regulations.

Throughout this article, reference is made to the Official Journal of the European Communities (OJEC) which is published by the Office for Official Publications of the European Communities.   All procurements conducted under the European rules have to be advertised via a notice in OJEC. There is no charge to an authority for placing an advertisement. The maximum length of any advertisement is 650 words which is about one page.  It is best to research other adverts for similar projects before finalising your own.  Some are so well worded that they remove the need for an extra Information Memorandum stage.   Feltham Associates has a collection of OJEC advertisements for laboratory information systems.

From the date of dispatch of the notice to the Official Publications Office, potential suppliers must be allowed a minimum of 37 days to submit expressions of interest.

The replies that come in should be accompanied by whatever financial and technical information was asked for in the advertisement. If some replies do not include the financial and technical information that was requested, it would be sensible to write back to the firms in question explaining that the authority is entitled to request this information and repeating the list from the notice in OJEC.

Authorities are obliged to debrief suppliers in writing on request to say why they have not been invited to tender or have not been successful with their bid – this must be done within 15 days of the date on which the request is received.

In POISE2 terms, that means that once a supplier has responded to the OJEC notice and asked to receive the OBS, if the authority decides that he is not suitable, and he asks why, it must explain, even though it will have not yet awarded a contract.

Some frequently asked questions (FAQs) on the EC procurement process can be checked here.

Thresholds

Procurements over the following thresholds need to be advertised in the OJEC via an OJEC notice:–

Directive Value Type
Services

£100,410 (€162,293)

Whole Life Cost
Supply

£100,410 (€162,293)

Total Contract Value

These thresholds are updated every two years in January. The above apply from January 2002 to December 2003.  The latest threshold levels can be found through the Office of Government Commerce (OGC).website.

Information Memorandum (IM)

The IM is an optional document which can be issued to potential suppliers to give them sufficient information to enable them to more clearly establish whether they wish to propose solutions, and to provide additional information to the authority.

The IM will always be developed from the OBS but the content of the IM will vary according to its use. The IM must always comply with the basic requirements for PFI testing and therefore it should not be a ‘tick list of requirements’ that have to be met. The IM must give sufficient information on the requirement and the reasons for the requirement to enable suppliers the maximum opportunity to propose innovative solutions.

A suggested contents for an IM are given with the POISE2 documentation.  Not every procurement will need an IM. It can sometimes help reduce an initial set of interested suppliers who responded to the OJEC advert.

As with all information passed to suppliers, it is helpful if the IM is sent both as hardcopy printout and electronically (e-mail and/or 3.5" floppy or CD-ROM).

Output Based Specification (OBS)

An Output Based Specification (OBS) is a detailed understanding of what the proposed system needs to do to meet the Trust's business requirements and what the obligations of the vendors will be. This material must be in writing because it will eventually form one of the schedules in the contract with the chosen vendor.

The OBS is a document used during an IM&T procurement that describes the authority’s output requirements for planned investments in new systems and/or services (such as operational running of the system, a help desk and technical support).

In concept, it replaces the Detailed Statement of Need (DSoN) and Summary of Need (SoN) defined in the original version of POISE. Just as the DSoN and SoN did, the OBS attempts to describe the authority’s needs without getting into the detail of how these might be satisfied.

The main differences are described in more detail below.-

  • Service boundaries
  • Linking requirements with existing systems and expected benefits
  • Giving suppliers the opportunity to innovate
  • Categorising requirements
  • Ability to evolve requirements
  • Level of detail
  • More emphasis on services requirements

The OBS describes the output requirements for planned investments in new systems and/or services plus any constraints that apply to the proposed solution(s), such as the need to meet national or local standards and the need to interface with existing systems.

It also includes additional information pertinent to the procurement, such as the authority’s proposals for risk transfer and for management of the contract, details of the procurement process and instructions to bidding suppliers.

It has the following uses:-

  • to ensure that both the authority and potential suppliers have a clear understanding of the authority’s requirements and expected benefits from the proposed investment
  • to provide the basis for the draft contract schedules regarding output requirements that will be inserted into the draft contracts with shortlisted suppliers
  • to provide a baseline against which shortlisted suppliers can be evaluated (including being rejected) in a clear and even handed manner.

An OBS should not be a System Specification. Specifically, it should not tell the vendors exactly what their proposed system should do. This approach worked fine when any new system involved a significant amount of reprogramming. Customisation was then the rule, not the exception.

The high cost of customisation and support of custom software has changed this situation completely. Most users try to use standard packages to the greatest extent possible. Suppliers are certainly aware of customisation problems as well. The only ones who boast of their willingness to customise are the ones who have partial systems and are looking for someone to help them complete the development effort. With packaged systems it makes more sense to ask the supplier what the system does and how it works rather than telling them what you would like it to do.  This way you are seeking to match your business requirements with the systems available.  You then have to evaluate which system is the best fit bearing in mind various features which are essential and others which have been deemed desirable.  It is very unlikely that any system will meet your requirements exactly so compromise will be important.

There are a number of common problems facing authorities when defining their needs:-

  • Inappropriate focus on solutions not needs
  • Too much/too little detail
  • Unworkable compromises
  • Injudicious use of the term ‘mandatory’
  • Copying others’ documents - they should be YOUR needs not some other organisation
  • Overlooking future changes to requirements

Asking questions has one other advantage. It allows the supplier to offer suggestions and demonstrate methods of operation that you may not have thought of during earlier stages of the process. The evaluating team has only to compare answers to the OBS and judge which ones they like best. They do not have to try to "invent" a best answer at the OBS stage and they are open to new ideas right up to the time of final ranking.

This approach is in distinct contrast to closeting a team early in the process so they can document their needs and describe the "optimum system" without the supposedly confusing influence of vendor claims and counter claims. Systems are too complex today to believe that a typical user can describe how a system should operate in detail without an intense interaction with a variety of vendors and users to learn about all the possibilities.

The more exposure key individuals have before a system is purchased, the less confusion there will be during the installation.

Vendors should not be asked to rush their responses unless there is good cause. One to two months for a detailed response for a complex system is not unreasonable. A busy supplier - the type you would like to work with - has many prospects in the sales cycle. They all have limited resources and the hospital that realises this and works with the vendors to make the most of their time will come out best in the long run.

As a rough guide, the OBS should include the following main sections:-

  • Introduction
  • Background to the requirement
  • Summary of the requirement
  • Core requirements
  • Possible additions and variations to core requirements
  • Management of the contract
  • Constraints
  • Risk transfer
  • Procurement process
    • Procurement timetable.
    • Implementation timetable.
    • Procurement procedures.
    • Information required from suppliers during procurement re costs.
    • Evaluation criteria and method.
  • Details of response required
    • Instructions for layout of proposals (to aid comparison and evaluation).
    • Corporate capability (eg nature of consortium).
    • Point by point response to everything set out in Sections 4 to 8.
    • Details of options and alternative proposals.
    • Description of approach to service provision (including project management, plans, timetable, organisation and staffing).
    • Indicative/budgetary commercial proposals (including funding arrangements, basis of charging, budgetary costs, incentives in contract, transfer of assets/staff, other revenue streams).

STEP Questionnaire

STEP stands for STandards Enforcement in Procurement and is the process which ensures that NHS IM&T systems, equipment and services, conform to NHS policy on standards. These policies are designed to encourage the interworking between systems and the exchange of information between appropriate NHS organisations. The latest questionnaire can be obtained from the STEP website: http://www.standards.nhsia.nhs.uk/spg/step/index.htm

Since it is not possible for products to conform immediately to new standards, three categories have been defined and these are updated annually.

  • Category 1 standards are those standards with which the NHS can reasonably expect products to conform.
  • Category 2 standards are standards agreed nationally by the NHS but ones that suppliers have not necessarily had the time to implement in their products.
  • Category 3 standards are those which are not yet formally agreed NHS standards but which may become agreed standards at some later stage.

Each annual update adds new standards and may move existing standards into different categories. The current version (from April 2000) is version 6.

Not all standards will apply to all procurements and some that are relevant may be more important for local considerations than others. The flexibility exists for users to consider which standards are relevant, what priority they have in any given procurement and to evaluate suppliers’ responses accordingly.

Each procurement should decide which standards are relevant to that procurement and indicate them on the STEP Questionnaire. The Questionnaire is issued with the OBS and suppliers’ responses are returned with their proposals. The responses requested to the STEP Questionnaire usually require a yes/no answer and so it is relatively easy to evaluate them. However it should be noted that the response will form part of any resulting contract. If any standards were made mandatory in the OBS then only those suppliers that can, or will in an appropriate timescale, comply with the mandatory standards should be taken through to the next stage of the procurement. The STEP Questionnaire can be sent in an electronic form.

On-Site Demonstrations

It is now time to bring in terminals or small systems for a live demonstration. The field has been narrowed so adequate time can be devoted to each system. With fewer vendors, the confusion about which system does what will be minimized. With a good background in the general capabilities of each system, team members are in a much better position to probe more deeply into how each system works than they would have been during initial presentations. Finally, It is close to the time for a detailed evaluation so the knowledge gained through very recent on-site demonstrations is more likely to be of benefit in this process.

As with all aspects of the selection, demonstrations should be structured so each vendor addresses roughly the same topics. Because each representative will be on-site for a day or more, it is possible to expose a significant number of people - not just the project team - to the intricacies of each product. If certain aspects of a system are to be demonstrated at a particular time, individuals who have knowledge in these areas should be brought in to observe.

The more exposure key individuals get before a system is installed, the less confusion there will be during the installation.

System Evaluation

Evaluation is an integral part of the competitive procurement process. It is the task of comparing and ranking potential suppliers and their proposals against predetermined criteria to assist in the selection of the solution which offers the most economically advantageous offer. A key component of an evaluation of privately financed options is risk measurement.

More evaluations grind to a halt during this task than at any other point in the process. The reason is simple. As mentioned earlier most organisations do not give adequate attention to the evaluation criteria and methodology before attempting to perform the evaluation. As a result, they realise at this point how difficult it will be to make a decision that is best for the laboratory organisation but that may not be considered best by one or more individuals or units. The simple decision making approach that works for small systems that affect few people cannot be "scaled up" to handle complex procurements.

At this critical juncture, the unprepared organisation is forced to confront the issue of "exactly how are we going to decide" for the first time. The lucky ones struggle with this problem and finally determine how the evaluation will be done and then realise they did not collect the correct information during previous tasks. So they go back with a follow-up Information Memorandum with a second round of questions. The unlucky ones are not able to devise a workable methodology and the process slows to a crawl or stops altogether.

If, however, the organisation worked on the evaluation process ahead of time it is only a matter of working through the steps that were agreed to before the OBS was sent out. Very little additional material should be needed precisely because the OBS questions were developed to collect the information the organisation felt would be useful in comparing systems.  A completed OBS summary spreadsheet is very worthwhile.

Without this core technical content all the legal language in the world will not form the basis for a satisfactory installation.

Quality of Solution

A list of possible low level criteria for evaluation is selected on the basis of need and fitness, some typical examples are given below:-

  • single entry of data
  • timeliness of information
  • quality of information
  • ease of access to information
  • better analysis of information
  • ability to improve efficiency/cost effectiveness of service
  • fit of application software with functional requirements
  • system availability
  • better utilisation of staff
  • better response times
  • more comprehensive reports.

Commercial criteria

Commercial criteria are concerned with the standing and general capability of the supplier. The assessment of this category of criteria are largely completed by the evaluation of OJEC responses (POISE2 Step 11) although further assessment is undertaken at the evaluation of the information memorandum (POISE2 Step 13) and evaluation of the responses to the OBS (POISE2 Step 15).

Suggested topics to include under this heading are:-

  • history, size, stability, financial viability and growth of the supplier organisation
  • standing of the supplier in the market place
  • details of proposed parties or consortia, and previous experience of working together
  • track record in the provision of the required services
  • current and planned areas of operation relevant to the requirement (such as types of client business, types of application and business activity, types of service delivery, scale of projects undertaken)
  • supplier’s approach to funding the PFI service provision, and access to financial resources
  • ability of the supplier to generate alternative revenue streams
  • ability of the supplier to benefit from economies of scale in the provision of the required services
  • reputation of the supplier for innovation in technology and in business re-engineering
  • supplier’s access to relevant technical resources and expertise
  • range of additional services which the supplier could deliver
  • flexibility of the supplier in dealing with customers
  • willingness to accept customer’s contractual terms and conditions

Cultural fit

This area of evaluation is concerned with assessing the likelihood of establishing a constructive and mutually beneficial working relationship. This will depend largely on observations made during the contract negotiation stage; site visits will be particularly relevant.

Suggested topics to investigate could include:-

  • the quality of the working relationship with the supplier
  • the supplier’s working practices
  • the supplier’s approach to project management
  • the calibre of the supplier’s staff
  • the ease of communication with the supplier’s staff, and their responsiveness and openness
  • the supplier’s approach to the negotiations and the degree of flexibility shown
  • the supplier’s willingness to reach agreement
  • the supplier’s willingness to accept and manage risk
  • the entrepreneurial nature of the supplier organisation
  • the relationships between the supplier management and staff
  • the supplier’s familiarity with the public sector culture
  • the supplier’s reputation for dealings with existing customers.

At the conclusion of this task, the team should have a tentative ranking of vendors along with a list of reasons why each was ranked as it was. The ranking should be a numeric score and the reasons justifying each score.

Phone Surveys

User contacts to this point have been limited to initial site visits - generally one per vendor. After the detailed evaluation, the team will undoubtedly have questions remaining. Some of these can best be answered by talking to other users. Another set of questions - relating to each vendor's recent installation experience - can only be addressed by the newest clients. A structured survey - where at least five recent clients of each of the top ranked vendors are called - is appropriate and recommended at this time. Each site should be ask similar questions except for items that are vendor specific.   If time and resources are available then phoning every UK reference site is not out of the question.  Clearly there will be limited scope for this with suppliers with reference sites on the continent or further abroad, e.g. USA, Canada, Australia, New Zealand.

A third set of questions have little to do with vendor selection but are helpful in any case. Questions about the "installation experience" in general will help the organization avoid some of the problems faced by others. Open ended questions like "What should we watch out for?" and "What would you do different next time?" will allow others to share their experience with your team right before the installation is initiated.

Visits

Follow-Up Site Visits

If a major capability of a leading system - a specific HIS or PAS interface for example - was not observed during the initial round of visits, it might be necessary to see it at this time. Such a visit would not involve the entire team - only those who are concerned with the item in question.

Home Office/Headquarters visit

No major purchase should be made based solely on contact with a sales representative and one or two marketing support analysts. Once a system is selected an entirely different set of people will take over including installers, support personnel and developers of future enhancements.

Before a contract is signed, it is helpful if the Project Team, hospital managers and executives can meet managers in the vendor organisation. A one day visit - possibly to two or more leading vendors - can uncover information that is impossible to find in volumes of printed material.  It is worth asking to be shown the Help Support arrangements. If possible ask the supplier to lay on a presentation by the various team leaders in the supplier's company i.e. support, implementation, training, development, administration and finance.

Important questions could be put to executives of the company if answers from the salesman have been unexpected or considered as "vapourware".  Do not be afraid to ask awkward questions.

Foreign visits

Some suppliers will offer to take part or all of the Project Team to visit their reference sites abroad, e.g. USA.  If they have UK reference sites of comparable size and workload to your own then this is unnecessary, but if their only comparable sites are abroad then advice should be sought from your local NHS Regional office and funds for the trip sought from Trust funds.  If is wise to allow for a contingency such as this when allocating the cost of procurement to the organisation during the project planning stage.

If the trip is funded by a supplier then it is a direct breach of standing orders.

Cost/Benefit Analysis (RoI)

If necessary, a detailed cost-benefit (Return on Investment - RoI) analysis can be performed at this time. All of the information including exact system costs and complete descriptions of capability are available. In addition, the experience of other users - in terms of installation and operational costs - can be factored in to provide an accurate life cycle projection. 

A useful companion for calculating RoI is Investment Appraisal for IM&T a computer-based training package using spreadsheets for practising the application of some of the core techniques.

Advice from the Finance department of the organisation is essential at this stage so that the figures can be obtained from the suppliers to meet the standing orders for PFI.

Final ranking

With much of the homework accomplished, the team is in a position to recommend the top two or three ranked suppliers for draft contract negotiation. The reasons these systems were selected are completely documented. If only one supplier's system meets the evaluation criteria then be prepared to provide evidence why no others match your criteria and why it is important that the features the "single supplier" can offer match your business requirements best. It is then possible to go forward with a single supplier but a word of warning - while most suppliers will still provide their "best" price it will be tempting for them to reduce or eliminate their discounts once a competitive element is missing!

The major tasks remaining are to finalise draft contracts with the final supplier(s) and to summarise the work to-date so it can be presented to busy hospital executives and project board members in the full business case (FBC) and final recommendations.

Draft contract negotiations

Business may be awarded on the basis of any contract but only a good contract will positively aid successful achievement of the common goal. Both parties must believe they can perform the contract and intend to do so. If this intention is carried through to implementation, and the message from POISE2 is that purchase and implementation are both part of a single procurement process, the parties will work together within the terms, to fulfil it. Such an attitude creates a business-like, working relationship which will withstand change, disagreement and problem resolution as a part of a way of life.

Contract negotiations almost always take longer than anticipated. The only way to make this task go reasonably quickly is to accept the standard NHS contracts (SSCON, SYSCON, MSCON - see NHS PASA website) with few changes. Significant changes must be reviewed and agreed to by executives and lawyers from both sides. This process takes time.  The standard NHS contracts have been accepted virtually unchanged by most suppliers to the NHS.  They consist of a core contract of terms and conditions together with several project specific schedules which are tailored to your procurement.

The complex final draft contract negotiation process should not be initiated without careful consideration of the implementation plan and timescales. Not just any plan will work. Much of the detail required for the contracts will need some site assessment and pre-implementation planning by each shortlisted supplier.  Suppliers should be prepared to invest in this planning process prior to contract award. Before any post contract evaluation activity is undertaken the following questions must be considered:

  • Has adequate effort gone into developing a comprehensive project plan?
  • Does the plan make sense in content and structure to all parties?
  • Do they truly believe it is the best approach?
  • Does it involve key individuals and units throughout the organisation?
  • Does it rely on internal expertise to the maximum extent possible?
  • Is the approach in line with the modern concept of packaged software as opposed to an earlier model based on custom programming?
  • Have all parties who will be relied on for significant contributions agreed to commit the time and energy?
  • Has the approach worked well in our organisation before? In other organisations? If not, what assurance do we have that it will be successful here?

A contract, however, should not be looked at strictly as the purview of lawyers. First and foremost it must describe - in user terms - exactly what the vendor is proposing to install. Without this core technical content all the legal language in the world will not form the basis for a satisfactory installation.

A poorly run selection process cannot be salvaged at the last minute by bringing in a shrewd contract negotiator. A good contract is the culmination of a sequence of well-planned evaluation and selection tasks. It is the formalisation of the understanding between two parties that takes weeks, and even months, to develop.

Each supplier will be required to agree formally to its own version of the draft contract. The draft contracts will then be complete. Each one will represent a contract which when the pricing information is added the supplier (if selected) and the authority will be able to sign without further change.

Invitation to offer (ITO)

An invitation to offer (ITO) is a request to a potential supplier to submit price information which, in combination with a draft contract, constitutes its best and final offer the aim of POISE2 Step 22 is to ensure that those suppliers who agreed draft contracts with the authority are invited submit offer and that their offers meet the authority’s legal and administrative requirements as well as IS needs. After final evaluation, the authority must be able to accept any offer by signing the now complete contract, whereupon the supplier whose offer it is, is committed to perform the contract.

The ITO must meet the requirements of the authority’s own Standing Orders (SOs) and Standing Financial Instructions (SFIs), both of which should be reviewed early in the procurement to ensure that their effect on the ITO is fully understood. It is the authority’s responsibility to ensure that the requirements of the SOs and SFIs relating to ITO’s are carried out and that any conflicts between such requirements and POISE2 are resolved in such a way that the integrity of the procurement is maintained and any disruption to the timetable avoided.

The ITO will request all suppliers to respond by a specified date. The time allowed for suppliers to respond must meet EC requirements, which provide for a minimum period of 40 days between the dispatch of ITO’s and the receipt of tenders by the authority. However it is usual for suppliers to be asked to formally agree to a shorter time, such as 10 working days.

Before proceeding with the ITO, references may be taken up or other investigations carried out to ensure that there has been no significant change which might adversely affect the decision to invite any of the remaining suppliers to tender. Examples of such enquiries might include:-

  • a supplier’s latest financial and credit information
  • current status of other contracts in which a supplier is involved, particularly any with another authority
Final justification (FBC)

Full Business Case (FBC) approval must be sought for a scheme before it proceeds to financial close.

The FBC will be given approval when it has met all of the requirements of the Capital Investment Manual and only minor details of contractual agreement (i.e. anything that will not affect the price or level of risk transfer in the deal) are outstanding. In practice, this means that a significant majority of the contractual documentation, including schedules, will have been agreed in some detail with the private sector partner, and the financiers to the scheme will have agreed the key elements of the deal.

For an FBC to be given approval, the deal should be in such a position that will enable full financial close to be reached within two months after approval. This means that financiers’ due diligence should have commenced prior to FBC approval.

Given that FBC approval will only be given at a highly advanced stage in the development of a deal, it is essential that early drafts of the FBC which reflect how the deal is developing are shared with the NHS Executive Regional Office. Drafts of the FBC serve an important role in order that both the NHS Trust and the NHS Executive can ensure that the scheme develops in line with PFI policy.

It should also be remembered that one of the key functions of the FBC is as the key document in the audit trail in recording the decision-making process culminating in the decision to proceed with the PFI scheme.

Final decision

Some organisations rank vendors and just negotiate with the top ranked company - "single tender" if only one supplier meets the OBS. Others negotiate with the top two or three. In either case, lower ranked vendors should not be ruled out completely until an agreement is signed.

The final presentation by the Project Manager to the Project Board is straight forward at this point since the necessary homework has been done. All questions about system capability, cost, benefits, installation effort and impact on the organisation have been addressed. A simple overview of the process and the resulting recommendation is backed up by volumes of technical detail which were needed for the FBC.

Since board members have limited understanding of many applications, their questions are more likely to address process issues. Did you consider ...? Why did you take this particular step. etc. Because the process was well-structured and all parties agreed it was the best approach, there should be little difficulty in answering questions of this nature.

Once the Project Board have accepted the Project team's recommendation the paperwork may need to be "signed off" and scrutinised by the NHS Trust Board and/or the NHS Executive Regional Office if the lifetime costs (see Delegated limits table below).  As discussed in the FBC section it is very worthwhile acquainting the relevant staff at the NHS Regional Office at an earlier stage about the timescales of the project and the proximity of the final decision.  That way some indication of any likely delay can be built into the final project planning.

Delegated limits

WHOLE LIFE COST OF SCHEME APPROVING AUTHORITY
Less than £1 million for NHS Trusts,  irrespective of turnover NHS Trust Board
More than £1 million, but less than £20 million NHS Executive Regional Office.
Outline and Full Business Cases required.
More than £20 million NHS Executive (Regional Office and Headquarters). Outline and Full Business Cases. Treasury approval of Full Business Cases also required
Award of contract

NHS Trusts should not underestimate the complexity and quantity of legal documentation that needs to be finalised both by the NHS Trust, the project company and accountants before financial close can take place. The completion of the documentation will also be affected by any issues which are raised during the due diligence process. NHS Trusts should ensure that adequate time is set aside and sufficient resources allocated for this part of the process.  This process can take several weeks or even months as meeting schedules are met and information collated for presentation.

The NHS Trust must also consider its own approvals mechanism for the contract award. Not only are approvals needed within the NHS Executive; the NHS Trust will also need to have the project approved through the cycle of Trust and board meetings to agree the contract. The board will need to consider what delegated authority the NHS Trust Chief Executive will have to agree variations in the price from the FBC to that eventually agreed at financial close.

Once the NHS Trust and, if necessary, the NHS Executive are happy with the pre-contract review, the NHS Trust can proceed to financial close. Within seven days of award of a contract the team must inform the unsuccessful suppliers that a contract has been awarded. The authority is also obliged to publish such a contract award notice within 48 days of award of contract in OJEC.

Post implementation review

Traditionally, implementation has been viewed as a project stage starting when the contract is signed, encompassing installation, testing, acceptance and training, and finishing at final acceptance and payment.

The move towards service-based solutions dictates that implementation has to be considered in a wider context than that of an IT system installation, taking account of changes in the way people work and the impact of the changes on the organisation.

Successful implementation requires a firm foundation based upon:-

  • business plans and strategy
  • IM&T strategy
  • human resources strategy
  • project management and planning
  • organisational plans
  • procurement plans.

Each will influence IM&T implementation and, in time, implementation may impact on each plan.

The standard methodology for evaluating completed IM&T projects in NHS acute provider trusts is PROBE, which stands for ‘Project Review: OBjective Evaluation’. The guidelines on PROBE reviews are available from the NHS Information Authority, Electronic Record Development and Implementation Programme, which should also be consulted should additional information be required.

The PROBE guidance describes an evaluation framework which encourages and supports an objective review. Evaluation is recognised as an ongoing process, from project initiation to post implementation review. By means of good evaluation practice, the appraisal, design, management and implementation of IM&T projects can be improved.

Evaluation is mandatory for investments greater than £1 million, but this approach may be usefully applied to smaller projects, as well as those forming part of a larger programme.

Typical objectives for a post implementation review
  • assess how well the objectives of the project are being achieved
  • assess value for money
  • determine if project is on plan and identify corrective actions
  • document the lessons to be drawn for others and for the future
  • take stock for the future: identify next steps
  • identify actions to consolidate current implementation
  • identify current performance improvements
  • If you find this advice has been useful then why not contact the company that produced it and get the "full monty"!  Examples of OBS, OBC, FBC, PID, scripts for demonstrations and draft contract schedules for hospital laboratory computer procurement projects are available for personalising to your project when you place a contract with us.
    •

How to contact Feltham Associates?
Carlton House, Kibworth Hall Park, Kibworth Harcourt, Leicestershire, UK LE8 0PE
Tel : +44 116 279 3232 - Fax : +44 116 279 2473 - Mobile: 07771 967323
e-mail: Webmaster

This site is maintained by Feltham Associates Limited (November, 2009)