[Home] [Databases] [World Law] [Multidatabase Search] [Help] [Feedback]
	England and Wales Court of Appeal (Civil Division) Decisions
You are here: BAILII >> Databases >> England and Wales Court of Appeal (Civil Division) Decisions >> Eisai Ltd, R (on the application of) v National Institute for Health and Clinical Excellence (NICE) [2008] EWCA Civ 438 (01 May 2008) URL: http://www.bailii.org/ew/cases/EWCA/Civ/2008/438.html Cite as: [2008] EWCA Civ 438

[New search] [Printable RTF version] [Help]

IN THE SUPREME COURT OF JUDICATURE
COURT OF APPEAL (CIVIL DIVISION)
ON APPEAL FROM THE HIGH COURT OF JUSTICE
ADMINISTRATIVE COURT
Mrs Justice Dobbs
[2007] EWHC 1941 (Admin)

		Royal Courts of Justice Strand, London, WC2A 2LL
		01/05/2008

Lord Justice Richards :

1. The issue in this appeal is one of procedural fairness. The National Institute for Health and Clinical Excellence ("NICE") is responsible for appraising the clinical benefits and cost effectiveness of health care interventions notified to it by the Secretary of State and for making recommendations as to their use in the NHS. The interventions relevant here are certain drugs for the treatment of Alzheimer's Disease ("Alzheimer's" or "AD"). The drugs, known as acetylcholinesterase inhibitors ("AChEIs"), had previously been recommended for use in the treatment of NHS patients with mild to moderately severe AD. In 2006, however, NICE issued fresh guidance recommending their use only for patients with moderately severe AD (as determined on a basis laid down in the guidance). The claimant ("Eisai") is a pharmaceutical company holding the UK marketing authorisation for one of the drugs concerned, donepezil, which is marketed under the name Aricept. Eisai brought judicial review proceedings challenging the fresh guidance on a number of grounds. At first instance before Dobbs J, it was partially successful on one ground, relating to non-compliance with anti-discrimination legislation, but failed on the other grounds of procedural unfairness and irrationality. On this appeal it pursues only the ground of procedural unfairness.
2. The point is a relatively narrow one, though arising in a context of some technical complexity. In its consultation process NICE made available to consultees, including Eisai, a read-only version of an economic model, in the form of an Excel spreadsheet, which was used to assess the cost-effectiveness of the drugs. Eisai requested, but was refused, a fully executable version of the model. Eisai's case is that the non-provision of a fully executable version rendered the consultation process unfair and that the decision to issue the guidance was in consequence unlawful.
3. That case was presented for Eisai by Mr Pannick QC. He was supported by Mr Kennelly for Shire, another pharmaceutical company with UK marketing rights for a drug affected by the guidance, which appeared as an interested party both in the court below and on the appeal. The case was also supported by a written intervention on the part of the Association of the British Pharmaceutical Industry ("the ABPI"), which did not appear as an interested party in the court below (though the court did receive evidence from its Director General). The Alzheimer's Society, which did appear as an interested party in the court below, did not pursue its intervention to this court but filed evidence making clear its continued support for the concerns expressed by Eisai about the consultation process. In the circumstances, because the ground was sufficiently covered in Mr Pannick's submissions, it will not be necessary for me to make much reference to what others have said in support of Eisai's case.
4. NICE, represented by Mr Giffin QC, not only resists Eisai's case in relation to procedural unfairness but also has a limited cross-appeal of its own, in relation to the order by Dobbs J that NICE pay 40 per cent of Eisai's costs of the judicial review proceedings. Whether the cross-appeal raises a live issue depends, however, on the outcome of Eisai's appeal.

NICE's functions

5. NICE was established as a special health authority by the National Institute for Clinical Excellence (Establishment and Constitution) Order 1999. By article 3 of that Order it is required to perform various functions, including functions in connection with the effective use of available resources in the health service, as directed by the Secretary of State. The current directions are the Directions and Consolidating Directions to the National Institute for Health and Clinical Excellence 2005. By para 2(1)(a) of those Directions, the Secretary of State directs NICE "to appraise the clinical benefits and the costs of such health care interventions as may be notified by the Secretary of State and to make recommendations". By para 2(4), in exercising that and other functions NICE is required to have regard to various factors, including (a) the broad balance of clinical benefits and costs, and (b) the degree of clinical need of patients with the condition or disease under consideration. By para 2(5) it is directed to develop a process to enable it to ascertain and take into account the views of the general public in connection with the exercise of relevant functions. By para 5 it is directed to make arrangements for holding appeals, on the application of persons aggrieved by recommendations made by it under para 2(1)(a), "on the grounds that the Institute has failed to act fairly, has exceeded its powers or has formulated guidance which cannot reasonably be justified".
6. NICE's guidance does not override the clinical judgment of healthcare professionals, though they are expected to take the guidance into account. Nor is there any prohibition on relevant NHS bodies funding treatment not recommended by NICE or in circumstances not recommended by NICE. By separate directions of the Secretary of State, however, Primary Care Trusts and NHS Trusts in England are required to provide funding to ensure that health care interventions recommended in guidance issued by NICE following technology appraisals conducted in accordance with its published procedure are normally available. NHS core standards also require health care organisations to ensure that they conform to NICE technology appraisals. In practice, therefore, NICE guidance has a vital role in determining the availability of treatments within the NHS.

NICE's system of technology appraisals

7. For a description of NICE's general system of technology appraisals, I can adopt with gratitude the relevant part of Dobbs J's clear and thorough judgment (though I omit detailed cross-references to the evidence):

"12. When a particular topic has been selected for appraisal, NICE identifies those organisations which might have an interest in the outcome of the appraisal. These typically include groups representing patients and carers, bodies representing healthcare professionals, manufacturers, and research groups. Such stakeholders are divided into 'consultees' and 'commentators', with only consultees having a right of appeal against NICE's eventual Guidance. NICE works with the Department of Health (DoH) to produce a scope for the appraisal, which, once finalised, the DoH formally instructs NICE to carry out.

13. An independent academic centre is commissioned to review and evaluate evidence on the health technology under appraisal. It produces a Technology Assessment Report (TAR) which presents an analysis of the cost-effectiveness of the health technology. Often (as in the present case) the academic centre will build an economic model to inform its analysis. Consultees and commentators are invited to comment on the Assessment Report and to make submissions. The Assessment Report and comments made on it are then drawn together in an Evaluation Report.

14. The appraisal is then allocated to one of NICE's Appraisal Committees (AC). The Appraisal Committees are standing committees, the members of which are drawn from a range of backgrounds including eminent clinicians, health administrators, academics, representatives of the pharmaceutical industry and lay members.

15. The Appraisal Committee considers the Evaluation Report. It then hears evidence from clinical experts nominated by the consultees and commentators, and from patients and carers before making initial recommendations in an Appraisal Consultation Document (ACD) about the use of the health technology. Consultees and commentators have four weeks in which to comment on the ACD. The Appraisal Committee considers all comments received. It then makes its final recommendations in a Final Appraisal Determination (FAD).

16. The FAD is submitted to the NICE Guidance Executive for its approval. Once the FAD is approved, it is circulated to consultees and commentators. Consultees have 15 days during which to appeal against its recommendations. If there are no appeals, or if the appeals made are dismissed, the final recommendations are issued as NICE Guidance. If an appeal is successful, the Appraisal Committee reconsiders its recommendations.

17. The key principle underlying NICE's approach to appraisals is that the NHS's limited resources should be targeted on those treatments which provide best value for money. The principle is to be found at paragraph 4.1 of NICE's 'Social Value Judgments – principles for the development of NICE Guidelines' (8^th December 2005) and is summarised by Professor Andrew Stevens, the chair of the Appraisal Committee which appraised the AChEIs in the present case:

'If money is spent on donepezil for patients with mild Alzheimer's disease, then that money is not available to spend elsewhere on the treatment of other patients. The role of NICE's appraisal committees is therefore to judge whether the clinical and cost effectiveness of a technology being appraised is such as to justify spending the NHS's money on that technology, rather than on one of the many other technologies competing for the finite resources available.'

18. In determining whether particular treatments are cost-effective, NICE generally seeks to ascertain the incremental cost per quality adjusted life year (or 'cost per QALY') of using a particular health technology. In broad terms, this is a standard measure of the cost of gaining a particular unit of utility through the use of the technology which can be applied across the whole spectrum of treatments appraised by NICE.

19. The general threshold for an 'acceptable' cost per QALY is approximately £20,000 per QALY: see 6.2.6.10 of NICE's 'Guide to the Methods of Technology Appraisal' (April 2004). Above approximately £30,000 per QALY, technologies are not normally recommended (para. 6.2.6.11), although there has been a small number of exceptions. Between £20,000 and £30,000 per QALY, NICE will consider whether there are specific considerations which justify recommending the technology, such as its innovative nature or the particular needs of the relevant patient group. The costs per QALY in this case were significantly in excess of these normal thresholds reflected even in the category for which NICE approved the use of AChEIs."

The appraisal process in this case

8. In order to put the submissions in context, it is also necessary to set out the history of the particular appraisal process under challenge. Again, I can take it from the judgment of Dobbs J (omitting detailed cross-references):

"20. … On 30^th January 2004, consultees and commentators were sent the final scope for the appraisal of drugs for the treatment of AD. On 1^st April 2004 consultees (including Eisai and the Interested Parties) were issued with the Protocol under which the cost-effectiveness of the AChEIs was to be assessed. On 3^rd June 2004 written submissions were received from a total of twelve consultees and commentators, including Eisai and the Interested Parties. All four of the relevant manufacturers submitted their own economic models of the cost-effectiveness of the drugs under appraisal.

21. On 31^st August 2004, NICE received the final Assessment Report from Southampton Health Technology Assessment Centre (SHTAC). This included an economic model to illustrate the cost-effectiveness of the AChEIs (the Model). The Model was in the form of an Excel spreadsheet. Both the design of and the assumptions underlying the Model were explained in Section 6 of the Assessment Report.

22. The initial assessment of the cost-effectiveness of Donepezil, based on the available clinical evidence, indicated a cost per QALY of approximately £94,000. This was far outside the range within which a treatment might be recommended as cost-effective. The view reached by the Appraisal Committee was, that if recommended for use amongst all mild to moderate AD sufferers, the AChEIs would not come even close to achieving the levels of cost-effectiveness generally required for recommendation by NICE for use within the NHS.

23. On 17^th September 2004, SHTAC's Assessment Report was sent out for consultation. All consultees and commentators were offered a copy of the Model (in 'read-only' format). Eisai did not take up this offer. On 4^th October 2004, comments on SHTAC's Assessment report were received from ten consultees and commentators, including Eisai and the Interested Parties.

24. On 20^th October 2004, the Appraisal Committee held its first meeting to discuss the formulation of the ACD. The Appraisal Committee requested that further analysis be undertaken on the cost-effectiveness of the AChEIs by NICE's secretariat before an ACD could be published. This was in response to representations made by consultees. In particular, the Appraisal Committee specified that alternative assumptions, which were generally more favourable to the drugs than the original assumptions made, should be used in the Model. This new set of assumptions was referred to as the 'augmented base case'. In addition, the Appraisal Committee asked that sensitivity analyses be carried out in relation to certain assumptions in the augmented base case to see if the Model was particularly sensitive to these changes.

25. In accordance with the Appraisal Committee's request, NICE's technical staff carried out further analyses of the cost-effectiveness of the AChEIs based on a variety of alternative assumptions. They also carried out further sensitivity analyses of the Model. These analyses were set out in Technical Report No 1 (TR1), circulated on 7^th January 2005 to all consultees and commentators for their observations. The effect of the augmented base case was to produce very similar conclusions on the benefits of treatment with AChEIs to those produced by the models submitted by the manufacturers. This further analysis indicated a cost per QALY for Donepezil of approximately £54,000. This figure was still well outside the range for which the treatment might be recommended for AD sufferers as a whole.

26. On 21^st January 2005, comments on Technical Report No 1 were received from ten consultees and commentators, including Eisai and the Interested Parties. On 25^th January 2005 the Appraisal Committee met for a second time to consider Technical Report No 1 and the comments received.

27. By an email dated 3^rd February 2005, all consultees and commentators were again given the opportunity to receive a copy of the Model, now in revised form and including the augmented base case, in 'read-only' format. On 8^th February 2005, Eisai, for the first time, requested a copy of the Model. On the same day, NICE sent Eisai a copy of the Model, in 'read-only' format.

28. Following the second meeting of 25^th January 2005, the Appraisal Committee reached a preliminary view not to recommend the use of AChEIs for the treatment of AD at all. The drugs were not considered cost-effective enough to justify their use, bearing in mind their limited clinical effectiveness. This view was set out in the first Appraisal Consultation Document (ACD1). ACD1 was sent to all consultees and commentators on 21^st February 2005, and further comments were invited. ACD1 was published on NICE's website on 1^st March 2005.

29. On 28^th February 2005, Eisai wrote to NICE requesting a fully-executable copy of the Model. On 4^th March 2005 NICE replied to Eisai explaining its policy to limit disclosure to 'read-only' copies of its economic models. On 22^nd March 2005, responses to ACD1 were received from eighteen consultees and commentators, including Eisai and the Interested Parties. In total, over 8,000 contributions were received during the consultation on ACD1. One recurring theme of this consultation was that, if a group of patients who did respond to the AChEIs could be identified, (a so-called 'responder group'), and treatment were targeted on that group, then the drugs might be sufficiently cost-effective at least for that group.

30. In response to this, NICE, on 12^th April 2005, requested data from the consultees and commentators, as well as from practitioners and others with an interest in data collection relating to the use of AChEIs in clinical practice. In particular, NICE sought data on patients who had responded, and data on how long non-responders had remained on treatment with AChEIs. On 20^th April 2005 Eisai responded to this request by providing additional data, as did other consultees and commentators.

31. On 1^st June 2005, the Appraisal Committee met for the third time. It considered the additional data submitted in response to NICE's request. Additional economic modelling was carried out on the basis of some of the suggestions made by consultees. The Appraisal Committee was not, however, persuaded by the data provided, purporting to show that initial responders could be regarded as a sub-group of patients that enjoyed significantly better-than-average benefits from treatment with AChEIs. Following the meeting on 1^st June 2005, the Appraisal Committee prepared a Final Appraisal Determination (FAD). However, uncertainty remained as to whether it might be possible to identify a sub-group of patients for whom the AChEIs could be demonstrated to be cost-effective. NICE's Guidance Executive reached the view that, before a FAD was issued, further analysis should be undertaken. Unusually, therefore, the FAD was not issued at this stage. Instead, a request was made by NICE to the manufacturers of the AChEIs (including Eisai) for further individual patient-level analyses of data from the key clinical trials. NICE held a meeting with the manufacturers on 10^th August 2005 to ask for such data. Following submission of the requested data, NICE held a meeting with the manufacturers on 31^st October 2005 and asked for still further analyses. On 6^th November 2005 Eisai provided further analyses in response to this request.

32. The Medical Research Council's Biostatistics Unit was commissioned by NICE to interpret the data provided by the manufacturers and to report on it to the Appraisal Committee. NICE's technical staff then used the data report by the MRC Biostatistics Unit to re-run the augmented base case Model. The results of this re-running of the economic model were reported in Technical Report No 2 (TR2), issued on 25^th November 2005. This report was provided to all consultees. Eisai did not ask for a copy of the re-run economic model (in non-executable form) at this stage, but did provide a response to the report on 7^th December 2005. On 9^th December 2005 a further twelve responses to Technical Report No 2 were received from commentators and consultees, including the Alzheimer's Society.

33. In the light of the additional analyses carried out, the Appraisal Committee now found that, by adjusting its assumptions in the drug's favour, and by limiting treatment to patients with moderately-severe (as opposed to mild) AD, the cost-effectiveness of the AChEIs began to approach the level usually required for a treatment recommended for use in the NHS. On this approach, the estimates of cost per QALY for donepezil ranged from £31,000 to £38,000. For the AChEIs, taken as a class of drugs, a cost per QALY of £35,000 was arrived at.

34. On 20^th December 2005, the Appraisal Committee met for the fourth time. This led to the formulation of the second Appraisal Consultation Document (ACD2), which was sent to all consultees and commentators on 16^th January 2006. It was published on NICE's website on 23^rd January 2006. On 10^th February 2006 Eisai submitted its response to ACD2. On 13^th February 2006 comments on ACD2 were received from twenty further consultees and commentators, including the Interested Parties.

35. On 27^th April 2006, the Appraisal Committee met for the fifth time to consider ACD2 and the comments that had been received on it. This led to the formulation of the Final Appraisal Determination (FAD). On 25^th May 2006 the FAD was sent to all consultees and commentators. It was published on NICE's website on 26^th May 2006 …."

9. As required by the 2005 Directions (see para 5 above), there is a right of appeal to an Appeal Panel against final appraisal determinations. Eisai was one of a number of consultees to appeal in this case. The appeals were heard by the Appeal Panel on 13–14 July 2006. The Appeal Panel's decision, rejecting the appeals, was sent to the parties on 9 October 2006.
10. Thereafter, on 22 November 2006, the guidance containing the final recommendations was published, including certain amendments to the FAD which arose from the Appeal Panel decision.

The disclosure issue

11. As can be seen from the summary set out above, the process of appraisal was exceptionally detailed and elaborate, involving extensive consultation and a high degree of disclosure at all relevant stages. This reflects NICE's fundamental (and highly commendable) philosophy towards the discharge of its functions. For example, its Guide to the Methods of Technology Appraisal states that "[t]o ensure that the guidance issued by the Institute is appropriate and robust, it is essential that the evidence and the analysis and their interpretation are of the highest standard and are transparent to scrutiny" (para 3.1.1). Its Guide to the Technology Appraisal Process states that "[i]deally, all the evidence seen by the Appraisal Committee should be available to consultees and commentators" (para 4.2.3).
12. Eisai's only complaint about the procedure relates to the non-disclosure of the fully executable version of the economic model.
13. NICE's policy on release of the model is set out in its Guide to the Technology Appraisal Process, at para 4.4.1.9:

"The Assessment Group may produce an economic model in support of the Assessment Report. If the model does not contain information that was designated as confidential in the submission, the Institute offers consultees and commentators the opportunity to receive by email a read-only version of the model, for information only. Requests for the model must be made in writing, and it is supplied on the basis that the consultee or commentator agrees, in writing, to the following conditions for its use.

The economic model and its contents are confidential and are protected by intellectual property rights, which are owned by the relevant Assessment Group. It cannot be used for any purpose other than to inform the recipient's understanding of the Assessment Report.

The model must not be re-run with alternative assumptions or inputs.

The consultees or commentators will not publish the model wholly or in part, or use it to inform the development of other economic models."

14. The reference to a "read-only" version is a convenient shorthand: that version of the model can in fact be run, but changes cannot be made to the inputs or assumptions. The fully executable version is required for the purpose of enabling such changes to be made. NICE's policy, consistent with the second of the conditions set out above, is not to release the fully executable version. There have been exceptions to that policy, but no such exception was made on this occasion.
15. Eisai contends that the failure to provide the fully executable version (and to do so on a basis that would allow it to be re-run with alternative assumptions and inputs) is unfair. It means that Eisai is unable to test the reliability of the model by running sensitivity analyses and by tracking the formulae so as to check their accuracy. This prevents it from making informed representations on a central element in the appraisal process. Eisai contrasts NICE's refusal to disclose the fully executable version of its model with NICE's requirement that, where manufacturers submit models of their own as part of their consultation responses, they must provide the fully executable versions of those models.
16. NICE's stance is that there is already sufficient disclosure, including provision of the read-only version on request, to meet the requirements of procedural fairness; and it puts forward two reasons why it is unwilling to go further. First, it says that the model is provided to it on terms as to confidentiality which preclude the disclosure of the fully executable version. The second reason is a practical one, that disclosure of the fully executable version and the additional representations that this would generate would be liable to create a great deal of extra work and delay, slowing down the appraisal process by 2 to 3 months, and that this would not be justified by the marginal gain to the consultees of having the fully executable version rather than just the read-only version. NICE also takes a separate point on delay in applying for judicial review, which is relevant primarily to the question of relief if Eisai is successful in its claim.

The decisions of the Appeal Panel and of Dobbs J

17. In rejecting Eisai's appeal, the Appeal Panel concluded that "whilst it was not possible for the appellant to check the accuracy of the cost effective calculations, there was sufficient interaction possible with the model to understand how the model operated, and comment adequately on the model" (para 2.22). It noted that Eisai had indeed commented on the model, along with other consultees, "demonstrating meaningful engagement". The Panel considered that it was not necessary for appellants to "quality assure" NICE, as NICE and consultees had different roles in developing the guidance: NICE had to quality assure any data or model which it took into account, which was why manufacturers were required to provide an executable version of any model they put forward. As to NICE's concerns that disclosure of a fully executable version would be burdensome for it, the Panel said (at para 2.29) that questions of proportionality and timing were relevant and that it was important for the Appraisal Committee to keep to the appraisal timetable.
18. Mr Pannick submitted that the Appeal Panel erred in three respects. First, the fact that there was "meaningful engagement", in the sense that Eisai made detailed representations as best it could on the material it did have, does not answer the complaint that the lack of the fully executable version prevented comment on crucial aspects of the evidence. Secondly, it was wrong in principle to say that consultees did not need the fully executable version because it was NICE's role to "quality assure": consultees are entitled to check and comment on the evidence relied on by the decision-maker, rather than having to take the decision-maker's work on trust. Thirdly, reliance on the burden to NICE was inconsistent with the evidence given to the Appeal Panel by NICE's chief executive, Mr Dillon, who said in terms that "[i]f there were no restrictions placed by those who produce these models on the way in which we can use them, we would have no objection to models being made available in that form to consultees". In any event, if fairness requires disclosure, then the additional burden and time are no answer; and it would be open to NICE to impose a timetable and other appropriate conditions so as to limit the impact of disclosure on NICE itself and on the appraisal process.
19. It is true that there was some material before the Appeal Panel raising the issue of disproportionate burden to NICE if the fully executable version were disclosed: a Note to the Appeal Panel from NICE said that it would create a number of practical difficulties and in particular that "if consultees were able to provide re-workings of the Assessment Group's models … it would become almost impossible for the Appraisal Committee to establish the basis of the re-workings at all … unless the Committee relied entirely on those responsible for the re-workings to identify fully and accurately the changes made to the data", and that "[i]t would certainly be impossible within the timescales of the appraisal". Yet this cannot have been a central part of NICE's case on the appeal: if it had been, the evidence of Mr Dillon to which I have just referred would not have made sense. Further evidence directed to the issue, substantially expanding on NICE's concerns, was adduced in the judicial review proceedings. Dobbs J, however, stated at para 59 of her judgment that she would not rely on the "new reasons" advanced by NICE, in the light of submissions made regarding the risk of ex post facto justification.
20. Dobbs J went on to say at para 60 that there was no right for consultees to quality assure the model, nor was there any obligation on NICE to allow them to do so. NICE was the decision-maker and the body responsible for obtaining the model and ensuring that it had been quality assured. At para 61 she posed the question: "was Eisai denied access to significant information and deprived of the opportunity to make intelligent response?" In her judgment the answer to that question was in the negative, for reasons set out in nine sub-paragraphs. She then summarised the matter as follows:

"62. In summary, the Claimant had more than sufficient information to make intelligent response and to give proper advice. What was not possible was to 'run or play with the system' to see how it would react to different data and assumptions being put in. Eisai was not, however, deprived of the opportunity of putting forward suggestions and concerns in the light of what it was able to observe. So intelligent indeed were some of the responses and advice that alterations were made by NICE to the Model as a result. Whilst highlighting its inability to run the model and thereby test its robustness and accuracy, Eisai was nevertheless still able to advise on the need to check the accuracy of figures and to run sensitivity analyses with the different assumptions. I agree with the decision of the Appeal Panel that the withholding of the fully executable model was not unfair."

21. The judge went on to deal with the issue of confidentiality, finding that there was a clear policy, of which consultees were fully aware, in relation to the receipt and release of models: the provision of the SHTAC model was on the basis that it was in confidence. Eisai wanted to be able to run the model, but SHTAC, as the owner of the intellectual property rights, was entitled to object to such a course of action. NICE was obliged, in the absence of any public interest requirement for disclosure, to honour the rights of SHTAC.
22. Mr Pannick made detailed criticisms of the judge's reasoning, submitting that she repeated the first two of the errors made by the Appeal Panel (see para 18 above) and that she also fell into error in her assessment of confidentiality.
23. Having explained the nature of the dispute and the conclusions reached by the Appeal Panel and Dobbs J, I turn now to consider the relevant legal principles before proceeding to apply them to the detailed facts of the case.

The legal principles

24. It is not in dispute that NICE is subject to the general principles of procedural fairness in relation to the appraisal process and, in particular, that it must act fairly in the consultation exercise. Mr Giffin suggested that NICE was under no duty to consult and had simply taken a voluntary decision that this was an appropriate way of performing its functions. Given the interests at stake and the obligation to comply with the Secretary of State's directions (which include requirements to develop a process to enable it to ascertain and take into account the views of the general public, and to make provision for an appeal on grounds of procedural fairness: see para 5 above), I very much doubt whether it would be open to NICE to dispense with consultation. In any event, whether or not consultation is a legal requirement, if it is embarked upon it must be carried out properly: R v North and East Devon Health Authority, ex parte Coughlan [2001] QB 213, per Lord Woolf MR at para 108.
25. Lord Woolf's judgment in ex parte Coughlan continues with helpful statements of general principle:

"108. … To be proper, consultation must be undertaken at a time when proposals are still at a formative stage; it must include sufficient reasons for particular proposals to allow those consulted to give intelligent consideration and an intelligent response; adequate time must be given for this purpose; and the product of consultation must be conscientiously taken into account when the ultimate decision is taken ….

112. … It has to be remembered that consultation is not litigation: the consulting authority is not required to publicise every submission it receives or (absent some statutory obligation) to disclose all its advice. Its obligation is to let those who have a potential interest in the subject matter know in clear terms what the proposal is and exactly why it is under positive consideration, telling them enough (which may be a good deal) to enable them to make an intelligent response. The obligation, although it may be quite onerous, goes no further than this."

26. The mere fact that information is "significant" does not mean that fairness necessarily requires its disclosure to consultees. In Coughlan itself, for example, it was held that there was no need to disclose to the claimant a report which had been received from a third party in response to the consultation exercise, even though that report was plainly significant. I think it better, therefore, not to follow Dobbs J's approach of asking whether Eisai was "denied access to significant information". Nevertheless the degree of significance of the undisclosed material is obviously a highly material factor.
27. What fairness requires depends on the context and the particular circumstances: see, for example, R v Secretary of State for Education, ex parte M [1996] ELR 162, at pp.206-207, where Simon Brown LJ emphasised the need to avoid a mechanistic approach to the requirements of consultation. It seems to me that the various cases cited to us provide illustrations of that, without adding materially to the statements of principle in ex parte Coughlan.
28. For example, Mr Pannick relied on a passage in the speech of Lord Diplock in Bushell v Secretary of State for the Environment [1981] AC 75, at p.96, where he said that "[f]airness … also requires that the objectors should be given sufficient information about the reasons relied on by the department as justifying the draft scheme to enable them to challenge the accuracy of any facts and the validity of any arguments upon which the departmental reasons are based". That was in the context of procedural fairness in relation to a public inquiry into draft schemes for the construction of motorways; and it is to be noted that Lord Diplock had previously observed, at p.95, that "[w]hat is a fair procedure is to be judged … in the light of the practical realities as to the way in which administrative decisions involving forming judgments based on technical considerations are reached" (p.95).
29. Mr Pannick also relied on R v Secretary of State for Health, ex parte United States Tobacco International Inc [1992] 1 QB 353. The applicant company, a producer of oral snuff, had opened a factory in the United Kingdom after the Government, acting on advice from an expert committee, had negotiated an agreement with it to restrict the marketing of the product. The committee, basing itself not on new evidence but on a changed evaluation of the existing evidence, subsequently advised the Secretary of State to ban the product; and the Secretary of State, without disclosing the advice to the company, laid regulations banning the product before Parliament. The court held that the Secretary of State had a duty to consult the company and had acted unfairly in failing to disclose the committee's advice. Taylor LJ described the advice as "crucial" (p.370H) and stated (at p.371F-H):

"One cannot help feeling that the denial of the applicants' request was due to an inbuilt reluctance to give reasons or disclose advice lest it give opponents fuel for argument. One can understand and respect the need for ministers to preserve confidentiality as to the in-house advice they receive on administrative and political issues from their civil service staff. But here, the advice was from a body of independent experts set up to advise the Secretary of State on scientific matters I can see no ground in logic or reason for declining to show the applicants the text of the advice. In view of the total change of policy the Regulations would bring about and its unique impact on the applicants, fairness demanded that they should be treated with candour. To conceal from them the scientific advice which directly led to the ban was, in my judgment, unfair and unlawful."

30. I do not accept, as Mr Pannick submitted, that one can extract from that passage any principle that fairness requires disclosure of advice obtained from an outside adviser. The fact that the material in question comes from independent experts is plainly relevant to the overall assessment, but it was a combination of factors – including the requirement of a high degree of fairness (see the reasons at p.370B-D), the crucial nature of the advice, the lack of good reason for non-disclosure, and the impact on the applicants – which led to what was on the facts a fairly obvious conclusion. The reasoning and conclusion depended heavily on the particular context.
31. Mr Giffin sought to argue, by contrast, that in the present case there was a significant element of internal advice, in that it was discussion within the Appraisal Committee which led to the incorporation of an "augmented base case" in the model (see paras 24-25 of the judgment of Dobbs J, quoted above). He relied on passages in Bushell (see above) and R v Secretary of State for Education, ex parte S [1995] ELR 71, of which one also sees some trace in the passage I have quoted from US Tobacco, to the effect that there is no duty to disclose and invite comment on internal advice. For my part, however, I see no real similarity between the model and the kind of internal advice under consideration in those cases, and in my view they do not provide material assistance.
32. Other cases to which Mr Giffin referred in his oral submissions were R v Airport Co-ordination Ltd, ex parte Aravco Ltd [1999] EuLR 939 and R (Bedford) v London Borough of Islington [2003] Env LR 22. Again, I regard them as illustrative rather than adding to the relevant principles. There is a passage in Aravco, at p.949G-H, which might be taken to suggest that the requirements of fair consultation are a matter of discretion for the decision-maker; but Mr Giffin rightly disavowed any such argument, submitting only that in deciding what fairness requires the court should attach some weight to the considered view of the decision-maker. He also submitted that the confidentiality of information and the fact that its disclosure may lead to delay in the decision-making process are factors militating against disclosure (see e.g. ex parte S at p.85D-G), but he accepted that they are not determinative. None of this gave rise to any serious disagreement.
33. Such debate as took place between counsel in relation to the authorities amounted in my view to minor skirmishing. Overall, as it seems to me, this case depends not on the resolution of any real dispute about the legal principles, but on the application of well established principles to the particular context and particular circumstances of NICE's appraisal process.

Application to the facts

34. In conducting the appraisal process, NICE is discharging an important public function which engages a strong public interest. I have referred above to the vital role that NICE guidance has in determining the availability of treatments within the NHS - treatments which, as Mr Pannick submitted, may at the lowest have a substantial effect on quality of life and may at the highest save life. NICE has recognised this in its acceptance of the need for a very high degree of transparency in the process, with an exceptional degree of disclosure and consultation. This sets the context within which the non-disclosure of the fully executable model has to be assessed.
35. The importance of the model within the appraisal process is not in doubt. It is central to the Appraisal Committee's determination of a drug's cost- effectiveness and in particular to the cost per QALY and whether it comes within the threshold of acceptable cost. NICE's Guide to the Methods of Technology Appraisal states in para 5.1 that "[t]he estimates of clinical and cost effectiveness are, individually, key inputs into the decision-making of the Appraisal Committee" and that "they are interdependent because comprehensive, transparent and reproducible synthesis of all relevant evidence on health effects is needed for high-quality cost-effectiveness analysis". It is through the model that the synthesis is achieved.
36. The robustness or reliability of the model is therefore a key question. For the thorough testing of reliability, there can be no doubt that a fully executable version is required. NICE uses that version for quality assurance of its own model and insists that, if consultees put forward a model of their own, they provide a fully executable version to facilitate review.
37. There is significant disagreement between the experts about the extent of disadvantage to a consultee if NICE provides it with only a read-only version of its model.. One of the witness statements on behalf of Eisai comes from Professor Jaime Caro, a US-based physician and epidemiologist specialising in the modelling of diseases and their treatments. He is highly experienced in the field and supplied the original model (the "AHEAD" model) which SHTAC then modified and provided to NICE for use in this appraisal process. He takes strong objection to the fact that NICE releases to consultees only the read-only (or "locked") version of the model. Thus he says:

"18. As NICE itself recognizes when requesting that models are submitted for its review, it is not possible to properly assess a model that is locked and does not allow changes to be made to inputs. The reason is that one of the main ways of detecting problems in a model is to carry out 'extreme value analysis' where input values are chosen so that the analyst can predict the results. For example, [if] is the probability of death is set to zero, then no deaths are expected and this can be checked easily. In this way, the technical validity of the model can be ascertained.

19. In addition, assessing a model requires elucidating its structural connections and the formulas that implement these. Although it is possible to view the formulas, it is difficult to follow them in a locked model as none of the normally available tools to do so are functional. Having served as external reviewer for numerous models over the years, I have never accepted a locked model. The assessment would be partial at best and as a reviewer I could not assert that I had determined the accuracy and validity of the model. I cannot believe that any serious reviewer of a model would agree to such a restriction.

…

24. The only reasons to lock a model are to protect it from inadvertently being modified in error – not a concern in this case – or because the developer fears that reasonable alternative assumptions and inputs (in a sensitivity analysis or its ilk) will reveal how weak the model and results really are.

…

29. In order to understand the impact of changes made on behalf of NICE to inputs and assumptions and to assess the validity of the resulting estimates, we examined the two models in detail [i.e. the original AHEAD model and the model as modified for NICE] and attempted to recreate the modified version, using our AHEAD model. In this effort, we spent several weeks involving the entire, original modelling team. We also ran AHEAD with the NICE inputs and modified it to correspond with the NICE version as much as possible. Despite having intimate knowledge of the AHEAD model, we were unable to replicate the results considered by NICE.

…

35. Economic modelling must be sufficiently valid to properly inform decision-makers. This requires that the model be verified to exclude errors as much as possible and that those affected by the decisions be allowed to assess the model fully – including changing inputs to other reasonable values …."

38. Professor Martin Knapp, of the London School of Economics, makes similar points about the limitations of the read-only version, stating that the inability to track formulae automatically makes it impossible in practice to confirm that the model contains no errors and has been correctly constructed with the appropriate formulae. The read-only version also allows inadequate consideration of sensitivity analyses carried out by technical staff at SHTAC and NICE and makes it impossible to carry out further sensitivity analyses. He identifies some important areas where alternative data assumptions or modifications to the model structure should in his view be tested.
39. That powerful evidence must be set against a body of evidence from NICE's expert witnesses, including Mr Meindert Boysen, who is NICE's Associate Director for Single Technical Appraisals and was the lead technical analyst in this review; and Mr Colin Green, now a senior lecturer in health economics at the University of Essex but who worked at the relevant time within SHTAC and was responsible for the production and delivery to NICE of the model which was used in this case.
40. Mr Boysen states that errors in the model can be detected easily with all the functions that are enabled in the read-only version. Formulae can be checked manually, and this was a relatively simple model in which a user would be expected to click no more than 107 cells (out of a total of over 96,000) in order fully to understand and evaluate the workings of the base case model. It would not in fact have been a particularly onerous task to "reverse engineer" the model from the information available and create a fully functioning version (though reverse engineering was prohibited by the conditions on which the read-only model was supplied).
41. Mr Green describes the read-only version as fully transparent: it contains all of the same information as the fully executable version. The inability to use the automatic tracking function is said not to be an obstacle to analysis, but merely an inconvenience, as the relationship between cells can still be checked manually; and he, too, points to the relatively small size of this model and the limited number of cells that need to be checked. He accepts that it is impossible to insert different values into the cells in order to run probabilistic sensitivity analyses.
42. I have referred to only a part of a large body of expert evidence in the case, concentrating on the passages to which our attention was drawn in oral submissions. Further witness statements are referred to in the skeleton arguments, but they do not appear to me to add materially to the contrasting views set out above.
43. Where does this evidence leave the court? In so far as there is a disagreement between the experts as to the significance of the inability to track formulae automatically in the read-only version, the court is plainly not in a position to resolve the dispute. On any view a consultee denied the fully executable version is in this respect at a practical disadvantage, but the court should proceed on the basis that it has not been shown to be an insuperable problem: the formulae can be checked manually, even if it is a more laborious and time-consuming process.
44. In relation to sensitivity analyses, however, there is no relevant disagreement: it was found by the Appeal Panel, and is common ground, that such analyses cannot be carried out with the read-only version. Further, I do not see anything in NICE's evidence to contradict what Eisai's witnesses say about the importance of sensitivity analyses in checking for problems in a model. It is not surprising, therefore, that Mr Pannick put the main focus of his submissions on the inability to carry out sensitivity analyses rather than on the problem of checking the formulae.
45. The Appeal Panel, as I have explained, took the view that it was for NICE to quality assure the model and that it was not for consultees to perform that function; and a similar point was made by Dobbs J. I accept, of course, that NICE, as the decision-maker, is responsible for checking the reliability of the model. But I agree with Mr Pannick that this does not answer the question whether fairness requires consultees to be given the opportunity to test the reliability of the model themselves, for the purpose of making informed representations on it. The Appraisal Committee has to rely on others to check that the model is robust (the fully executable version was provided in this case to only 2 out of 32 members of the committee). The checking is not something on which SHTAC and NICE's technical staff alone may have a relevant input. Whether the model has weaknesses is a matter on which consultees may properly have something to say. Indeed, they already do have things to say on the basis of the read-only version, and attention is properly paid to their representations. The carrying out of additional tests that are possible only with the fully executable version does not give rise to any difference of principle. On the face of it, to limit the extent to which consultees can engage in the legitimate task of testing such an important element in the appraisal process does seem to me to be unfair. Moreover, the possibility cannot be excluded that work done by them on the fully executable version would bring to light hitherto unrecognised weaknesses in the model.
46. Mr Giffin submitted that Eisai's ability to engage in critical appraisal was not hampered materially by the lack of the fully executable version of the model. Quantifying cost-effectiveness requires the application of assumptions; an economic model is the way of synthesising and working with those assumptions. The proper subject-matter of consultation was a debate about the appropriate assumptions. The consultees received a very detailed Technical Assessment Report as well as the read-only version of the model (if they requested it); they knew the assumptions and were in a position to debate them, as well as being in a position to make comments on the model itself.
47. Mr Giffin took us through the Technical Assessment Report, sent out for consultation in September 2004, to show the level of detail in which the economic analysis was set out in it. The report included a description of the structure of the model and the rationale for the inputs and assumptions that were used. It also set out the results of the sensitivity analyses which had been undertaken to check the robustness of the model. It explained the limitations of the model and gave the rationale for any simplifications made. It considered the models provided by the manufacturers, including a comparison of the manufacturers' cost-effectiveness analyses with the results obtained by running SHTAC's alternative input assumptions on the same models.
48. We were also taken to Eisai's initial response to the consultation, dated 4 October 2004, at a time when Eisai had not even asked for the read-only version of the model (a point to which I will come back later). That response included a section on economic analysis, in which Eisai submitted first that the use of QALYs in relation to Alzheimer's was entirely inappropriate, but went on to challenge a number of the assumptions made in the model. Similarly, a letter from Eisai dated 21 January 2005 (still before Eisai had asked for the read-only version) responded to NICE's Technical Report No.1, issued earlier that month, on the work done using an augmented base case. The letter stated that Eisai had consulted Professor Knapp, and the representations in it included comments on the sensitivity analyses carried out by NICE, as well as reiterating Eisai's concern that the use of QALYs was inappropriate. A further letter, dated 22 March 2005 (by which time Eisai had requested and received a copy of the read-only model and had also requested the fully executable version), responded in detail to the Appraisal Consultation Document issued in February 2005. The response included a report from a Dr Towse which the judge described as a detailed and comprehensive critique of the model, with recommendations for improvement, including the type of sensitivity analyses needed to be carried out. Mr Giffin submitted that all this material showed that Eisai was perfectly able to make an intelligent response to the proposal on which it was being consulted.
49. I accept that Eisai was given a great deal of information and was able to make representations of substance. It knew the assumptions that were being applied and could comment on them. It knew what sensitivity analyses had been run and could make comments on those. It could and did make an intelligent response, as far as it went. In my judgment, however, none of that meets the point that it was limited in what it could do to check and comment on the reliability of the model itself.
50. Mr Giffin sought to downplay the consequences of not having the fully executable version of the model. As to the lack of an automatic tracking function, he relied on the evidence, to which I have already referred, that manual checking was possible. As to the inability to apply alternative assumptions and carry out sensitivity analyses, he pointed out that it was open to consultees, if they thought that an alternative assumption should be applied or that the model might be unduly sensitive to changes in one or more variables, to make a case to that effect to NICE, which would itself be in a position to run the model on the alternative assumptions or to carry out further sensitivity tests. Again, however, that does not seem to me to counter the force of Mr Pannick's argument. The problem is that, without the fully executable model, a consultee cannot check whether there are variables to which the model is particularly sensitive and make informed representations accordingly. As Mr Pannick put it graphically in reply, the consultee is left making shots in the dark, in circumstances where the light could so easily be switched on.
51. Mr Giffin pointed to the fact that Eisai did not even take up the offer of the read-only model when that offer was originally made, but delayed until February 2005 before doing so. He suggested that this cast doubt on the genuineness of the complaint subsequently made. He also pointed out that in Eisai's March 2005 response to the Appraisal Consultation Document – a response which included Dr Towse's report with its detailed critique of the model and recommendations for improvement – there was no suggestion that Eisai was unable to address the relevant issues in the absence of a fully executable version. Eisai's position on this, as explained in the witness statement of Dr Hooper, was that it did not believe that NICE would rely on the model: it expected to succeed in its submissions that the use of QALYs was inappropriate and that the model should not be used at all. It was only on receipt of the Appraisal Consultation Document that it became clear that NICE did intend to proceed on the basis of the model and that Eisai needed to address it; and at that point, therefore, Eisai asked for the fully executable version of the model. Whilst I do not find that explanation wholly convincing, I bear in mind that the evidence was not challenged in cross-examination (for good reason, since the case was lengthy enough already), and I do not think that the delay in requesting the model can properly be relied on as a ground for rejecting the genuineness of the complaint advanced in these proceedings.
52. Mr Giffin relied in addition on the fact that none of the consultees apart from Eisai asked for a fully executable version or took the point before the Appeal Panel. Even Shire, although intervening in support of Eisai in these proceedings, did not seek the fully executable version, which is said to be all the more significant when one bears in mind that the expert advising Shire at the relevant time was the same Professor Caro as has given evidence on behalf of Eisai in these proceedings (and to whose evidence I have referred at some length above). Mr Kennelly, on the basis of instructions, sought to explain Shire's stance in the matter, referring to NICE's clear policy not to release the fully executable version and to Shire's belief that it was more likely to persuade with other points than by fighting on this issue. The suggestion that Shire might not have the enthusiasm for a fight with NICE is belied by its participation in the present proceedings, but I think there is some force in the point that consultees were aware of the policy to provide only a read-only version of the model and may not have thought it worthwhile to go down the route of seeking the fully executable version. In any event, only a small number of consultees could be expected to have the resources to carry out thorough checks on the model or, therefore, even to think of asking for a fully executable version. Further, even though other consultees did not request the fully executable version at the time, it is clear from the ABPI's written intervention that some pharmaceutical companies do want access to the fully executable versions and feel sufficiently strongly about it to have prompted ABPI to intervene.
53. Accordingly, despite Mr Giffin's effective forensic deployment of the relevant history, I am not persuaded that it should cause the court to doubt the substance of the complaint now made. That Eisai has a genuine concern and is not raising the issue simply as an opportunistic ground of challenge to the guidance is further supported by the quality of its expert evidence and by Mr Pannick's indication that, if Eisai succeeds on the ground of procedural unfairness, it will not seek to have the existing guidance quashed, provided it is given the opportunity to comment on the fully executable version and have those comments taken into account in a fresh decision.
54. I have concentrated so far on the reasons why Eisai says that disclosure of the fully executable version is necessary in the interests of fairness, and why NICE says that it is not necessary. I turn now to examine the reasons why NICE positively opposes disclosure.
55. The first reason is confidentiality. Models are commissioned from academic centres by the Secretary of State and are supplied to NICE on the basis set out in a letter of 2 November 2005 from the NHS's Health Technology Assessment Programme Director:

"(a) The IPR for all models rests with the institution of the author/s

(b) That while NICE may receive an executable model, it may not transmit that model to anyone else without the explicit consent of the holder of the IPR.

(c) That NICE may transmit a non-executable model to consultees subject to the usual conditions …"

Professor Sally Davies, Director General of Research and Development at the Department of Health, says in her witness statement that "economic models are commissioned on the basis that the executable version would not be disclosed". In his witness statement, Professor Andrew Clegg, a professor at Southampton University and Director of SHTAC, expresses the opinion that economic models are confidential, states that it was the University's expectation that the executable model would not be released to consultees, and confirms that "beyond the already agreed release of models to NICE in confidence, and of the non-executable model to consultees on the terms detailed above, the University did not agree to release of the model". He concludes by saying that the University considers that its agreement to release is necessary.

56. That all goes to explain why, as a practical matter, NICE treats the model as subject to obligations of confidentiality and is unwilling for that reason to volunteer disclosure of the fully executable version. In legal terms, however, the explanation given does not withstand scrutiny. Mr Pannick took us to the research contract entered into between the Secretary of State for Health (defined as "the Authority") and the University of Southampton ("the Contractor") pursuant to which the relevant work (falling within the definition of "Research") was carried out. Clause 2.1 provides that "Research commissioned by the Authority is open and, subject to the provisions of the agreement, details of Research are normally published"; and by clause 2.2 the Authority may publish details of the "Project" (defined as the Research together with any data or information generated by the Project) for any non-commercial purpose. Under the heading "Confidentiality", clause 7.1 imposes restrictions on the release by the Contractor, without the prior written consent of the Authority, of information relating to the Agreement or the Project. Clause 7.2 provides:

"Each Party shall at all times use its best endeavours to keep confidential (and to procure that its employees and agents shall keep confidential) any confidential information which it or they may acquire in relation to the business and affairs of the other Party to this Agreement and shall not use or disclose such information except with the written consent of that other Party or in accordance with the order of a court of competent jurisdiction."

The duty of confidentiality in respect of the Contractor's business and affairs is the only express obligation of confidentiality on the Secretary of State. There is no express duty of confidentiality restricting the use or disclosure of the model; and there is no reason why, in the circumstances, such an obligation might be implied so as to prevent the disclosure that Eisai seeks. It would, as Mr Pannick submitted, be very surprising if a model commissioned and paid for by the Secretary of State for the purposes of NICE's appraisal process were subject to obligations of confidentiality preventing disclosure of the fully executable version to consultees.

57. Moreover, the read-only version is disclosed by NICE as a matter of course, subject to standard undertakings as to confidentiality (which Eisai is equally willing to give in respect of the fully executable version). NICE's own position in these proceedings is that all the relevant information is to be found in that read-only version: it is said that the formulae can be checked manually and that reverse engineering is possible (though not permitted). If provision of the read-only version gives rise to no issue of confidentiality, and no additional information is to be found in the fully executable version, it is difficult to see how release of the fully executable version can be subject to objection on grounds of confidentiality.
58. In any event, there are occasions in the past when NICE has disclosed the fully executable version, with the consent of the relevant assessment group; and there is nothing to show that, if it had wanted to disclose that version (on appropriate undertakings) during this appraisal and had sought the consent of SHTAC for the purpose, such consent would not have been given. The evidence is that SHTAC did not in fact agree to disclosure; but it is not said that SHTAC would have withheld its agreement (and it is difficult to see why it should have done so) if it had been pressed on the point by NICE.
59. Accordingly, the argument concerning confidentiality is not one to which I would attach any weight. It should not in my view have a material effect on the court's decision as to whether procedural fairness requires the fully executable version to be disclosed to consultees. I should add, though I do not think it arises, that even if disclosure were prima facie a breach of confidence, Mr Giffin conceded that NICE would have a public interest defence available to it if disclosure were necessary in order to meet the requirements of procedural fairness.
60. There was at one stage a separate argument concerning copyright, but it was not pursued before Dobbs J or on the appeal to this court and I need say nothing about it.
61. The other matter relied on by Mr Giffin is that provision of the fully executable version to consultees will have adverse practical consequences for the appraisal process. I have touched on this already at para 19 above. In the witness statement of Professor Karl Claxton, one of the members of the Appraisal Committee, it is said that release of a fully executable version would have a profoundly counter-productive effect on NICE appraisals. Professor Claxton states:

"32. In my opinion manufacturers in particular want to have access to, and to use, the assessment group's executable models so that they can run this process in reverse. I believe that they wish to begin with a favourable cost effectiveness figure that they believe NICE will accept, in other words, with the output, and then reverse the model and work backwards to see what combinations of inputs and assumptions will deliver that outcome. Unlike their claims for consultation and error checking, this truly is something for which they must have the executable model (and indeed is the only reason which I can see why they would need such a model, and hence my belief as to their motivations). The overall intention, of course, is to maximise profit from sales to the NHS. That is a legitimate aim for a commercial company, but it is not an aim which NICE is required to facilitate.

33. Clearly, 'mining' a model in this way may be desirable commercially but it has no intellectual validity at all. Starting from the conclusion you want to reach and working backwards is the exact opposite of the scientific approach. For the purposes of understanding the clinical and cost effectiveness of a treatment it would be a complete waste of time.

34. However, the results of such investigations would not be presented to NICE in this way. Instead NICE would have to devote a great deal of time and resource to understanding the various amendments made to its model, and to investigating the claimed basis for whatever inputs and adjustments it was that gave a supposedly cost effective outcome. Given that consultees can already advance whatever views of the evidence they wish, and that, so long as these are at least superficially credible they will be investigated by the NICE (as they were in this case), it would add nothing except cost and delay to allow the assessment group economic model to be used in this way."

62. The last point is expressed even more strongly in the evidence of Mr Boysen, who says this:

"127. As the Claimant's witnesses correctly assert, the analysis of an economic model is not a trivial task. It requires time, expertise and money. In a world where resources were unlimited this would not matter, but NICE's resources are finite. It is one thing to devote time and resource to understanding the manufacturers' own approaches to economic modelling. It would be quite another to devote yet further resources to unpicking manufacturers' modifications to the assessment group modelling …. In my view, it would not be practicable for NICE to rigorously analyse all analyses that would be submitted by consultees, if NICE allowed for an executable model to be used for that purpose. The appraisal process would simply grind to a halt."

63. In his oral submissions, Mr Giffin did not go so far as to contend that the process would grind to a halt; but he did go over the difficulties of the task that NICE would face, submitting that disclosure would have the consequence of adding 2 to 3 months to the appraisal process. He argued that the practical implications of what Eisai is seeking can and should be taken into account in determining whether fairness requires such a course to be adopted, and that they should weigh heavily against it.
64. The concern raised in NICE's evidence is a serious one, to which weight should in my view be given. Its force is weakened by the fact that NICE's chief executive made no mention of it in his evidence to the Appeal Panel and, on the contrary, suggested that NICE itself had no objection to release of the fully executable version (see paras 18-19 above). Unlike Dobbs J, however, I would not decline to have regard to the further evidence placed before the court on the ground that it advances "new reasons". It seems to me that it expands on a point that was raised, albeit briefly, in the written material before the Appeal Panel. In any event if, as I have no reason to doubt, this is a genuine concern of NICE, it would be wrong for the court to close its eyes to the point when considering the issue of procedural fairness.
65. On the other hand, for reasons already given, I do not accept Professor Claxton's view that there is no valid reason for wanting the fully executable version: it is needed for the purpose of checking the reliability of the model. The fact that it would also enable the manufacturers to work backwards so as to find inputs and assumptions that would produce an acceptable outcome does not seem to me to be a fatal objection: NICE has to reach a decision in any event on what the appropriate inputs and assumptions are. As to the concern about additional work and delay, the problem would be mitigated to some extent, as Mr Pannick pointed out, by the removal of a degree of inefficiency in the present process, whereby NICE has to consider suggestions made by consultees even though they are shots in the dark and the work done by NICE in relation to them may prove to be a complete waste of time and effort: to that extent, it should be more efficient to let consultees do the work of testing themselves and focus their submissions on points that, in the light of that work, are considered to be of real substance. Further, as Mr Pannick also pointed out, it would be open to NICE to impose reasonable conditions on the release of the fully executable version, for example by requiring sensitivity test results to be presented in a specified manner or limiting the scale of submissions or laying down an appropriate time limit. But even if one accepts the possibility that release of the fully executable version would add 2 to 3 months to the appraisal process, that has to be viewed in the context of an already lengthy process (this one took almost 2½ years in total). I do not think that either the additional time or the additional cost to NICE should weigh heavily in the balance in deciding whether fairness requires release of the fully executable version. If fairness otherwise requires release of the fully executable version, the court should in my view be very slow to allow administrative considerations of this kind to stand in the way of its release.
66. Pulling the various strands together, I can express my conclusion briefly. The view I have come to is that, notwithstanding NICE's considered position to the contrary (to which in itself I am prepared to give some weight), procedural fairness does require release of the fully executable version of the model. It is true that there is already a remarkable degree of disclosure and of transparency in the consultation process; but that cuts both ways, because it also serves to underline the nature and importance of the exercise being carried out. The refusal to release the fully executable version of the model stands out as the one exception to the principle of openness and transparency that NICE has acknowledged as appropriate in this context. It does place consultees (or at least a sub-set of them, since it is mainly the pharmaceutical companies which are likely to be affected by this in practice) at a significant disadvantage in challenging the reliability of the model. In that respect it limits their ability to make an intelligent response on something that is central to the appraisal process. The reasons put forward for refusal to release the fully executable version are in part unsound and are in any event of insufficient weight to justify NICE's position.
67. Strictly, I should express myself in the past tense, by reference to the particular circumstances of the appraisal process to which the challenge relates. The reality, however, is that much of the argument has been advanced in wider terms, by reference to NICE's general policy and the general implications for its work of appraisal if it is required to release fully executable versions of its economic models to consultees. I have borne firmly in mind, in reaching my conclusion, the wider significance of the court's decision in this case.
68. For the reasons given, therefore, I respectfully differ from the conclusion reached by Dobbs J on this issue.

Relief

69. That brings me to the question of relief. It is at this point that I must consider Mr Giffin's submission that Eisai should be refused relief under s.31(6)(b) of the Supreme Court Act 1981 on grounds of its delay in applying for judicial review. The argument is that the refusal to supply the fully executable version was the subject of a clear decision by NICE which was capable of being challenged at the time; yet Eisai waited some 18 months, until the end of the appraisal process, before mounting its challenge. There was a failure to apply within the time limit laid down in CPR 54.5(1) ("promptly and in any event not later than 3 months after the grounds to make the claim first arose"), and there was therefore "undue delay" within the meaning of the statute. Had a prompt challenge been made, the court would have entertained it at that time, rather than allowing the appraisal process to continue for over a year in circumstances of doubt as to its lawfulness.
70. I do not accept that the court would have viewed an early challenge in that way. It is more likely that such a challenge would have been considered premature and inappropriate. At the time when NICE refused to release the fully executable version, it was uncertain what the outcome of the appraisal process would be. The Final Appraisal Determination might have proved to be acceptable to Eisai, in which case the issue concerning release of the fully executable version would have been academic. Further, and very importantly, Eisai had a right of appeal to the Appeal Panel against that determination, and the grounds on which such an appeal lay included procedural unfairness. That might well have been viewed as providing an appropriate alternative remedy, rendering a judicial review challenge inappropriate at that stage.
71. Even if there had been undue delay in applying for judicial review, it would only be a factor to be taken into account in the exercise of the court's discretion as to the grant of relief; and in the light of the matters set out below it would in my view be of no materiality.
72. Mr Pannick made clear that Eisai does not seek to have the existing guidance quashed if that can be avoided. It simply wants the fully executable version to be released to it and to have an opportunity to make representations on it, with a view to NICE making a further determination in the light of any such representations and with a right of appeal to the Appeal Panel if the further determination is adverse to Eisai. If that course can be achieved, as may well be possible through the provision of an appropriate undertaking by NICE, then it seems to me to be an eminently sensible one.
73. Mr Pannick also indicated that Eisai would want the court to make a declaration. For my part, I doubt whether a formal declaration would be appropriate. The judgment will speak for itself.
74. In the circumstances, however, the whole issue of relief is best deferred until the parties have had an opportunity to consider the judgment. An agreed solution may then be forthcoming. If not, the parties can make further submissions on the appropriate form of relief.
75. Subject to consideration of the precise terms of the relief to be granted, I would allow Eisai's appeal.

The cross-appeal on costs

76. The conclusion I have reached on Eisai's appeal makes it unnecessary for me to deal with the cross-appeal by NICE against the costs order made by Dobbs J. It is fair to indicate that, but for Eisai's success on the issue of procedural fairness, I would have been receptive to the cross-appeal: in my view the appropriate order in the court below, on the basis of the findings made by the judge, would have been no order as to costs or a limited costs order in NICE's favour. As it is, however, it will be necessary to reconsider costs below in the light of the success of Eisai's appeal. If the parties are unable to agree an order as to the costs in this court and the court below, I would invite further written submissions on the issue.

Lord Justice Jacob :

77. I agree.

Lord Justice Tuckey :

78. I also agree.