EXPERTS:YOU'RE NOT RIGHT JUST BECAUSE YOU'RE FAMOUS: A DECISION NOT WRITTEN ON THE BACK OF A FAG PACKET

May 19, 2016 · by · in Civil evidence, Expert evidence, Experts, Uncategorized

The judgment of Mr Justice Green  in British American Tobacco (UK) Limited -v- Secretary of State for Health [2016] EWHC 1169 is 1,000 paragraphs long. It covers many aspects of law and procedure. I want to concentrate upon one issue.  There are some interesting and important observations on experts. In particular expert independence , research methods and disclosure.

“It was submitted to me that the experts instructed by the tobacco companies were highly skilled and experienced professionals. Some of the work that they have produced for the purpose of this litigation (and in particular the empirical work) is indeed extraordinarily sophisticated. However, as was observed in the US Courts the simple fact that an expert has a high pedigree or is a Harvard professor or a Nobel Prize winner is not a reason not to apply to their work exactly the same rigorous standards as are applied to the work of others. The report of a Nobel Prize winner as presented to a Court might be a remarkably good piece of work but if it lacks peer review or ignores contradictory internal documents or is unverifiable, its probative value may nonetheless be substantially diminished. Nobel Prize winners should in any event be strong adherents of the very highest of international research best standards; and if they fail to live up to these standards a Court must say so and act accordingly.”

 

THE CASE

A number of tobacco companies brought an action seeking to challenge regulations that imposed specific wording on cigarette packets.  One of the issue was the validity of expert evidence and the independence of that evidence.

THE JUDGMENT (SELECTED EXTRACTS ON EXPERT EVIDENCE)

(1) The issue
  1. The Claimants contend that the Regulations are unlawful because generally only “limited” weight was accorded to the Claimants’ evidence submitted during the consultation exercise. This was an argument advanced primarily by BAT in relation to its own evidence but the general thrust of the submission was adopted by all the Claimants. There is a closely related but subsidiary aspect to the ground raised only by BAT which is thatits own evidence (i.e. irrespective of whether the Court rejected the submission in relation to the generality of the tobacco company evidence), and especially its expert evidence, was not accorded proper or fair weight. In respect of both the primary general argument and the secondary specific argument the Claimants contend that their evidence was of the very highest quality and integrity and demanded to be accorded great weight. The issue arises because the Defendant’s Chief economist, Mr Derbyshire, stated in evidence that “limited weight” was attached to the expert evidence of the Claimants during the consultation exercise because it, in effect, fell short of methodological best practice.
  2. This stance is thus a discrete ground of challenge in these proceedings relating to the legality of the consultation process. But the point also resonates in the context of the other freestanding grounds which involve a consideration of the Claimants’ evidence such as proportionality and in the context of alleged violations of property rights. Professor Hammond, an expert instructed on behalf of the Secretary of State, conducted a comprehensive analysis of the expert evidence tendered by the Claimants both during the consultation and for the purposes of the litigation from the perspective of adherence to methodological best practice. His conclusions are that in myriad and material ways the evidence falls short and as such was not and is not entitled to be accorded significant weight. He attacks the Claimants’ experts for, inter alia: lacking independence; failing to peer review their work or rely upon the peer reviewed and/or independent research work of others; ignoring the weight of pre-existing research on the topic; failing to benchmark or verify their conclusions against the internal documents generated by the tobacco companies; and, setting up impossibly high and unrealistic standards against which to measure the justifications for the Regulations.
  3. The Claimants have retaliated. They have launched an attack upon the independence of the experts relied upon by the Secretary of State complaining that they are biased because they adhere to the “tobacco control” lobby and, for instance in relation to Professor Hammond, accept substantial research grant money from tobacco control interests. In their written submissions on proportionality in relation to the Pechey Elicitation review (see paragraphs [139] – [142] above) they refer to: “The Pechey Study, which consisted of asking 33 anonymous (but far from impartial) “experts” in anti-tobacco research for their “best guess estimates” of the likely impact of standardised packaging over a two year period from which an average prediction was calculated (the “Pechey Estimate”)“. The Claimants have also adduced expert opinion evidence which attacks the methodological “best practice” rules for research that are held up as the appropriate benchmark by the Secretary of State. For instance, they put forward expert opinion evidence from a Professor Klick which, if correct, suggests that, more or less, the total evidence base generated over the past few decades by scientists and researchers (worldwide) on the issue of the effect of advertising and promotion on the behavioural patterns of actual and potential tobacco consumers is wrongheaded, irrelevant and based upon flawed thinking and techniques.
  4. The issue is thus live and hotly contested. I address it below (a) in order to analyse the specific (BAT led) complaint about the consultation process but also (b) because it is important in analysing the Claimants’ evidence on other grounds.
  5. The question of the intrinsic quality of the evidence is a fundamental one, not least because of Article 5(3) FCTC and the WHO guidelines (see paragraphs [168] – [175] above) to the effect that the tobacco industry should be treated as having adopted a deliberate policy of subverting public health policy through, inter alia, the deployment of its substantial capital and organisational resources to generate evidence designed to contradict the established policy consensus. The premise behind Article 5(3) FCTC is that, to put the point in an unvarnished form, this evidence is unreliable, i.e. false.
  6. The present litigation has thrown into sharp relief all of these controversies and the issue for this Court is how they translate into practical rules of evidence which can be applied to the facts of this case. The legal framework for this ground takes as one point of reference Article 5(3) FCTC and the WHO guidance. But it also engages (i) with ordinary common law principles of evidence and how a Court, in the context of a judicial review, should evaluate expert and other evidence; and (ii) with the effect of CPR 35 on expert evidence.
(2) The basic methodological principles
  1. I start by setting out the basic methodological principles which the Secretary of State put forward as, in effect, internationally recognised best practice and which he submits should be used in the evaluation of the evidence in this case. He cites numerous literature sources for these principles and, moreover, characterise them as common sense quality controls to be applied during a consultation but also during Court proceedings.
(i) Independence & bias / conflict of interest
  1. The importance of independence is obvious: a researcher who has no affiliation which could give rise to a conflict of interest is less likely to be subject to bias. Independence can be compromised by any sort of financial relationship with a person or party who seeks a particular result. This can extend from the provision of research funding to fees for the preparation of expert reports. But bias can arise from less overt and far more subtle sources. So for instance academics have long recognised the concept of “confirmation bias” which is said to arise when a decision maker seeks only to collect or give credit to evidence which leads to (the confirmation of) a particular preferred result. Such confirmation bias can be subconscious; it need not indicate a deliberate intent to distort an evidence collection or decision making process. It is sometimes said to arise as a risk when an administrative decision is quashed by a Court and remitted to a decision maker to be retaken; in such circumstances there is a risk that the decision maker will set out with determination to take the same decision again but this time curing the defect that led to the adverse judicial decision. In an appropriate case good administration can mean that a new team of officials is instructed to take the remitted decision with a clear instruction to apply a fresh mind to the issue.
  2. Independence is not an absolute requirement; in normal litigation where expert evidence is required experts are instructed by parties and they may be very well paid to present an opinion to the Court. The quality of that evidence cannot be automatically discounted simply because it is advanced on behalf of those who are parti pris. The same will apply to evidence submitted in the course of a consultative process. Consultees advance their point of view and frequently support that with expert evidence. Expert opinion evidence is thus submitted in a multiplicity of different circumstances ranging from consultations through regulatory proceedings to litigation. A common question therefore is how such opinion evidence is to be evaluated especially given that in the context of scientific and technical research (much of which is conducted by academics with no compromising affiliations) a premium is placed upon independence.
  3. The problems associated with a lack of independence can be overcome. Where there is full disclosure of the facts giving rise to the actual or perceived lack of independence those who subsequently come to read or rely upon the research output can evaluate the research through the optic of possible bias and predisposition. The more acute the possible bias the more extensive might need to be the extent of the disclosure. A researcher who receives a research grant from an interested party which served only to defray research costs may be in a different position to an individual who stands personally to gain through the receipt of a substantial fee. Disclosure of the nature and extent of the interest may therefore be important but it is not a complete answer. Sometimes the expert evidence might concern a subject matter which is of great complexity and which is, thereby, exceedingly difficult for a Court or decision maker to unravel so that the mere fact that the author has declared an interest does not equip a reader with the tools needed to determine whether in actual fact the research output is affected or distorted by that declared interest. This is certainly true of the research which is before this Court, whether it derives from pre-existing literature sources or from the new research conducted by experts instructed by the parties, such as the regression analyses. In the present case both sides accuse the other of bias or predisposition. Chantler rejected the suggestion that tobacco control experts were biased as “absurd” (Chantler Report paragraph [6.9]; see paragraph [113] above). In my view I would not wholly dismiss the proposition that tobacco control experts might, albeit subconsciously, feel so strongly about the correctness of their cause that their opinion might be influenced by that view. It is for this reason that whilst independence is a relevant factor it is not necessarily the determinative factor and adherence to other quality control practices such as peer review and/or benchmarking against internal documents can play an important and possibly crucial role in providing the guarantee that the research output is of the highest quality.
  4. The conclusion that I have arrived at about bias is not dissimilar to the observations of Judge Kessler in the US Judgment (ibid at pages [7] – [9]). She concluded:
“Much of the Defendants’ criticisms of Government witnesses focused on the fact that these witnesses had been long-time, devoted members of “the public health community.” To suggest that they were presenting inaccurate, untruthful, or unreliable testimony because they had spent their professional lives trying to improve the public health of this country is patently absurd. It is equivalent to arguing that all the Defendants’ witnesses were biased, inaccurate, untruthful, and unreliable because the great majority of them had earned enormous amounts of money working and/or consulting for Defendants and other large corporations, and therefore were so devoted to the cause of corporate America that nothing they testified to, even though presented under oath in a court of law, should be believed. Such simplistic attacks on the credibility of the sophisticated and knowledgeable witnesses who testified in this case are foolish.
All of this is not to deny that there were significant differences in the overall qualifications of the Government’s witnesses and the Defendants’ witnesses. There were. The Government’s witnesses, viewed as a whole, were far more experienced, credentialed, and active in the area of smoking and health, whatever their particular area of specialty, than were the Defendants’. Many of the Government experts had participated extensively, over many years, in the long and drawn-out process of ascertaining the consensus of scientific opinions embodied in each Surgeon General’s Report. Virtually every one had taught at a well-regarded academic institution and written numerous peer-reviewed articles in their particular area of specialty. Many of the Government witnesses continued “hands on,” clinical work in their fields despite heavy commitments for research, writing, teaching, and lecturing to their peers.
The Defendants’ witnesses were obviously well educated in their areas of specialty. Indeed, as was mentioned on many occasions, Defendants even presented the testimony of an impressive Nobel Prize winner. However, rarely did these witnesses have the depth and breadth of experience of the Government witnesses. Many had worked only in large corporations, and many for only one or two such employers. Many — although not all — had written relatively few peer-reviewed articles. Many of the highest paid experts of Defendants, while well credentialed in their particular fields, such as economics, presented relatively narrow testimony tailored to the particular problem or issue they were retained to opine on for purposes of this litigation. A few of Defendants’ experts had done virtually no individual research and written virtually no peer-reviewed articles, and a few were unfamiliar with the relevant facts and/or the major scientific literature on the issue about which they testified.
While the testimony of each person — expert or fact witness — was evaluated on its own merits, there can be no denying that, as a group, the Government’s witnesses were far more knowledgeable, experienced, and active in their respective fields”.
(ii) Peer review
  1. Peer review is the process by which an authored work is submitted to the scrutiny of others for constructive criticism. It is a process of intellectual democratisation whereby anyone can access the research and evaluate it. Not infrequently a previously unknown researcher or Ph.D student emerges from a non-mainstream academic institution who manages to puncture the previous orthodoxy and thereby contribute to the debate. The underlying premise is that “sunlight bleaches” – by exposing research results to scrutiny their strengths and deficiencies are highlighted and this not only enables the original researchers to go back and improve the work but it also enables other researchers to build on the peer reviewed platform. The process of peer review is routine in the editorial practices of the better scientific and technical publications. Material that is not peer reviewed will not by definition be of inferior quality but since the practice of peer review is so widespread an absence of peer review may be a legitimate reason for querying the integrity of that research; and even more so if it is deliberately not peer reviewed. The advantages of peer review are obvious: it imposes upon researchers an incentive to ensure that their material is intellectually and evidentially robust; it enables proposals for publication to be criticised and thereby improved; and it ensures that as thinking on an issue evolves it does so with the weight of academic and scientific opinion in support. It is a process which enables concerns relating to an absence of independence to be mitigated. The advantages can be seen by considering how research results would evolve without the process. It would mean that errors or weaknesses or bias in original research risk not being identified and there is correspondingly diminished incentive for researchers to get it right first time around. If research that has not been peer reviewed is then used as a platform for subsequent researchers to build upon it can lead to errors being self-perpetuated. Mr Derbyshire, for the Secretary of State, put the point in the following way:
“…the degree to which the data used and the analyses of it has been independently or widely scrutinised should be considered. Such consideration helps counteract the conflict of interest issues referred to above. The analyst or decision-maker is able to place greater weight on data and analysis that describes transparently what work has been done and any issues arising (such as conflicts), has been peer reviewed and has been published for critique by a wider audience. Wider scrutiny can help ensure all analysis is being considered and there is not selective reporting of favourable findings and non-reporting of unfavourable results. Such scrutinised analysis is more informative than a non-peer-reviewed, opaque analysis seen only by a few people”.
  1. The Claimants reject this analysis. They submit that the fact that their evidence is exposed in litigation and subject to judicial review is a superior process to peer review. I fundamentally disagree. I have, in this litigation, had the opportunity to test the proposition thoroughly. I set out my conclusions in relation to Ground 3 and as to the sort of process that would have to occur to enable a Court adequately to resolve disputes of this type at paragraphs [630] – [648] below. Courts do not have the time or resources to take research away and then spend months unpicking and reverse engineering it so that it can be re-performed using different and improved assumptions, even assuming that the Court has the technical ability to do so. In judicial review the argument might not even focus upon the actual nuts and bolts merits as opposed to issues such as margin of appreciation. In the present case nearly 30 expert reports have been tendered and relied upon, predominantly from the Claimants. For the most part the evidence was simply used in written submissions and as cross references in footnotes to the written submissions and only a modest proportion was in actual fact highlighted during the course of oral argument. These reports however cover an enormous array of different issues and many seek to build upon prior research a significant portion (but not all) of which is not peer reviewed or from independent researchers. It is an almost impossible task for a Court in such circumstances to assess the accuracy of the entirety of a vast body of evidence such as this. In fact this case serves to highlight the importance of the Court having available to it methodological tools, such as research best practice guidelines and principles, with which to assess the evidence.
  2. I give below one illustration of how the process of peer review can result in an iterative and incremental perfection of results. This is found in the exchange between the parties as to the relative reliability of different data sources. In his first report Professor Mulligan (for the Claimants) was critical of the 2014 Impact Assessment for failing to consider data sources that measured smoking prevalence frequently enough to permit a valid comparison of rates of prevalence immediately prior to, and following, the introduction of standardised packaging in Australia. Professor Mulligan relied, in particular, upon two pieces of research by Messrs. Kaul & Wolf[8] (“Kaul & Wolf”). There is evidence that this research was funded by the tobacco industry. In this research the authors sought evidence of an effect brought about by standardised packaging upon smoking prevalence in Australia and found none. Professor Mulligan points out that the researchers modelled the trend of smoking prevalence in Australia prior to introduction of standardised packaging and considered the degree to which prevalence fell faster than that trend following standardised packaging. Professor Mulligan attributed substantial weight to the Kaul & Wolf Reports. He rejected the criticisms made of that work in the 2014 Impact Assessment for an alleged lack of statistical “power”. Professor Mulligan pointed out that Kaul & Wolf did not control for changes in cigarette prices and accordingly even if standardised packaging had exerted no impact upon prevalence one would then have expected Kaul & Wolf to observe a decrease in prevalence beyond trend but since they observed no decrease beyond trend at all this suggested that standardised packaging had an, unanticipated, upward effect upon smoking prevalence.
  3. In his second report Professor Mulligan returned to this theme. He, once again, relied upon Kaul & Wolf to undermine the data sources relied upon in the 2014 Impact Assessment. Other experts instructed by the Claimants also relied upon this same research.
  4. Kaul & Wolf have, however, subsequently been peer reviewed by Diethelm & Farley. These researchers were critical of the conclusions arrived at by Kaul & Wolf. They sought to re-work the data relied upon and concluded that, properly understood, it demonstrated the opposite conclusion to that arrived at by Kaul & Wolf. The episode demonstrates the importance of peer review. It is especially important if a vested interest seeks to rely, and build upon, research that it has funded because rigorous peer review minimises the risk that non-independent research results are perpetrated by other non-peer reviewed researchers with the consequence that a growing body of un-reviewed research gains traction. I have subjected this particular episode to more detailed substantive analysis at paragraphs [619] – [624] below. Ms Demetriou QC, in her closing submissions, then subjected Diethelm & Farley to her own forensic criticism. No doubt, other (independent) researchers may take the work of Diethelm & Farley and subject it to additional review and over time the process of peer review might result in a perfected analysis. If, at the end of the day, it is established that the data relied upon initially by Kaul & Wolf disproves the proposition the Claimants seek to assert then it will do no more than demonstrate the critical importance of a proper peer review process being applied to precisely the sorts of evidence which are in issue in this case. I am not (remotely) in a position to decide who is right and who is wrong. I can, however, conclude from this that the process of peer review is an important one with serious implications for the issues arising in the present litigation.
(iii) Internal documents: The need for corroboration and benchmarking of expert evidence
  1. I turn next to the importance of being able to benchmark the Claimants’ expert opinions against internal documents generated by the Claimants themselves. It has been a striking feature of the evidence adduced by the tobacco companies during the consultation process (and replicated in the Court proceedings) that it is virtually devoid of any reference to the internal documents of the tobacco companies themselves. This has been the subject of litigation in the US. It was one of the prime reasons why in the FCTC the WHO expressed such profound scepticism at the motives of the tobacco companies and as to the reliability of their evidence. Chantler, in his Report, commented adversely upon the fact that the tobacco companies criticised the evidence base relied upon by the tobacco control lobby (which evidence Chantler accepted) but declined to produce the internal focus reports and other analysis which he considered undoubtedly existed. One of the reasons given by the Defendant for attributing limited weight to the Claimants’ expert reports was precisely because of an inability to have confidence in the data and facts which underlay the assumptions and conclusions in the Claimants’ expert evidence and the concern that the outward (expert-led) views of the Claimants would be contradicted or qualified by their internal documents. Mr Derbyshire, the Department of Health Chief Economist and Analyst, explained in evidence to the Court that whilst some of the primary data used by the Claimants’ experts was available to the Department much was not. As such some of the expert analysis submitted was simply not verifiable.
  2. BAT however complained specifically that the Department did not have access to every data source that it should have had access to when the Minister decided to lay draft Regulations before Parliament. In a sector and market where there is a systemic concern about the reliability of data and research submitted by the tobacco industry if those companies and parties do not ensure, when they place research before a decision maker, thattheir research is fully verifiable including, where necessary, disclosure of underlying internal documents (for and against), then this is a factor that the decision maker is entitled to take account of when weighing and evaluating that evidence. If that minimum standard is not adhered to then it cannot subsequently be complained that the decision maker failed to obtain access to all the underlying data and information sources. There is an onus on consultees as well as upon the decision maker. If a consultee provides fully and comprehensively its research and everything underpinning it which would enable it to be checked and verified then the objection that the decision maker has failed to perform a verification (since if it had it would then have attributed greater weight to the evidence) might carry greater conviction; but if a consultee chooses not to do this then a complaint made later that the decision maker erred in not obtaining more data may ring hollow.
(iv) Internal documents: Tobacco companies’ statements to the High Court
  1. In response to my repeated questions to the tobacco companies during the hearing (and in view of the Secretary of State’s sustained criticism of the Claimants’ evidence) as to the nature and extent of their internal assessment of the issues arising, JTI and PMI (only) produced carefully crafted written statements.
  2. JTI stated that it had not conducted research on whether the introduction of standardised packaging in the UK would or would likely discourage children in the UK from taking up smoking. It stated also that this was in conformity with its “global position”. I set out verbatim JTI’s position to the Court:
“(a) JTI does not market its products to minors;
(b) JTI does not market its products in order to encourage anyone to take up smoking or to discourage anyone from quitting;
(c) JTI does not conduct market research involving or in relation to minors;
(d) JTI has no interest in information about minor’s consumption of tobacco products; and
(e) JTI does not seek, collect or accept marketing data about minors, or analyse general data to learn about minors”.
  1. I find this statement remarkable. The evidence from the Secretary of State (which is not disputed by the Claimants and indeed was reflected in the evidence of JTI’s own experts[9]) is that the vast majority of smokers take up smoking before they are in their early 20s and most before they are 18 years of age. Youth smoking is critical to the future of the tobacco companies. 600 children per day between the ages of 11 and 15 start smoking in the United Kingdom (see paragraph [63] above). Children can quickly become addicted. These children become the long term customers of the future and then replenish the customer base depleted by adult quitters. Yet, JTI states as a component part of its global strategy and as a specific reason for not having internal documents: “JTI has no interest in information about minor’s consumption of tobacco products”. How, rhetorically one asks, can JTI have “no interest” in information about consumption of tobacco by children? Growth in sales depends upon the advent of youth smoking which is an enormous social and future health problem. And what sort of Nelsonian knowledge is reflected in the global strategy of a company that is not only disinterested but also refuses to even “accept” (see paragraph (e) of the JTI position (supra)) marketing data about children and which has no interest in even learning about children?
  2. In relation to this formal statement the Secretary of State expresses profound scepticism. He submitted: (i) that these statements were not supported by statements of truth and witness statements from senior management (though a short statement was provided during the hearing); (ii) that no other tobacco company claimed to lack internal documents of this type; (iii) and that the statements were in any event inconsistent with statements made elsewhere in JTI’s submissions that it “continuously conducts analysis in respect of brand equity and downtrading … based on the current package of its brands in the UK“. From this the Secretary of State submitted that even if this statement in JTI’s submissions reflected the full extent of the internal research conducted by JTI it inevitably follows that JTI did in fact hold relevant internal documentation about the impact of its currentpackaging on smoking habits, which is a very relevant issue; and also, by applying the same logic, it must have carried out analysis in respect of the impact of its current packaging (which is plain packaging) in Australia. The statement is also inconsistent with the findings of fact made by Judge Kessler about the parties in that case in the US Judgment.
  3. PMI was the other Claimant company to submit a statement of its position. It accepted that it was a matter of record that although prevalence and consumption had been declining for years its profits had been increasing. It submitted: “What matters for PMI is its market share and profit margin; not aggregate consumption or prevalence“. PMI set out the following written statement during the hearing:
“As far as those instructing us and our solicitors are aware, neither the PMI Claimants nor the Australian business of the Philip Morris international, Inc group has conducted any detailed assessment of the impact of SP in Australia on prevalence or consumption. Nor did PMI produce any business plan or other documents which carried out a detailed assessment of the predicted impact of SP on prevalence and consumptions in the UK. Indeed PMI does not generally conduct analysis of the impact of regulatory measures on prevalence or consumption. Its analyses are directed towards the performance of its own brands”.

DOMESTIC CIVIL PROCEDURE RULES

(vii) Internal documents: Domestic civil procedure rules / CPR 35.
  1. The domestic civil procedural rules in this jurisdiction require experts to adopt a balanced approach to the evidence and to take account of points both for and against. They may not act as hired guns. From the above citation from the US Judgment which refers to a “mountain” of “overwhelming” evidence, the reference to the internal documents of the tobacco companies showing how they conducted extensive internal analysis of the impact on children in paragraph [3298] is significant. The documents relied upon in the US litigation were held to reflect the current position of the tobacco companies as of the date of the judgment, 2006. It has not been suggested by the tobacco companies in this litigation that these documents are inaccurate or no longer reflective of their actual, internal, policies and positions or that they no longer reflect the reality of the tobacco market (though see paragraphs [294] – [297] above). The companies seek to contend that they do not target children through their advertising and promotional policies. The Secretary of State challenges this. He submits that protestations made in Court without the backing of internal disclosure is unacceptable and unconvincing. He points out that, as Judge Kessler found as a fact, disclosed documents from the US show that targeting youth has been a major and enduring plank in the marketing strategies of tobacco companies and that in curtailing branding the Regulations will serve to prevent adverse impacts on children. He cites as illustrative only a 1984 Strategic document from RJ Reynolds which states:
“Younger adult smokers have been the critical factor in the growth and decline of every major brand and company over the last 50 years. They will continue to be just as important to brands/companies in the future for two simple reasons: The renewal of the market stems almost entirely from 18-year old smokers. No more than 5% of smokers start after age 24. The brand loyalty of 18-year old smokers far outweighs any tendency to switch with age”.[10]
  1. Professor Hammond cites an extensive list of research analysis which describes a causal connection between tobacco marketing and youth smoking[11]. In the Report he prepared in the context of the consultation in Ireland (the Hammond Ireland Report) he stated:
“A wide range of industry documents highlight the importance of tobacco marketing targeted at youth. A published review[12] of tobacco company documents concluded:

“Industry documents show that the cigarette manufacturers carefully monitored the smoking habits of teenagers over the past several decades. Candid quotes from industry executives refer to youth as a source of sales and as fundamental to the survival of the tobacco industry. The documents reveal that the features of the cigarette brands (that is, use of filter, low tar, bland taste, etc), packaging (that is, size, colour and design), and advertising (that is, media placements and themes and imagery) were developed specifically to appeal to new smoker (that is, teenagers). Evidence also indicates that relevant youth orientated marketing documents may have been destroyed and that the language used in some of the more recent documents may have been sanitised to cover up efforts to market to youth””.

  1. Yet the Claimants submitted in these proceedings that internal documents were not relevant to the tasks the experts were instructed to perform. I do not agree. For instance the Claimants’ experts addressed and were profoundly damning of the pre-existing literature and research base which indicated that branding and advertising were causative of changes in consumer behaviour and, it could logically be inferred therefore, that restrictions upon such branding and promotion would equally also affect consumer behaviour (i.e. away from smoking). The internal disclosed documents suggest that tobacco companies engage in extensive marketing and research into this very issue and that these documents support the Secretary of State’s position. Yet, the Claimants’ external experts who addressed this issue ignore this source of evidence. Professor Devinney was instructed to examine the consumer research evidence which was relied on by the Defendant in introducing the Regulations. Professor Steinberg was instructed to examine whether the Regulations were likely to discourage children from taking up smoking. The Secretary of State submitted in relation to these two examples that the best, and possibly the only, way properly to test the Defendant’s evidence or to assess the likely impact of standardised packaging on smoking initiation in children: “… is by reference to evidence concerning the impact of tobacco marketing and branded packaging on smoking behaviour generally. On any view tobacco industry documents which address the impact of packaging and marketing on smoking behaviour must be relevant to the questions being addressed by JTI’s experts. Professor Steinberg’s assertion that such evidence is “irrelevant” because most forms of tobacco packaging have been prohibited in the UK is patently absurd in light of the fact that tobacco marketing on packaging and cigarettes has (until the coming into force of these Regulations) always been permitted in the UK“.
  2. Professor McKeganey conducts a review of some of the main pieces of research literature in a report dated 5th August 2014. It was prepared on behalf of BAT for the purposes of the consultation process. But it was, as with all of BAT’s other evidence used in that process, also tendered as relevant to the broader issues arising in the litigation and was therefore relied upon for grounds which went beyond the limited objections relating to the consultation process itself. But (as with so much of the other evidence) it was not updated nor was it subject to the normal and requisite expert declaration under CPR 35. Professor McKeganey adopted the position that at times researchers who generated results in favour of tobacco control were biased and that this was a real obstacle to providing a clear assessment of the evidence results they generated. In this he distinguished between the “natural and inevitable divergences of opinion on the part of different experts“, and bias. He set out his view that tobacco research had become as much a “moral activity as an investigative one, a weapon used by the “researcher-activist” in the “fight against tobacco“”. The upshot of this was that, in substance, a Court should discount the evidence advanced in favour of tobacco control; it was unreliable and biased:
“What this had meant in practice is that the principles of transparency and openness, for so long foundational tablets of the scientific enterprise, have on occasion been sacrificed in what is seen as an academic and political war to be fought against the tobacco industry”.
Professor McKeganey then goes on to review various pieces of research literature and he concludes with an overarching and central proposition that decisions to start smoking during adolescence are related to factors other than those related to packaging and design of tobacco products. He argues that the appeal of branded packaging is not empirically supported as a factor that significantly increases the likelihood of smoking initiation during childhood and it does not operate significantly to motivate changes in smoking behaviour in adults. In short evidence that supported tobacco control was unreliable because it was subject to moral crusader bias; and the reliable evidence was that which opposed tobacco control. Yet, and once again I repeat the criticism that I make of so many of the Claimants’ experts, his conclusions contradict a very great deal of research by independent peer reviewed researchers, and also the internal research of the tobacco industry, and his conclusions contradict the findings of Judge Kessler who, after a comprehensive nine month trial, found the opposite, and he ignores the adverse conclusions of WHO upon analysis of the tobacco companies’ internal documentation. What I find unacceptable is the preparation of a report which by its total refusal to engage with any of this contra-material simply conveys the impression that it does not exist and that the best way to refute it is to ignore it. Yet, at the same time and inconsistently, Professor McKeganey accepts that the principles of transparency and openness are “foundational tablets of the scientific enterprise“. Had Professor McKeganey confronted head-on the contrary evidence, including that from the tobacco companies, then it is hard to see how he could have advanced the opinions that he did; at the very least he would have been compelled to provide a proper rationale for why his opinion could be sustained in the light of this inconsistent evidence. Further analysis of this evidence is set out at paragraphs [381] – [383] below.
  1. This point is important in the overall context of this case because to succeed on evidential grounds the Claimants must not only establish that their own, new, post-Australian implementation quantitative evidence is powerfully probative, but also refute or oust or at the very least massively discount the probative value of pre-Australian qualitative evidence which is against them. And that substantial task will not be achieved by reports from experts, howsoever distinguished, which simply fail to address the contrary evidence base (which must include internal assessments) and the adverse conclusions on that evidence base by researchers, Courts and international organisations such as the WHO. One is left in the case of Professor McKeganey with an expert report that is prima facie inadmissible because it fails to adhere to domestic civil procedure rules but that, even when fully considered, appears slanted and partial.
  2. Experts owe their primary duty to the Court. Detailed rules governing the conditions under which experts give evidence in this jurisdiction are set out in CPR 35. 35PD2.1 provides that expert evidence should be the independent product of the expert uninfluenced by the pressures of litigation. 35PD 2.3 states: “Experts should consider all material facts, including those which might detract from their opinions”. How can an expert consider all material facts including those that are inculpatory to their client if they do not ask for and/or receive relevant internal documentation? Moreover, where experts rely upon data or information selected and provided to them by their clients then the obvious concern arises that the client has felicitously chosen to provide only exculpatory information and data that serves its cause. Where this arises, an expert, in conformity with CPR 35 and PD 2.1, should give careful consideration to explaining in a transparent way in the resultant report what steps if any he or she has taken toensure that the data and information provided is accurate and fairly representative. And an expert might, if he or she suspects that the client has been carefully selective or adopted a policy of destroying internal documents or not recording (in disclosable form or otherwise at all) the substance of sensitive issues, say as much in the final report.
  3. What is conspicuously missing in the present case is evidence based upon the internal documents of the companies in question or any satisfactory explanation as to why this has not been given or in those strictly limited cases where it has been provided in part any account of the efforts made by the expert to obtain satisfaction that the material provided meets the highest standards of fairness and impartiality.
  4. Uniquely in this case there is an international consensus from within the WHO and across the world that tobacco companies are set on subverting national health policies antithetical to their financial interests. This is, in part, due to experiences in the US courts and the sharp conflict between public utterances and private analysis. There is in such circumstances a real premium upon full observance with the principles laid down in the CPR and (in so far as there is day light) with best and transparent research and publication practices generally. It is in this way that the tobacco companies can persuade a systemically sceptical world that their research is valid and worthy of the great probative weight they claim for it.
  5. My concern lies not just with the position of a single Claimant company but, rather, with what has the appearance of being an industry wide practice not to adduce internal documents or to allow their experts to see and review and then rely upon internal documents. The position in this case is quite different to that which arises in typical civil litigation when the experts will prepare their reports following disclosure and taking the disclosure into full account and where the opposing side to the litigation and the Court can evaluate the expert’s report in the light of the disclosure or discovery. Experience tells one that very frequently the best experts of all are the middle and senior management within client companies who live with the issues on a daily basis and understand deeply the dynamics of their own markets. These internal views, in all their unvarnished glory, are routinely the most telling and perspicacious of all….
(iv) Referencing of the existing literature base.
  1. I now move away from internal documents to the next aspect of methodological best practice which concerns the efficacy of researchers and experts addressing the existing literature base. I start with the criticism made by the Secretary of State that the tobacco companies and their experts failed to grapple with the pre-existing evidence base which has been generated over a number of decades by scientists and researchers worldwide. The Claimants’ experts however attack that evidence as fundamentally unsound as a matter of principle, and, in any event, superseded by their up to date regression analysis. The Defendant makes three points which concern: (i) the intrinsic value of the existing literature base which is based upon successive pieces of peer reviewed material; (ii) the evidential value of consistency between the outcome of peer reviewed research; and (iii) evidential problems related to selectivity of prior research
…
(x) The analysis is context sensitive
  1. I must add a caveat to all of the analysis above. I do not suggest that in all cases the above principles will inevitably apply either at all or to the same degree. Every case is fact and context sensitive. So, for instance, in a specific piece of civil litigation where experts are instructed to express an opinion upon, for instance, an issue of alleged clinical negligence it cannot be a sensible criticism that their reports are not peer reviewed. But it may be a valid criticism if the experts fail to study and take into account the contemporaneous clinical notes. Equally, in a consultation where consultees are given (say) 6 weeks only to respond the fact that the material submitted is not peer reviewed may also be a false complaint. On the other hand in both of these instances if the expert fails, for example, to address fairly and comprehensively the underling pre-existing relevant literature/evidence base or the points in the literature against that expert’s opinion that may still be a fair criticism.

RELATED POSTS

 

Tags: ,