| Keynote speakers | ||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||
| List of participants | ||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||
| Agenda | ||||||||||||||||||||||||||||||||||||||||||||
|
Monday 4 July 2011:
Tuesday 5 July 2011:
|
||||||||||||||||||||||||||||||||||||||||||||
| Abstracts | ||||||||||||||||||||||||||||||||||||||||||||
Mark Burgman Status is a poor guide to performance Typically, experts are defined by their qualifications, track record and experience. Society broadly expects experts to have privileged access to knowledge and to use it effectively. We call this the social expectation hypothesis. This study asks experts to predict how they will perform, and how their peers will perform, on sets of questions. It then compares these predictions with actual performance on the questions. The results indicate that experts rank themselves consistently, but ranks are a poor guide to actual performance. Relatively inexperienced and unqualified people perform well, especially if they are given the opportunity to discuss the substance and context of a set of questions. Just as importantly, the group average regularly outperforms and provides more reliable estimates than the best-regarded people in the group. Jan Sprenger The Epistemic Benefits of Deliberation: An Estimation Model We set up a model of deliberation where by means of talking to each other, agents learn to relate their own competence to a given benchmark. We do not require that they be able to judge the competence of other individual agents in the group. Consecutively, we compare straight averaging of individual estimates of an unknown quantity to a weighted average that is informed by the deliberation process. Although that estimate is rather rough and heuristic, it demonstrates the merits of deliberation because it outperforms the straight average according to mean square error. We extend this model in several directions and compare it to other approaches in the literature. Oswaldo Morales Elicitation and Combination of Rank Correlations for Dependece Modeling We discuss the elicitation of rank and conditional rank correlations through conditional probabilities of exceedance and/or ratios of unconditional rank correlations. These techniques have been used in the quantification process of hybrid non-parametric Bayesian Belief Nets in aviation and earth dam safety. The combination of experts dependence estimates is discussed in the context of results from these applications. The methods used are based on techniques previously introduced for elicitation of bivariate dependence.
Concomitant with growing industry interest in nanotechnology for the food sector are concerns about safety and relevant regulatory issues. These are difficult to address due to the lack of knowledge of possible interactions of nanomaterials at the molecular and/or physiological level and their effects on human health either directly or indirectly (Chaudhry et al. 2008). This level of uncertainty requires input of expert judgment, and it is expected that experts’ opinions will vary. The issue in question then becomes: how to capture experts’ current knowledge and uncertainties & to understand how experts use their knowledge when thinking about possible risk of nanotechnology-enabled food products. Our approach employs a multi criteria decision making (MCDM) model based on probabilistic inversion; it enables us to model experts’ judgments regarding safety of such products in terms of scores on 10 criteria (Flari et al., 2011). An advantage of these sample-based techniques is that they provide out-of-sample validation and therefore a robust scientific basis. This validation in turn adds predictive power to the model developed.
A rather large (theoretically infinite) number of low-dose response models for genotoxic carcinogens exist. Different countries and organisations support different approaches. For the moment, regulatory agencies in the USA, and possibly The Netherlands, favour the linear model. The UK and others do not like to make any extrapolation as they say it is too uncertain, and also they do not like to make estimates of cancer incidence that may alarm people when they are so uncertain. The EFSA has a similar opinion (EFSA, 2005) and that was the motivation behind developing the margins of exposure (MoE) approach – to have a measure of cancer risk other than estimated incidence. Different suggested models may be based on different physiological aspects regarding possible effects of genotoxic carcinogens; nevertheless, so far, they all lack a detailed, transparent, rigorous scientific rationale to justify their employment in risk assessment of genotoxic carcinogens. Also, some experts advocate the choice of a dose response model on a “case by case” basis. In a structured elicitation exercise, which included experts from both the EU and the US, we captured expert opinion on low dose-response relationships for genotoxic carcinogens to assess the relative likelihood of alternative forms of dose-response and the extent to which this might vary between different classes of chemicals. The questions that we tried to answer via this expert elicitation exercise were:
We intent to couple the information acquired with a systematic review of the empirical form of dose-response relationships at low exposures for genotoxic carcinogens in animals to gain a greater appreciation of what is known about low-dose extrapolation.
There is a growing body of experience with the application of Cooke's Classical Model using EXCALIBUR to real scientific, engineering and medical issues. Some interesting and prominent - newsworthy even - examples will be described from a facilitator's perspective, including the unique repeated use of structured elicitations for decision support in the Montserrat volcanic eruption, and risk estimation for emergent diseases and zoonoses. Insights, limitations and pitfalls will be discussed; possible issues and topics for further research will be floated.
Expert elicitation is often employed in cases where there is little to no information about the uncertainties in question. In most of these cases the experts are situated at different locations and often have a limited time they can spend on an expert elicitation exercise. That is why gathering experts’ opinion is often a time consuming and costly exercise.
Anna Hicks and Jenni Barclay Structured elicitation of expert opinion was first applied to volcanology in 1995 when the present eruption of Soufrière Hills (Montserrat) commenced. The application of this technique has subsequently evolved, but is still used successfully by the Montserrat Volcano Observatory, and for assessment of future activity at other volcanoes. Unlike Montserrat, Tristan da Cunha (South Atlantic) is an active volcano currently with a poorly defined eruptive record and little or no effective monitoring capability. The last eruption in 1961 prompted a temporary and traumatic evacuation of the island’s whole population. In common with many other poorly defined volcanic settings, contingency measures and mitigation plans are needed for responding to future eruptive activity, but this requires consideration of the range of plausible eruptive scenarios and potential hazards – within a context of considerable scientific uncertainty. One of the goals of this project was to examine the suitability and applicability of expert elicitation in a data-impoverished setting. Tim Bedford Expert Judgement with Moment Methods I will present work with Wisse and Quigley on combining expert judgements. We show how a combination method can be developed based on moment assessments, and then discuss the analogies between this method and Cooke's classical method. Roger Cooke Update on Validation An overview of analyses with the TU Delft expert judgement database, and some thoughts on validation. |
||||||||||||||||||||||||||||||||||||||||||||
| Useful materials | ||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||
| Getting there | ||||||||||||||||||||||||||||||||||||||||||||
|
The offices of the participants from TU Delft are located at the 6th floor of the Faculty of Electrical Engineering, Mathematics and Computer Science (EWI) high building, at Mekelweg 4, Delft. Please visit room 06.050 to find Roger M. Cooke, 06.040 for Dorota Kurowicka and 06.160 for Anca Hanea. The workshop will be held in the EWI low building, in the Snijderzaal (room 01.010 on the first floor) at Mekelweg 4, Delft.
EWI building
These websites may be helpful for our participants from abroad
|
||||||||||||||||||||||||||||||||||||||||||||
| Accommodation | ||||||||||||||||||||||||||||||||||||||||||||
|
You may book a hotel in the centre of Delft (here www.delfthotels.nl). |
||||||||||||||||||||||||||||||||||||||||||||
|
Maps To and From Hotel Plataan Hotel Plataan - Workshop (EWI)
To and From Hotel Juliana |
Expert Judgement Workshop, 4 - 5 July 2011
