Delphi method - Wikipedia, the free encyclopedia

Page 1 of 4

Delphi method
From Wikipedia, the free encyclopedia

(Redirected from Delphi technique) The Delphi method (pron.: /dlfa/ DEL-fy) is a structured communication technique, originally developed as a systematic, interactive forecasting method which relies on a panel of experts.[1] In the standard version, the experts answer questionnaires in two or more rounds. After each round, a facilitator provides an anonymous summary of the experts forecasts from the previous round as well as the reasons they provided for their judgments. Thus, experts are encouraged to revise their earlier answers in light of the replies of other members of their panel. It is believed that during this process the range of the answers will decrease and the group will converge towards the "correct" answer. Finally, the process is stopped after a pre-defined stop criterion (e.g. number of rounds, achievement of consensus, stability of results) and the mean or median scores of the final rounds determine the results.[2] Other versions, such as the Policy Delphi,[3] have been designed for normative and explorative use, particularly in the area of social policy and public health.[4] In Europe, more recent web-based experiments have used the Delphi method as a communication technique for interactive decision-making and e-democracy.[5] Delphi is based on the principle that forecasts (or decisions) from a structured group of individuals are more accurate than those from unstructured groups.[6] This has been indicated with the term "collective intelligence".[7] The technique can also be adapted for use in face-to-face meetings, and is then called mini-Delphi or EstimateTalk-Estimate (ETE). Delphi has been widely used for business forecasting and has certain advantages over another structured forecasting approach, prediction markets.
[8]

Contents
1 History 2 Key characteristics 2.1 Structuring of information flow 2.2 Regular feedback 2.3 Anonymity of the participants 3 Role of the facilitator 4 Use in forecasting 5 Acceptance 6 Use in policy-making 7 Delphi applications not aiming at consensus 8 Delphi vs. prediction markets 9 Online Delphi forecasting systems 10 Software 11 See also 12 Further reading 13 References 14 External links

History
The name "Delphi" derives from the Oracle of Delphi. The authors of the method were not happy with this name, because it implies "something oracular, something smacking a little of the occult".[citation needed] The Delphi method is based on the assumption that group judgments are more valid than individual judgments. The Delphi method was developed at the beginning of the Cold War to forecast the impact of technology on warfare.[9] In 1944, General Henry H. Arnold ordered the creation of the report for the U.S. Army Air Corps on the future technological capabilities that might be used by the military. Different approaches were tried, but the shortcomings of traditional forecasting methods, such as theoretical approach, quantitative models or trend extrapolation, in areas where precise scientific laws have not been established yet, quickly became apparent. To combat these shortcomings, the Delphi method was developed by Project RAND during the 1950-1960s (1959) by Olaf Helmer, Norman Dalkey, and Nicholas Rescher.[10] It has been used ever since, together with various modifications and reformulations, such as the Imen-Delphi procedure. Experts were asked to give their opinion on the probability, frequency, and intensity of possible enemy attacks. Other experts could anonymously give feedback. This process was repeated several times until a consensus emerged.

Key characteristics
The following key characteristics of the Delphi method help the participants to focus on the issues at hand and separate Delphi from other methodologies:

Structuring of information flow

The initial contributions from the experts are collected in the form of answers to questionnaires and their comments to these answers. The panel director controls the interactions among the participants by processing the information and filtering out irrelevant content. This avoids the negative effects of face-to-face panel discussions and solves the usual problems of group dynamics.

Regular feedback

The Delphi Method communication structure

Participants comment on their own forecasts, the responses of others and on the progress of the panel as a whole. At any moment they can revise their earlier statements. While in regular group meetings participants tend to stick to previously stated opinions and often conform too much to the group leader; the Delphi method prevents it.

Anonymity of the participants

Usually all participants remain anonymous. Their identity is not revealed, even after the completion of the final report. This prevents the authority, personality, or reputation of some participants from dominating others in the process. Arguably, it also frees participants (to some extent) from their personal biases, minimizes the "bandwagon effect" or "halo effect", allows free expression of opinions, encourages open critique, and facilitates admission of errors when revising earlier judgments.

http://en.wikipedia.org/wiki/Delphi_technique

1/18/2013

Delphi method - Wikipedia, the free encyclopedia

Page 2 of 4

Role of the facilitator

The person coordinating the Delphi method can be known as a facilitator, and facilitates the responses of their panel of experts, who are selected for a reason, usually that they hold knowledge on an opinion or view. The facilitator sends out questionnaires, surveys etc. and if the panel of experts accept, they follow instructions and present their views. Responses are collected and analyzed, then common and conflicting viewpoints are identified. If consensus is not reached, the process continues through thesis and antithesis, to gradually work towards synthesis, and building consensus.

Use in forecasting
First applications of the Delphi method were in the field of science and technology forecasting. The objective of the method was to combine expert opinions on likelihood and expected development time, of the particular technology, in a single indicator. One of the first such reports, prepared in 1964 by Gordon and Helmer, assessed the direction of long-term trends in science and technology development, covering such topics as scientific breakthroughs, population control, automation, space progress, war prevention and weapon systems. Other forecasts of technology were dealing with vehicle-highway systems, industrial robots, intelligent internet, broadband connections, and technology in education. Later the Delphi method was applied in other areas, especially those related to public policy issues, such as economic trends, health and education. It was also applied successfully and with high accuracy in business forecasting. For example, in one case reported by Basu and Schroeder (1977),[11] the Delphi method predicted the sales of a new product during the first two years with inaccuracy of 34% compared with actual sales. Quantitative methods produced errors of 1015%, and traditional unstructured forecast methods had errors of about 20%. The Delphi method has also been used as a tool to implement multi-stakeholder approaches for participative policy-making in developing countries. The governments of Latin America and the Caribbean have successfully used the Delphi method as an open-ended public-private sector approach to identify the most urgent challenges for their regional ICT-for-development eLAC Action Plans.[12] As a result, governments have widely acknowledged the value of collective intelligence from civil society, academic and private sector participants of the Delphi, especially in a field of rapid change, such as technology policies. In this sense, the Delphi method can contribute to a general appreciation of participative policy-making.

Acceptance
Overall the track record of the Delphi method is mixed. There have been many cases when the method produced poor results. Still, some authors attribute this to poor application of the method and not to the weaknesses of the method itself.[citation needed] It must also be realized that in areas such as science and technology forecasting, the degree of uncertainty is so great that exact and always correct predictions are impossible, so a high degree of error is to be expected. Another particular weakness of the Delphi method is that future developments are not always predicted correctly by consensus of experts. Firstly, the issue of ignorance is important. If panelists are misinformed about a topic, the use of Delphi may only add confidence to their ignorance.[13] Secondly, sometimes unconventional thinking of amateur outsiders may be superior to expert thinking. One of the initial problems of the method was its inability to make complex forecasts with multiple factors. Potential future outcomes were usually considered as if they had no effect on each other. Later on, several extensions to the Delphi method were developed to address this problem, such as cross impact analysis, that takes into consideration the possibility that the occurrence of one event may change probabilities of other events covered in the survey. Still the Delphi method can be used most successfully in forecasting single scalar indicators. Despite these shortcomings, today the Delphi method is a widely accepted forecasting tool and has been used successfully for thousands of studies in areas varying from technology forecasting to drug abuse.[14]

Use in policy-making
From the 1970s, the use of the Delphi technique in public policy-making introduces a number of methodological innovations. In particular: the need to examine several types of items (not only forecasting items but, typically, issue items, goal items, and option items) leads to introducing different evaluation scales which are not used in the standard Delphi. These often include desirability, feasibility (technical and political) and probability, which the analysts can use to outline different scenarios: the desired scenario (from desirability), the potential scenario (from feasibility) and the expected scenario (from probability); the complexity of the issues posed in public policy-making leads to give more importance to the arguments supporting the evaluations of the panelists; so these are often invited to list arguments for and against each option item, and sometimes they are given the possibility to suggest new items to be submitted to the panel; for the same reason, the scaling methods, which are used to measure panel evaluations, often include more sophisticated approaches such as multi-dimensional scaling. Further innovations come from the use of computer-based (and later web-based) Delphi conferences. According to Turoff and Hiltz,[15] in computer-based Delphis: the iteration structure used in the paper Delphis, which is divided into three or more discrete rounds, can be replaced by a process of continuous (roundless) interaction, enabling panelists to change their evaluations at any time; the statistical group response can be updated in real-time, and shown whenever a panelist provides a new evaluation. According to Bolognini,[16] web-based Delphis offer two further possibilities, relevant in the context of interactive policy-making and e-democracy. These are: the involvement of a large number of participants, the use of two or more panels representing different groups (such as policy-makers, experts, citizens), which the administrator can give tasks reflecting their diverse roles and expertise, and make them to interact within ad hoc communication structures. For example, the policy community members (policy-makers and experts) may interact as part of the main conference panel, while they receive inputs from a virtual community (citizens, associations etc.) involved in a side conference. These web-based variable communication structures, which he calls Hyperdelphi (HD), are designed to make Delphi conferences "more fluid and adapted to the hypertextual and interactive nature of digital communication".

Delphi applications not aiming at consensus

A web-based communication structure (Hyperdelphi).[16]

Traditionally the Delphi method has aimed at a consensus of the most probable future by iteration. The Policy Delphi (http://www.is.njit.edu/pubs/delphibook/ch3b1.html) , launched by Murray Turoff, is instead a decision support method aiming at structuring and discussing the diverse views of the preferred future. The Argument Delphi (http://en.vatt.fi/publications/latestPublications/publication/Publication_343_id/221) , developed by Osmo Kuusi, focuses on ongoing discussion and finding relevant arguments rather than focusing on the output. The Disaggregative Policy Delphi (http://dx.doi.org/10.1016/S00401625(01)00177-9) , developed by Petri Tapio, uses cluster analysis as a systematic tool to construct various scenarios of the future in the latest Delphi round.[17] The respondent's view on the probable and the preferable future are dealt with as separate cases.

http://en.wikipedia.org/wiki/Delphi_technique

1/18/2013

Delphi method - Wikipedia, the free encyclopedia

Page 3 of 4

Delphi vs. prediction markets

As can be seen from the Methodology Tree of Forecasting (http://www.forecastingprinciples.com/index.php?option=com_content&task=view&id=16&Itemid=16) , Delphi has characteristics similar to prediction markets as both are structured approaches that aggregate diverse opinions from groups. Yet, there are differences that may be decisive for their relative applicability for different problems.[8] Some advantages of prediction markets derive from the possibility to provide incentives for participation. 1. They can motivate people to participate over a long period of time and to reveal their true beliefs. 2. They aggregate information automatically and instantly incorporate new information in the forecast. 3. Participants do not have to be selected and recruited manually by a facilitator. They themselves decide whether to participate if they think their private information is not yet incorporated in the forecast. Delphi seems to have these advantages over prediction markets: 1. Potentially quicker forecasts if experts are readily available.

Online Delphi forecasting systems

A number of Delphi forecasts are conducted using web sites that allow the process to be conducted in Real-time Delphi. For instance, the TechCast Project (http://www.techcast.org/) uses a panel of 100 experts worldwide to forecast breakthroughs in all fields of science and technology. Further examples are several studies conducted by the Institute for Futures Studies and Knowledge Management (http://www.ebs.edu/ifk) that use an online-based Delphi method. Another example is the Horizon Project, where educational futurists collaborate online using the Delphi method to come up with the technological advancements to look out for in education for the next few years.

Software
Principles of Forecasting (http://armstrong.wharton.upenn.edu/delphi2/) A free service to support Delphi forecasting and references are available on this site. However source code is not currently available. Institute for Futures Studies and Knowledge Management (http://www.ebs.edu/ifk) Access to several free studies that illustrate the use of the Real-time Delphi method

See also
Real-time Delphi Online-based Delphi method Cooke method of expert elicitation, similar to the delphi method, but weightens expert's estimates to derive an estimate which takes the expert's proficiency into account[18] The Wisdom of Crowds Wideband delphi John Brunner's description of a Delphi pool in his novel The Shockwave Rider DARPA's Policy Analysis Market Prediction markets Forecasting Reference class forecasting Planning Poker Nominal group technique Social engineering (political science) Psychological manipulation Group think Conformity Bandwagon effect

Further reading

von der Gracht, H. A./ Darkow, I.-L.: Scenarios for the Logistics Service Industry: A Delphi-based analysis for 2025 (http://www.sciencedirect.com/science? _ob=ArticleURL&_udi=B6VF8-5004PXW-3&_user=10&_coverDate=09%2F30% 2F2010&_alid=1445830816&_rdoc=1&_fmt=high&_orig=search&_origin=search&_zone=rslt_list_item&_cdi=6004&_sort=r&_st=13&_docanchor=&view=c&_ct=1&_acct= In: International Journal of Production Economics, Vol. 127, No. 1, 2010, 46-59. - A case for a scientificly robust Delphi-based scenario process Harold A. Linstone and Murray Turoff, Editors Linstone & Turoff (1975). The Delphi Method: Techniques and Applications (http://www.is.njit.edu/pubs/delphibook/) : a heavily referenced work on this method with an extensive bibliography (http://www.is.njit.edu/pubs/delphibook/appendix.pdf) . Sackman, H. (1974), "Delphi Assessment: Expert Opinion, Forecasting and Group Process", R-1283-PR, April 1974. Brown, Thomas, "An Experiment in Probabilistic Forecasting", R-944-ARPA, 1972 - the first RAND paper. Bernice B. Brown (1968). "Delphi Process: A Methodology Used for the Elicitation of Opinions of Experts." (http://www.rand.org/cgibin/Abstracts/ordi/getabbydoc.pl?doc=P-3925) : An earlier paper published by RAND (Document No: P-3925, 1968, 15 pages)

References
1. ^ Harold A. Linstone, Murray Turoff (1975), The Delphi Method: Techniques and Applications (http://is.njit.edu/pubs/delphibook/) , Reading, Mass.: Addison-Wesley, ISBN 9780-201-04294-8, http://is.njit.edu/pubs/delphibook/ 2. ^ Rowe and Wright (1999): The Delphi technique as a forecasting tool: issues and analysis. International Journal of Forecasting, Volume 15, Issue 4, October 1999. 3. ^ Murray Turoff, "The Design of a Policy Delphi" (http://is.njit.edu/pubs/delphibook/ch3b1.pdf) , Technological Forecasting and Social Change 2, 2, 1970. 4. ^ Michael Adler, Erio Ziglio (eds.) (1996), Gazing Into the Oracle: The Delphi Method and Its Application to Social Policy and Public Health, London: Kingsley Publishers, ISBN 978-1-85302-104-6 5. ^ Maurizio Bolognini (2001) (in Italian), Democrazia elettronica. Metodo Delphi e politiche pubbliche (Electronic Democracy. Delphi Method and Public Policy-Making), Rome: Carocci Editore, ISBN 88-430-2035-8. An example of e-democracy application is DEMOS (Delphi Mediation Online System), whose prototype was presented at the 3rd Worldwide Forum on Electronic Democracy, in 2002. 6. ^ Rowe and Wright (2001): Expert Opinions in Forecasting. Role of the Delphi Technique. In: Armstrong (Ed.): Principles of Forecasting: A Handbook of Researchers and Practitioners, Boston: Kluwer Academic Publishers. 7. ^ Hiltz, S.R., Turoff M. (1978), The Network Nation: Human Communication via Computer, Addison-Wesley, ISBN 978-0-262-08219-8 8. ^ a b Green, Armstrong, and Graefe (2007): Methods to Elicit Forecasts from Groups: Delphi and Prediction Markets Compared. Forthcoming in Foresight: The International Journal of Applied Forecasting (Fall 2007). PDF format (http://mpra.ub.uni-muenchen.de/4999/01/MPRA_paper_4999.pdf) 9. ^ "JVTE v15n2: The Modified Delphi Technique - A Rotational Modification," Journal of Vocational and Technical Education, Volume 15 Number 2, Spring 1999, web: VT-eduJVTE-v15n2 (http://scholar.lib.vt.edu/ejournals/JVTE/v15n2/custer.html) : of Delphi Technique developed by Olaf Helmer and Norman Dalkey. 10. ^ Rescher(1998): Predicting the Future, (Albany, NY: State University of New York Press, 1998). ([1] (http://www.rand.org/pubs/papers/P3558/) , [2] (http://www.rand.org/pubs/research_memoranda/RM5888) , [3] (http://www.rand.org/pubs/papers/P1513/) )

http://en.wikipedia.org/wiki/Delphi_technique

1/18/2013

Delphi method - Wikipedia, the free encyclopedia

Page 4 of 4

11. ^ Basu, Shankar; Roger G. Schroeder (May 1977). "Incorporating Judgments in Sales Forecasts: Application of the Delphi Method at American Hoist & Derrick" (http://interfaces.journal.informs.org/content/7/3/18.abstract) . Interfaces 7 (3): 1827. doi:10.1287/inte.7.3.18 (http://dx.doi.org/10.1287%2Finte.7.3.18) . http://interfaces.journal.informs.org/content/7/3/18.abstract. 12. ^ Martin Hilbert, Ian Miles and Julia Othmer: "Foresight tools for participative policy-making in inter-governmental processes in developing countries: Lessons learned from the eLAC Policy Priorities Delphi" Technological Forecasting and Social Change, Volume 76, Issue 7, September 2009, Pages 880-896, Volume 15 Number 2, web: [4] (http://linkinghub.elsevier.com/retrieve/pii/S0040162509000031) 13. ^ Green, K. C., Armstrong, J. S., & Graefe, A. (2007). Methods to elicit forecasts from groups: Delphi and prediction markets compared. Foresight: The International Journal of Applied Forecasting, 8, 1720. [5] (http://qbox.wharton.upenn.edu/documents/mktg/research/Delphi-WPv33.pdf) 14. ^ The Delphi Method:Techniques and Applications,Harold A. Linstone and Murray Turoff, Editors 2002, Murray Turoff and Harold Linstone, TOC III.B.3. The National DrugAbuse Policy Delphi: Progress Report and Findings to Date, IRENE ANNE JILLSON {http://is.njit.edu/pubs/delphibook/ch3b3.html 15. ^ Murray Turoff, Starr Roxanne Hiltz, "Computer-based Delphi processes", in Michael Adler, Erio Ziglio (eds.), Gazing Into the Oracle, op. cit. 16. ^ a b Maurizio Bolognini (2001) (in Italian), Democrazia elettronica. Metodo Delphi e politiche pubbliche (Electronic Democracy. Delphi Method and Public Policy-Making), Rome: Carocci Editore, ISBN 88-430-2035-8. A summary is also in Jerome C. Glenn, Theodore J. Gordon (eds) (2009), The Millennium Project. Futures Research Methodology, New York: Amer Council for the United Nations, ISBN 978-0981894119, chap. 23. 17. ^ Tapio, P. 2003. Disaggregative Policy Delphi: Using cluster analysis as a tool for systematic scenario formation, Technological Forecasting and Social Change 70(1): 83-101. 18. ^ Nature, Vol. 463, pp.294-5

Sharon Colton and Tim Hatcher (2004). The Web-based Delphi Research Technique as a Method for Content Validation in HRD and Adult Education Research. [6] (http://www.mpc.edu/FacultyStaff/SharonColton/Pages/SharonColton.aspx) Tim Hatcher and Sharon Colton (2006). Using the Internet to Improve HRD Research: The case of the web-based Delphi research technique to achieve content validity of an HRD-oriented measurement. [7] (http://www.mpc.edu/FacultyStaff/SharonColton/Pages/SharonColton.aspx)

External links
Need of Forecasting in Online Business (http://salesstrategy.wordpress.com/2007/01/07/need-of-intelligent-forecasting-in-online-business/) The Delphi Method: Techniques and Applications (http://www.is.njit.edu/pubs/delphibook/) , edited by Harold A. Linstone and Murray Turoff a comprehensive book on Delphi method (free download, 11Mb PDF, 618 pages) RAND publications on the Delphi Method (http://www.rand.org/international_programs/pardee/pubs/futures_method/delphi.html) Downloadable documents from RAND concerning applications of the Delphi Technique. Retrieved from "http://en.wikipedia.org/w/index.php?title=Delphi_method&oldid=533285220" Categories: Anticipatory thinking Estimation theory Futurology Technology forecasting

Navigation menu
This page was last modified on 16 January 2013 at 00:20. Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. See Terms of Use for details. Wikipedia is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.

http://en.wikipedia.org/wiki/Delphi_technique

1/18/2013