The main purpose of this brief paper is to present different actual conceptions and models of academic/professional conferences and their respective papers/abstracts reviewing processes as a means to show the desirability, if not the necessity, of using a multi-methodological processes in order to review the submissions made to a multi- and inter-disciplinary conference.
There is a high diversity in conference conceptions with a considerable number of different models and papers/abstracts reviewing process, even among the most prestigious, old, large and veteran scholarly societies and institutes. The types of required submissions and their respective reviews encompass a very wide specter going from the extreme of requiring from authors to submit just full papers which would be reviewed through methods and processes very similar to the ones used by Journal's editorial processes (several ACM and IEEE conferences are examples of this extreme), to the other extreme of accepting submission of no more than 50 words abstracts requiring no reviewing process (The prestigious and large INFORMS' and IFORS' conferences and annual meetings are examples of this other extreme.). Acceptance rates have as well a very wide range going from 14.9% ACM's SIGGRAPH 98, or lower, to about 100%, which is what is explicitly stated in INFORMS 2006, for example, in the following terms: "Contributed abstracts are not reviewed and virtually all abstracts are accepted."
Mediating between these two extremes we find a plethora of different conferences' models and conceptions. The following are some representative examples:
The prestigious ACM's STOCs (ACM Symposium on Theory of Computing) and IEEE's FOCS (IEEE Symposium on Foundations of Computer Science) require submissions of extended abstracts (no more than 2000 words) and explicitly inform that they will not be accepting full papers submissions.
The International Society for Systems Sciences: ISSS (previously named International Society for Systems Research: IFSR) the oldest and the most prestigious scholar association in the Systems movement, founded, 50 years ago, by Ludwing Von Bertalanffy, Kenneth Boulding. Ralph Gerard, James Grier Miller and Anatol Rapaport, accept abstracts as submission to its yearly conferences.
The 43rd Annual Technical Meeting of the Society of Engineering Science held at PennState required the submission of abstracts and explicitly announced that "The proceedings of the 2006 SES conference will consist in one-page abstracts.
IEEE's 31st International Conference on Electronics Manufacturing and Technology (IEMT 2006), invited to the submission of extended abstracts of about 500 words and once the abstract is accepted, after its reviewing, a full paper would be included in the proceedings.
DEANZ 2006 conference Enabling E-Learning Approaches held at the Auckland University of Technology (AUT) in New Zealand, informed that "Submissions may be reviewed papers which will go through a double-blind review process to ensure quality, non-reviewed papers or posters." For the non/reviewed papers, the organizers asked for an abstract of 200 words which deadline was the same as the deadline of the Final paper submission of the reviewed option.
The 22nd International Free Electron Laser Conference held in Duke Free Electron Laser Laboratory in Duke University, announced that the "The proceedings will consist of two parts: Refereed Papers: (Part 1) comprising the invited talks and a selection of contributed papers…and Non-Refereed Papers: (Part 2) will contain contributed papers in camera-ready form with the same page size."
The 2006 International Conference of Higher Education Research and Development Society of Australasia (HERDSA) announced the inclusion of refereed and non reefed papers for their presentation at the conference, held at the University of Western Australia. Submissions for non refereed presentations are required as abstracts limited to 250 words.
These are just examples of the huge variety of the conceptions of peer reviewing that are supporting conferences' models used by conferences' organizers. This variety is a consequence of the diversity of objectives or functions that conferences might have, which explicitly or implicitly orient conference organizational processes and methodologies.
These conferences' objectives and functions differ significantly among scientific disciplines, between scientific and engineering, between epistemological and professional values, between academy and industry, between scholars, and consultants and managers, and even among different academic elites competing for the same resources. So, significant inconveniences should be expected organizing multi- and inter-disciplinary conferences, especially in those cases where an explicit purpose of the conference organizers is to bring together, not just scientists, from different disciplines, but also engineers, professionals, practitioners, consultants, managers and people from industrial, corporative and governmental realms. One way to deal with this kind of problem and hindrances in multi- and inter- disciplinary conferences is to use a hybrid model and a hybrid methodology for peer reviewing, integrated by the most used models and peer reviewing methods applied in conventional, established and habitual conferences, organized by prestigious and/or age-old academic or professional associations or institutes. Such a hybrid model may amplify the very known weaknesses of peer reviewing and, for being open to different epistemological values and teleological variables, may be more vulnerable for electronic vandalism. Consequently, additional security measures and procedures should be added to a hybrid model with hybrid methodology for peer reviewing.
There is a very limited literature with regards to what conferences are or should be; what are the most used models; what are, or how should be, the peer reviewing methods to be used, etc. This limited literature along with the increasing number and variety of conferences being organized is a baffling paradox. A significant and an increasing number of publications (papers and books) are being produced regarding journals' peer reviewing, but very few publications has been made dealing with conferences' peer reviewing. Indeed, there are also very few publications related to conferences and symposia as such. It is a paradox that so many conferences are being organized with such a high diversity of objectives, models and peer reviewing methods and the publications are scant regarding these issues.
In Walker and Hurt (Scientific and Technical Literature, 1990) we find one of the very few publications made regarding what conferences are, or should be. The first chapter of their book is about Journals. The second chapter is about conference proceedings where the nature and the objectives of conference proceedings are differentiated and contrasted to journals publishing. Walker and Hurt affirm that scientific and technical written communications are produced on a continuum between the two extremes of formal and informal communications. They characterize the formal communications as (p. XII)
And they characterize informal communications as (p. XI)
After establishing these characteristics of formal and informal written communications, Walker and Hurt affirm that "The formal media may be the more permanent means of recording ideas but they are only part of the overall enterprise…The formal media are also not all equally formal…The informal exchange of information among colleagues is much more important at certain periods of the research process than the formal media." (p. XX; emphasis added). It is this informality what represents the teleological essence of conferences. The degree of formality or informality is what generates such a plethora of conferences models and their inherent peer reviewing methods, if peer reviewing exists at all. Walker and Hurt affirm that conference proceedings, along with patents and technical reports, belong to what it is named "grey literature" which they associate with three features, as follow:
"it is not necessarily literature with original scientific contributions, but of an informational nature." (p. XXI). This is why in many conferences potential participants are invited to present position papers, case studies, white papers, etc. and in other panels are organized. This is also why we find an increasing number, in the last 25 years, of meetings with conversational formats, where no paper presentations are made at all. As it is known the input of conversational meetings are questions and problems, not answers and solutions as it is the case of conventional conferences.
Literature "is distributed in a non conventional manner and not within the normal marketing systems that make up the publishing and book trade." (p. XXI). This is why some conferences have no proceedings at all, others have just electronic proceedings with no printed version, and still others have just abstracts in their proceedings. In these conferences some authors distribute copies of their papers to those who attended the session where their paper was presented and to colleagues with whom they informally interacted in coffee breaks, lunches, etc.
Literature which "is not bibliographically controlled and therefore difficult to locate and procure." (p. XXI)
Walker and Hurt affirm that "the sources, kind and amount of grey literature are each increasing and at present [1990] may represent as much as 50% of all technical and scientific literature." (p. XXII). If in 1990, grey literature represented 50% of all technical and scientific literature, it is to suppose with a high level of certainty that now it represents a larger percentage due to the exploding publications via Internet.
Walker and Hurt affirm as well that "Along the informal-formal continuum, conference papers may rest at or near the middle…Depending on the field, papers presented at conferences may not have gone through the development or review process as that required of the journal article…In some fields the conference is a place where ideas are tested, a place to distribute research results, or a place to claim priority. This last point reduces the journal to the role of repository of the archival record." (p. 79).
According to these authors the main conference functions are the following:
"To learn of the latest work being done
To learn of the latest available equipment
To make contacts with fellow researchers
To discuss details with colleagues" (p.81)
They add that "Other purposes are served by conferences…there is a general sense that the "real" value of conferences and other lies in the informal communication that take place during, between, and after the formal presentation of prepared conference papers." (pp. 81-82) The wide specter of the purposes that a conference might have plus the high plurality of means which might be used to facilitate informal communications, which are the real value of conferences, generate the huge variety of conference models as well as the high variety of reviewing methods (or no reviewing at all) used in different conferences. This variety of conference models, approaches and perspectives require tolerance from the advocates of competing models in order to allow the emergence of the real purposes of conferences. The intolerance that sometimes emerges between advocates of different conference models and their reviewing approaches and methods, or between elites from different disciplines, or between scientific and engineering conferences, or between academic and industrial/managerial conferences is really harmful to the global effectiveness of the conferences. This kind of problems is amplified in multi- and inter-disciplinary conferences, specially in those where one of the main purpose is to facilitate inter-disciplinary communications as a means of fostering cross-disciplinary fertilization and the analogical thinking that precedes logical thinking in scientific research, engineering design and technological innovations.
In our opinion, this is why multi- and inter-disciplinary conferences; especially those trying to foster cross-fertilization and interdisciplinary communications, should accept different kind of submissions (papers, drafts, extended abstracts, short abstracts, etc.) and implement different reviewing processes, from the formal papers reviews, to semi-formal extended abstracts review, to short abstract informal review, and even to a completely informal screening of very short abstracts of no more than 50 words (as it is the case of the age old INFORMS and IFORS conferences and meetings).
We introduced, in the conception, design and organization of WMSCI 2007, and its collocated conferences, in the context of action-design, action learning and action-reflection (that may conduce to an action-research project), four kinds of submissions for 2007 conferences, along with their respective four different kinds of reviewing methods and processes. In our opinion (a hypothesis in the context of action-research), this multi-methodological approach is highly desirable - and it might be even a necessary one - for an adequate organization of a multi- or inter-disciplinary conference. When we first began with this multi-methodological approach, we stated that these different ways of accepting submissions and reviewing them (four methods in a multi-methodology for a Multi-disciplinary conferences) might change in the future because the same dynamic nature of action-learning, action design and action-reflection. After implementing these four methods for the 2007 conferences, and after experiencing the implementation of them we learned (in action learning context) that Kaplan Method might not be feasible for conference organization. It was conceived for Journal reviewing and in this context it might be a desirable and feasible method. We also learned that short abstract reviewing is not always desirable. Consequently, we removed, for the 2008 conferences, these two methods, placed at the extremes of the formal-informal specter of reviewing.
Let us first describe the 4 methods that were applied to submissions made to 2007 conferences, and then we will describe the three methods that will be applied to submissions made to 2008 conferences.
These four ways used to make submissions to the WMSCI 2007, and its collocated conferences, are the following, corresponding to review methods which go from semi-formal to the formal ones.
A brief Abstract (100-400 words, not a full paper and not more than 400 words)
An Extended Abstract (500-2000 words, not a full paper)
A draft paper to be reviewed by a Modified Kaplan's Method.
A draft paper pre-reviewed according to Kaplan's Method (Kaplan, 2005, , "How to Fix Peer Review", The Scientist, Volume 19, Issue 1, Page 10, Jun. 6)
In the context of action-learning and action-reflection, we decided for the 2008 and the 2009 conferences, as we said above, to remove the two extreme methods: the most formal and the most informal one, and following some suggestions made in conversational meetings we decided to add the option of accepting submissions for presentation only, which will not be included in the pre-conference proceedings.
Hence, for 2008 and 2009 conferences the 3 submission options that authors had were the following:
A draft paper to be reviewed by a Modified Kaplan's Method. Accordingly, authors submitting a draft paper should suggest 2-3 colleagues for the non-blind reviews of their paper.
An Extended Abstract (500-2000 words, not a full paper). Accordingly, authors submitting and extended abstract should suggest 1-3 colleagues for the non-blind reviews of their paper.
An abstracts (200-500 words, not a full paper and not more than 500 words) for presentation only. Accordingly, authors submitting a brief abstract for-presentation-only may suggest 1-3 colleagues for the possible non-blind reviews of their paper.
Their respective presenters were willing to include them in the post-conference volume of the proceedings; AND
The full paper is received, according to the required format, by the respective deadline (about 20 days after the conference is over); AND
The chair of the session where the paper was presented recommends its inclusion in the post-conference volume of the proceedings, supporting his/her recommendation on the opinions of the session's attendees.
Through 2008 and 2009 conferences we learned (from experience and from authors feedback) that the "for-presentation-only" option was not cost-effective for authors nor for the Organization Committee; and, generating confusion among authors, it was a source of undesirable entropy and miscommunication between authors and the Organizing Committee. We also learned, mainly from the conversational sessions and from informal feedback given by authors that it was a good idea to explore the option specifically oriented to support inter-disciplinary communications. Consequently, and in the context of the process of action-learning and action-reflection we were following, we decided to remove the option of "for-presentation-only" and to add an option oriented to specifically support inter-disciplinary communications.
Virtual Participation
Given the Global Recession, and thinking of those scholars, researchers and professionals related with the conference topics but unable to attend it personally (usually due to insufficient funding for the traveling costs) a Virtual Participation mode has been established, with the same peer reviewing and validity than face-to-face ones.
Submissions made for Virtual Participation would go through the same reviewing processes of the regular papers (double-blind, non-blind, and participative peer reviewing) and, if accepted (according to the same acceptance policy), they will be included in the proceedings and will be eligible for journal publication, with no additional cost, if they are, according to their reviewers, among the best 25%-30% of those physically and virtually presented at the conference.
Each regular session, included in the conference program, will be associated to a corresponding virtual session where all final versions of the articles to be presented will be displayed and authors can comment them via electronic forums. Registered authors of virtual participations will have access to all conference program sessions (and papers). Their article will be displayed as the regular ones. Virtual authors also have the option of sending, besides the final version of their article in a PDF document, an electronic presentation (PowerPoint, flash, etc. and/or a 15-20 minutes video) After paying the respective shipping and handling costs, registered authors of virtual participation, who have paid their registration fee, can get delivered the same conference material that the regular attendees receive at the registration desk.
Submission Options for the 2024 Conferences
According to what we described above, for de 2024 conferences (and following
conferences if no more changes are made) there will be three submission
options for face-to-face participation, and two corresponding
options for virtual participation. These options are the following
(and are explained in more detail below):
FA. A Draft Paper (2000-5000 words), for face-to-face participation (authors should suggest 2-3 non-anonymous reviewers) (authors should suggest 2-3 non-anonymous reviewers)
FB. An Extended Abstract (600-2000 words), for face-to-face participation (authors should suggest 1-3 non-anonymous reviewers) (authors should suggest 1-3 non-anonymous reviewers)
FC. An Abstract written for Inter-Disciplinary Communication (300-600 words), for face-to-face participation (authors may suggest up to 3 non-anonymous reviewers) (authors may suggest up to 3 non-anonymous reviewers)
VA. A Draft Paper (2000-5000 words), for virtual participation (authors should suggest 2-3 non-anonymous reviewers) (authors should suggest 2-3 non-anonymous reviewers)
VB. An Extended Abstract (600-2000 words), for virtual participation (authors should suggest 1-3 non-anonymous reviewers)
Authors should fulfill certain requirements for each of these six options as indicated below:
FA. Full Draft papers (2000-5000 words) submitted for their presentation at the conference and for inclusion in the conference proceedings. This kind of submissions will be reviewed by a Modified Kaplan's Method, where the submission's author should suggest at least two scholars (and up to three) researchers and/or professionals for the open, non-blind review of his/her paper. Each paper will also be sent to at least 3 reviewers for its double-blind review as well. Acceptance decisions will be based on both kinds of reviews: Non-blind and double-blind ones. [David Kaplan's article titled "How to fix Peer Review" (The Scientist, Volume 19, Issue 1, Page 10, Jun. 6, 2005) can also be accessed at http://www.the-scientist.com/?articles.view/articleNo/16474/title/How-to-Fix-Peer-Review]
FB. Extended abstracts (600-2000 words, not a full paper) submitted for presentation at the conference and for inclusion in the conference proceedings. Authors submitting Extended Abstracts should suggest at least one scholar, researcher, or professional for the open, non-blind review of his/her abstract. Each extended abstract will also be sent to at least one (and up to 3) reviewers for its double-blind reviewing as well. Acceptance decisions will be based on both kinds of reviewing: Non-blind and double-blind ones. "The submission should contain a scholar [or a professional] exposition of ideas, techniques, and results, including motivation and a clear comparison with related work." (as it is indicated for submissions to be made to the Annual IEEE Symposia on Foundations of Computer Science: FOCS).
FC. Abstracts, written for Inter-Disciplinary Communication (300-600 words), may be submitted for presentation at the conference and for inclusion in the conference proceedings. The purpose the Organizing Committee seeks by allowing this kind of submissions is to foster communications among different knowledge domains, different disciplines, and different kinds of experiences, as for example between academic and corporate knowledge/experience. Authors submitting abstracts for Inter-Disciplinary Communication should write both, the abstract and the full paper in a way as to be understood by scholars from other disciplines, i.e. they should be written in non-technical, non-disciplinary terms, and should clearly state the contributions the authors are making in their respective disciplinary or interdisciplinary field, and/or the potential impact of the article's content in other disciplines. Analogical thinking is suggested for these articles as complement of the usual logical-disciplinary one. Consequently, this kind of articles may contain inter-disciplinary analogies, expressional metaphors, analogical inferences, communicational analogies, analogy-based hypothesis formulations, design proposals, etc.
Authors submitting Abstracts may suggest up to 3 scholars, researchers, or professionals for open, non-blind reviewing of their respective abstract. Each abstract will also be sent to at least three reviewers for its double-blind reviewing as well. Acceptance decisions will be based on both kinds of reviewing: non-blind and double-blind. The submission should be similar to the abstracts or introductions usually written at the beginning of a full paper, containing "a scholarly [or a professional] exposition of ideas, techniques, and results, including motivation and a clear comparison with related work" (as it is indicated for submissions to be made to the Annual IEEE Symposia on Foundations of Computer Science: FOCS).
VA. Full Draft papers (2000-5000 words) submitted for Virtual Participation at the conference and for inclusion in the conference proceedings. Similarly to the face-to-face option above (indicated as FA), "this kind of submissions will be reviewed by a Modified Kaplan's Method, where the submission's author should suggest at least (and up to three) scholars, researchers and/or professionals for the open, non-blind review of his/her paper. Each paper will also be sent to at least 3 reviewers for its double-blind review as well. Acceptance decisions will be based on both kinds of reviews: Non-blind and double-blind ones. [David Kaplan's article titled "How to fix Peer Review" (The Scientist, Volume 19, Issue 1, Page 10, Jun. 6, 2005) can also be accessed at http://www.the-scientist.com/?articles.view/articleNo/16474/title/How-to-Fix-Peer-Review]"
VB. Extended abstracts (600-2000 words, not a full paper) submitted for Virtual Participation at the conference and for inclusion in the conference proceedings. Similarly to the face-to-face option above (indicated as FB), "authors submitting Extended Abstracts should suggest at least one (and up to 3) scholar, researcher, or professional for the open, non-blind review of his/her abstract. Each extended abstract will also be sent to at least three reviewers for its double-blind reviewing as well. Acceptance decisions will be based on both kinds of reviewing: Non-blind and double-blind ones. "The submission should contain a scholar [or a professional] exposition of ideas, techniques, and results, including motivation and a clear comparison with related work" (as it is indicated for submissions to be made to the Annual IEEE Symposia on Foundations of Computer Science: FOCS)."
