Human and social sciences encompass disciplines with significantly different practices that call for evaluation methods adapted to their differences. Some of these disciplines, for example, place books at the top of the publications list, while others favour articles published in peer-reviewed journals, or studies presented in international congresses. Thus, an abstract, or a simple text intended for the 29 layman that has little value in some disciplines, will be considered a top-ranking publicatio in some areas of law. In some cases, English is the language of scientific research and, to quite a significant extent, the language of evaluation; in others, other languages are recognised as such. The greatly contrasting use of bibliometrics and variable review rankings and even simple bibliographic overviews . – From one discipline to another, gives an idea of these variations. KEIS has constantly endeavoured to tackle these differences in conscientiously carrying out its evaluations – without seeking to remove them completely
Although the methodology chosen by KEIS pays careful attention to these specific features, it does not create as many special cases as there are disciplinary singularities or disciplinary groups with a specific identity, such as humanities or cultural domains. Furthermore, it does not define a field that would stand completely apart with no measure in common with the others, as this would give human and social sciences an exception status in the evaluation field. Indeed, the singularities are far from being limited to that field alone. Research in mathematics also takes distinctive forms and responds to distinctive uses if it is compared to research conducted in engineering. The differences and complementarities between applied and basic research are relevant to molecular and clinical research as much as to economics and management. The problems posed by disciplinary specificity are something that goes well beyond the major disciplinary fields: the longer the list of differences, the longer the list of similarities and once more raises the question of the commensurability of disciplines. Many traits that appear to be specific to the practices of some are also present in others when it comes to evaluation.
That is why KEIS has decided to draw up fairly flexible and adaptable multidisciplinary standards both common and specific, since they combine broad generality with characteristics that make sense in each discipline. Accordingly, KEIS standards take into account the specific character of human and social sciences in the field of the evaluation.
This attention to their specific features is expressed in two complementary ways. On the one hand, in keeping with the principles of qualitative evaluation, determination of the disciplinary characteristics is entrusted to expert committees, the “peers” who, by definition, belong to the same scientific communities as the assessed entities. On the other hand, specifications tailored to human and social sciences have been introduced in the evaluation criteria standards on the basis of discussions between the KEIS scientific officers and external experts, held during a weekly seminar from September 2020 to January 2021. The practical consequence of this approach is that the result is not another standard but a joint standard incorporating the perspectives of human and social sciences on the same footing as the others that can adapt accordingly when necessary
We will not, therefore, define new versions of the six evaluation criteria intended for human and social sciences alone: there would be no point in doing so as it would go against the purpose for which the KEIS evaluation criteria standards were designed. Admittedly, it is not a matter of ironing out certain difficulties: the interactions of research with the non-academic environment, covered by criterion 3 are, for example, a subject of variable interest in human and social sciences. In fact, the work of all disciplines in the field, at close examination, is of interest to social groups and economic or cultural stakeholders. Very often, without distorting the 31 nature and focus of the research specific to these disciplines, the difficulty merely involves revealing the reality – often overlooked or downplayed – of their impact on the economy, on society and on cultural life. That is why the standards for criterion 3 (cf. p. 8) contain specifications bringing the observable facts and quality indicators into line with the uses of human and social sciences.
It is important to remember this key point : research institutions, owing to their diversity, will not completely and uniformly satisfy all the items selected: these should be tailored according to the identity of the entities, their missions and the subject of their research. This is precisely what gives its full meaning to peer evaluation, experts, who themselves belong to the disciplinary field(s) of the research entities they evaluate, know how to adapt this common language and give it the emphasis required for their field, in order to be recognised and understood by their community. Another subject that is acknowledged to be difficult with regard to human and social sciences – even if its extension is much broader in reality – is the relative weight of the types of publication and other scientific outputs according to discipline, hence the difficulty of making a uniform assessment of these subjects in the scientific production and quality
criterion (criterion 1).
The most commonly cited example to back up this observation is the insufficiency of scientometric tools for a significant proportion of disciplines in the field. In order to integrate the variety of publication forms and other scientific outputs in human and social sciences as well as the relative diversity of languages used for research in this field, KEIS has therefore considered it worthwhile to offer certain clarifications with respect to the observable facts and quality indicators relating to this criterion. These further specifications are presented in the following pages.
Scientific output and quality in human and social sciences: observable facts
Scientific outputs gives overwhelming precedence to books in many disciplinary sectors of human and social sciences, particularly the humanities. These disciplinary sectors are also hampered by the low presence of the journals in which they publish in relevant bibliometric databases. That is why the evaluation of scientific outputs and quality in human and social sciences requires special attention to be paid to the preliminary characterisation of scientific books and journals. KEIS proposals are listed below.
– The characterisation of journals The characterisation of journals, which supports the elements of the standards provided for the first criterion (see above, p. 6), is intended to facilitate evaluation and self-evaluation in the perspective of collective qualitativeevaluation by expert committees who remain the most competent to assess the scientific production and quality of research entities.
Alt is therefore necessary to characterise journals without claiming to pass judgement on the quality of the articles using that mode of dissemination. Not all of the characterisation elements listed below are necessarily relevant to the same degree for all the disciplines of human and social sciences; they must therefore be assessed in light of the features that are specific to each of these disciplines.
Characterisation elements of journals in human and social sciences To characterise a journal, the following data can be collected:
Identification:
– Title
– ISNN
– IeSSN
– Website address
– Disciplinary field(s)
– Name of the director of the publication
– Institutional support (university, organisation,
scientific society, public authority, etc.)
Dissemination:
– Dissemination start date (age of journal)
– Publisher
– Distributor
– Print run per issue (average over 5 years)
– Number of copies sold per issue (average over 5 years)
– Publication language(s) (French/other language, monolingual/multilingual)
– Publication at regular intervals (yes/no)
– Number of issues per year
– Type of publication (paper and/or online)
– Access to online publications (open access, pay access, embargo period)
– Abstract (none, in French, in English, in another language, multilingual)
– Key word indexing (none, in French, in English, in another language, multilingual) Selection of articles:
– Display of selection criteria (yes/no)
-Open calls for papers (for thematic issues)
– Peer evaluation (none, single blind, double blind, single non-anonymous, double non-anonymous)
– Selection by the issue editor (yes/no)
– Articles refused (yes/no)
– Average volume of articles published (in number of signs)
Scientific quality:
– Scientific advisory board (yes/no)
– Editorial board (yes/no)
– Peer-review committee (yes/no)
– Scientific reference system : notes, bibliography,
etc. (yes/no)
– Type of articles selected (thematic review, meta analyses, articles reporting original research, theoretical or critical discussions, viewpoints,
debates or controversy, empirical research, etc.)
Editorial policy:
– Identifiable editorial line (yes/no)
– Diversity of published authors (outside laboratory or unit, etc.)
– Multidisciplinarity (yes/no)
– Cultural areas (yes/no)
– Foreign language authors translated in the journal
Reputation:
– International (yes/no)
– Indexing in international lists of journals (yes/no)
– Award-winning articles (yes/no) 19
– The characterisation of scientific publications On the basis of other observable facts, it is possible to distinguish diverse categories of scientific publications in human and social sciences, without claiming to be exhaustive and taking into account the specific uses of each discipline:
Elements for the characterisation of scientific publications and books in human and social sciences.
Three main elements can be distinguished.
The first is the type of authorship:
– Publications containing a single, homogeneous text, by a single author;
– Publications containing a single, homogeneous text, by several authors;
– Collective publications comprising essays, studies and chapters written by different authors, under the responsibility of one or more academic editor(s);
– Collective publications comprising essays, studies 35 and chapters written by different authors with no identifiable academic editor.
The second element concerns the type of approach with regard to its subject. This makes a distinction between:
– Publications presenting original research findings on a question or topic for a restricted, specialised readership;
– Publications based on philological research : editions of texts (and, notably, critical editions) as well as translations;
– Publications synthesising other scientific work to present current knowledge on a research topic or question. Such syntheses, often designed to inform a broader readership rather than the scientific community, differ from publications for a general readership, which exploit previous research findings (one’s own or of other researchers) in the sense that they offer both added scientific value and original research.
The third element concerns the presence, in such publications, of a clear critical apparatus (notes and bibliographic references) and consultation tools (index of names, works, thematic index and glossary).
Scientific output and quality in human and social sciences: quality indicators KEIS provides the expert committees with two types of instruments to assess scientific production and quality in human and social sciences: lists of journals and the definition of the conditions for accessing the research publication category for conference proceedings and collective publications
– List of journals
The increase in periodicals at international level illustrates not only the growth in the world’s community of researchers, but also a profound change in the way in which research findings 36 are published – such as
– the development of multidisciplinary approaches, which leads numerous researchers in human and social sciences to publish their finding in journals devoted to disciplines other than their own.
– Conference proceedings and collective works With regard to conference proceedings and, more generally, collective works in the field of human and social sciences, KEIS distinguishes what constitutes a genuine work of scientific publication – which should be taken into account in the evaluation of research works – from the mere juxtaposition of communications.
The scientific publication of conference proceedings and collective works The publications comprising texts from presentations or conferences delivered at symposia, congresses or seminars will be considered as research if they have undergone a process of scientific editing characterised by:
– A clear, rationalised critical apparatus (notes and bibliographic references) for the entire work; consultation tools (index of names, works, thematic index and glossary);
– An in-depth disciplinary or interdisciplinary development, identifiable in the general presentation; the appropriateness of the publication’s structure in this regard; the selection of contributions according to their relevance to the subject; the work carried out on each of them to ensure scientific quality.
That scientific editing work also constitutes the minimum condition for considering the other works comprising texts by different authors as research works.