Close
In early 2020, 宅男视频 academic committee approved a policy on the responsible use of bibliometrics at UCL. Below you will find an introduction to the policy, and the policy's eleven principles
Introduction
Skip ahead to the policy's principles
Bibliometrics is a term describing the quantification of publications and their characteristics. It听includes a range of approaches, such as the use of citation data to quantify the influence or impact听of scholarly publications. When used in appropriate contexts, bibliometrics can provide valuable听insights into aspects of research in some disciplines.听
However, bibliometrics are sometimes used uncritically, which can be problematic for researchers听and research progress when used in inappropriate contexts.听For example, some bibliometrics have听been commandeered for purposes beyond their original design. The journal impact factor was听reasonably developed to indicate average journal citations (over a defined time period), but is often听used inappropriately as a proxy for the quality of individual articles within a journal.听Further, research 鈥渆xcellence鈥� and 鈥渜uality鈥� are abstract concepts that are difficult to measure听directly but are often inferred from bibliometrics.听
Such superficial use of research metrics in听research evaluations can be misleading. Inaccurate assessment of research can become unethical听when metrics take precedence over expert judgement, where the complexities and nuances of听research or a researcher鈥檚 profile cannot be quantified. When applied in the wrong contexts, such as听hiring, promotion, and funding decisions, irresponsible metric use can incentivize undesirable听behaviours, such as chasing publications in journals with high impact factors regardless of whether听this is the most appropriate venue for publication, or discouraging the use of open science听approaches such as preprints or data sharing.
As such, UCL has produced a policy and associated guidance on the appropriate use of metrics at听UCL. This builds on a number of prominent external initiatives on the same task, including the 厂补苍听Francisco Declaration on Research Assessment (DORA); the Leiden Manifesto for Research Metrics听and Metric Tide report. The latter urged UK institutions to develop a statement of principles on the听use of quantitative indicators in research management and assessment, where metrics should be听considered in terms of robustness (using the best available data); humility (recognising that听quantitative evaluation can complement, but does not replace, expert assessment); transparency听(keeping the collection of data and its analysis open to scrutiny); diversity (reflecting a multitude of听research and researcher career paths); and reflexivity (updating our use of bibliometrics to takeaccount of the effects that such measures have had). These initiatives and the development of听institutional policies are also supported or mandated by research funders in the UK (e.g., UK听Research Councils, Wellcome Trust, REF).
This Policy Statement aims to balance the benefits and limitations of bibliometric use to create a听framework for the responsible use of bibliometrics at UCL and to suggest ways in which they can be听used to deliver the ambitious vision for excellence in research, teaching, and learning embodied in听the UCL 2034 strategy.听We recognize 宅男视频 is a dynamic and diverse university, and no metric or set of metrics could听universally be applied across our institution. Many disciplines or departments do not use research听metrics in any way, because they are not appropriate in the context of their field. UCL recognises听this and will not seek to impose the use of metrics in these cases. For those fields where metrics are听used, this Policy Statement is deliberately broad and flexible to take account of the diversity of听contexts, and is not intended to provide a comprehensive set of rules. To help put this into practice,听we will provide an evolving set of guidance material with more detailed discussion and examples of听how these principles could be applied. 宅男视频 is committed to valuing research and researchers based听on their own merits, not the merits of metrics.
Principles for the responsible use of bibliometrics
- Quality, influence, and impact of research are typically abstract concepts that prohibit direct听measurement. There is no simple way to measure research quality, and quantitative听approaches can only be interpreted as indirect proxies for quality.
- Different fields have different perspectives of what characterises research quality, and听different approaches for determining what constitutes a significant research output (for听example, the relative importance of book chapters vs journal articles). All research output听must be considered on their own merits, in an appropriate context that reflects the needs听and diversity of research fields and outcomes.
- Both quantitative and qualitative forms of research assessment have their benefits and听limitations. Depending on the context, the value of different approaches must be considered听and balanced. This is particularly important when dealing with a range of disciplines with听different publication practices and citation norms. In fields where quantitative metrics are听not appropriate nor meaningful, UCL will not impose their use for assessment in that area.
- When making qualitative assessments, avoid making judgements based on external factors听such as the reputation of authors, or of the journal or publisher of the work; the work itself听is more important and must be considered on its own merits.
- Not all indicators are useful, informative, or will suit all needs; and metrics that are听meaningful in some contexts can be misleading or meaningless in others. For example, in听some fields or subfields, citation counts can estimate elements of usage, but in others they听are not useful at all.
- Avoid applying metrics to individual researchers, particularly metrics which do not account听for individual variation or circumstances. For example, the h-index should not be used to听directly compare individuals, because the number of papers and citations differs听dramatically among fields and at different points in a career.
- Ensure that metrics are applied at the correct scale of the subject of investigation, and do听not apply aggregate level metrics to individual subjects, or vice versa. For example, do not听assess the quality of an individual paper based on the impact factor of the journal in which it听was published.
- Quantitative indicators should be selected from those which are widely used and easily听understood to ensure that the process is transparent and they are being applied听appropriately. Likewise, any quantitative goals or benchmarks must be open to scrutiny.
- If goals or benchmarks are expressed quantitatively, care should be taken to avoid the听metric itself becoming the target of research activity at the expense of research quality.
- New and alternative metrics are continuously being developed to inform the reception,听usage, and value of all types of research output.Any new or non-standard metric or听indicator must be used and interpreted in keeping with the other principles listed here for听more traditional metrics. Additionally, consider the sources and methods behind such听metrics and whether they are vulnerable to being gamed, manipulated, or fabricated.
- Bibliometrics are available from a variety of services, with differing levels of coverage,听quality and accuracy, and these aspects should be considered when selecting a source for听data or metrics. Where necessary, such as in the evaluation of individual researchers, choose听a source that allows records to be verified and curated to ensure records are comprehensive听and accurate, or compare publication lists against data from the UCL IRIS/RPS systems.
This policy was approved by UCL Academic Committee, 27th听February 2020.
听
Supporting information
External initiatives and UCL's committments听
听
The text of the and can be adapted听or redistributed by third parties. However, to avoid confusion, please ensure that any modified version is not听labelled as a UCL policy.