{"id":44273,"date":"2018-06-27T13:22:21","date_gmt":"2018-06-27T20:22:21","guid":{"rendered":"http:\/\/world.edu\/?p=44273"},"modified":"2018-06-27T13:22:21","modified_gmt":"2018-06-27T20:22:21","slug":"with-great-metrics-comes-great-responsibility","status":"publish","type":"post","link":"https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/","title":{"rendered":"With great metrics comes great responsibility"},"content":{"rendered":"

\"\"<\/p>\n

The real challenge around \u201cresponsible metrics\u201d is only partly about the metrics themselves; it is also, and perhaps more, about\u00a0how people use them.<\/p>\n

First, I must repeat a mantra that I have used widely: we mean indicators, not metrics. A metric is something like a citation count; it doesn\u2019t tell you a great deal, and it usually depends on the data source. An indicator is what you get when you try\u00a0to assess something else, like research performance. You can\u2019t measure research performance directly, because that requires expert and experienced interpretation, but you can create an indicator of something associated with performance, such as the average number of citations per paper after accounting for publication date and research field.<\/p>\n

\n
\n
\n
\n
\n

One widely used indicator is the h-index, created by Jorge Hirsch in 2005. A scholar with an h-index of n<\/em> has published at least n<\/em> papers, each of which has been cited at least n<\/em> times. This indicator is not very responsible because it does not reflect the age of papers or the rate at which citations accumulate in different fields. Older biomedical researchers almost invariably have a higher h-index than younger quantum technologists, so their relative h-indices not only tell us very little but may hide real differences in achievement.<\/p>\n

The list of papers someone produces over their career, their full CV, is a much better source of information. The problem for research managers is that they have too little time to\u00a0pore over those lists, let alone to look at even the summary sections of the papers. Much the same challenge faces peer reviewers when they\u00a0consider grant proposals with a publication history attached. The challenge escalates for members of UK\u00a0Research and Innovation grant programme or\u00a0research excellence framework evaluation panels who may be looking at large and diverse portfolios.<\/p>\n

\n
\n
\n
\n
\n

What do people do with an overwhelming task? They look for proxies to supplement the information gaps that they won\u2019t have time to fill through their own analysis. And that is where the problem of responsibility lies, because if those proxies are not as good as the user likes to think, that may lead to weak judgments, poor decisions and bad investments.<\/p>\n

To make the grade as a popular proxy indicator, and to become a \u201cgreat metric\u201d, the index needs to be very simple: preferably a one-dimensional number, so a big value means good and a small value means not so good. It also needs to appear to be transparent and comprehensible; in other words, it needs to look as if it represents the thing you really want to know. The h-index fulfils these needs because it\u2019s about more papers with more citations (must be a good thing \u2013 right?) and it\u2019s a simple, counted metric. It is just too appealing to ignore, yet, as noted, the value you get depends on other things around research culture, and it requires detailed interpretation.<\/p>\n

At the Institute for Scientific Information, we worry a lot about research indicators and how they are interpreted. In particular, we worry about another widely used \u201cgreat metric\u201d: the journal impact factor. Citation counts take time to build, so reviewers sometimes choose to look at recent papers in the context of the journal that published them. The problem is that the JIF is a journal impact factor, not an index of the papers or their authors. It\u2019s great to get a paper in Nature<\/em> (in the 2008 research assessment exercise, every UK-authored Nature<\/em> paper that could be submitted for assessment was submitted), but Nature<\/em>\u2019s high JIF value includes a wide range of article citation counts, with the median value of article citations being lower than the journal-level score.<\/p>\n

What can the ISI do to mitigate this problem? JIF is a great metric for publishers and librarians, but we think that other research folk turn to JIF as a substitute for decision-making because they lack a choice of more relevant data. So, this year\u2019s journal citation report unpacks the data to enable researchers and managers to see where each JIF fits into the overall distribution of citations for a journal. We\u2019ve separated the median (mid-point, not average) citation values for articles and more frequently cited reviews and split out \u201cnon-citable\u201d items\u00a0such as commentaries and editorials, with the full calculation of each JIF shown against the data display. Anyone looking at the\u00a0journal citation report\u00a0will now see the full context, and they can also see where all the papers that contributed the citations came from.<\/p>\n

JIF is a great metric when used for the purpose intended. Responsibility in its use requires choice, and we have made a firm move to increase every research assessor\u2019s ability to make the responsible choices for their specific requirements. Other metrics require similar development. We and other data organisations will need to work to support the responsible use of \u201cmetrics\u201d that world-class research deserves.<\/p>\n

Author Bio: <\/strong>Jonathan Adams\u00a0is a director at Clarivate Analytics\u2019 Institute for Scientific Information.<\/em><\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n

<\/div>\n
<\/div>\n
\n
<\/div>\n<\/div>\n
\n
<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

The real challenge around \u201cresponsible metrics\u201d is only partly about the metrics themselves; it is also, and perhaps more, about\u00a0how people use them. First, I must repeat a mantra that I have used widely: we mean indicators, not metrics. A metric is something like a citation count; it doesn\u2019t tell you a great deal, and […]\n… learn more→<\/a><\/p>","protected":false},"author":33,"featured_media":44274,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[86115],"tags":[209359],"coauthors":[11733],"class_list":["post-44273","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research","tag-responsible-metrics"],"acf":[],"yoast_head":"\nWith great metrics comes great responsibility - World leading higher education information and services<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:locale\" content=\"en_US\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:type\" content=\"article\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:title\" content=\"With great metrics comes great responsibility - World leading higher education information and services\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:description\" content=\"The real challenge around \u201cresponsible metrics\u201d is only partly about the metrics themselves; it is also, and perhaps more, about\u00a0how people use them. First, I must repeat a mantra that I have used widely: we mean indicators, not metrics. A metric is something like a citation count; it doesn\u2019t tell you a great deal, and […]\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:url\" content=\"https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:site_name\" content=\"World leading higher education information and services\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"article:published_time\" content=\"2018-06-27T20:22:21+00:00\" class=\"yoast-seo-meta-tag\" \/>\n<meta property=\"og:image\" content=\"https:\/\/world.edu\/wp-content\/uploads\/2018\/06\/zimm-stacks.jpg\" class=\"yoast-seo-meta-tag\" \/>\n\t<meta property=\"og:image:width\" content=\"680\" class=\"yoast-seo-meta-tag\" \/>\n\t<meta property=\"og:image:height\" content=\"253\" class=\"yoast-seo-meta-tag\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" class=\"yoast-seo-meta-tag\" \/>\n<meta name=\"author\" content=\"Kevin\" class=\"yoast-seo-meta-tag\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" class=\"yoast-seo-meta-tag\" \/>\n\t<meta name=\"twitter:data1\" content=\"Kevin\" class=\"yoast-seo-meta-tag\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" class=\"yoast-seo-meta-tag\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" class=\"yoast-seo-meta-tag\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/world.edu\\\/with-great-metrics-comes-great-responsibility\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/world.edu\\\/with-great-metrics-comes-great-responsibility\\\/\"},\"author\":{\"name\":\"Kevin\",\"@id\":\"https:\\\/\\\/world.edu\\\/#\\\/schema\\\/person\\\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee\"},\"headline\":\"With great metrics comes great responsibility\",\"datePublished\":\"2018-06-27T20:22:21+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/world.edu\\\/with-great-metrics-comes-great-responsibility\\\/\"},\"wordCount\":841,\"image\":{\"@id\":\"https:\\\/\\\/world.edu\\\/with-great-metrics-comes-great-responsibility\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/world.edu\\\/wp-content\\\/uploads\\\/2018\\\/06\\\/zimm-stacks.jpg\",\"keywords\":[\"responsible metrics\"],\"articleSection\":[\"Research\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/world.edu\\\/with-great-metrics-comes-great-responsibility\\\/\",\"url\":\"https:\\\/\\\/world.edu\\\/with-great-metrics-comes-great-responsibility\\\/\",\"name\":\"With great metrics comes great responsibility - World leading higher education information and services\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/world.edu\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/world.edu\\\/with-great-metrics-comes-great-responsibility\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/world.edu\\\/with-great-metrics-comes-great-responsibility\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/world.edu\\\/wp-content\\\/uploads\\\/2018\\\/06\\\/zimm-stacks.jpg\",\"datePublished\":\"2018-06-27T20:22:21+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/world.edu\\\/#\\\/schema\\\/person\\\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/world.edu\\\/with-great-metrics-comes-great-responsibility\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/world.edu\\\/with-great-metrics-comes-great-responsibility\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/world.edu\\\/with-great-metrics-comes-great-responsibility\\\/#primaryimage\",\"url\":\"https:\\\/\\\/world.edu\\\/wp-content\\\/uploads\\\/2018\\\/06\\\/zimm-stacks.jpg\",\"contentUrl\":\"https:\\\/\\\/world.edu\\\/wp-content\\\/uploads\\\/2018\\\/06\\\/zimm-stacks.jpg\",\"width\":680,\"height\":253},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/world.edu\\\/with-great-metrics-comes-great-responsibility\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/world.edu\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"With great metrics comes great responsibility\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/world.edu\\\/#website\",\"url\":\"https:\\\/\\\/world.edu\\\/\",\"name\":\"World leading higher education information and services\",\"description\":\"Global Higher Education Jobs, University jobs, Lecturer & tenure jobs.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/world.edu\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/world.edu\\\/#\\\/schema\\\/person\\\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee\",\"name\":\"Kevin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g8ba4c352a279c450aa9435d770c1ebf4\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g\",\"caption\":\"Kevin\"},\"description\":\"Kevin is founder of the world.edu project. The past 28 years have been involved in publishing to the education sector in print and the internet. Kevin has a degree in Education and has a many years experience in developing companies and projects.\",\"sameAs\":[\"https:\\\/\\\/world.edu\"],\"url\":\"https:\\\/\\\/world.edu\\\/author\\\/kevin\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"With great metrics comes great responsibility - World leading higher education information and services","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/","og_locale":"en_US","og_type":"article","og_title":"With great metrics comes great responsibility - World leading higher education information and services","og_description":"The real challenge around \u201cresponsible metrics\u201d is only partly about the metrics themselves; it is also, and perhaps more, about\u00a0how people use them. First, I must repeat a mantra that I have used widely: we mean indicators, not metrics. A metric is something like a citation count; it doesn\u2019t tell you a great deal, and […]","og_url":"https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/","og_site_name":"World leading higher education information and services","article_published_time":"2018-06-27T20:22:21+00:00","og_image":[{"width":680,"height":253,"url":"https:\/\/world.edu\/wp-content\/uploads\/2018\/06\/zimm-stacks.jpg","type":"image\/jpeg"}],"author":"Kevin","twitter_misc":{"Written by":"Kevin","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/#article","isPartOf":{"@id":"https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/"},"author":{"name":"Kevin","@id":"https:\/\/world.edu\/#\/schema\/person\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee"},"headline":"With great metrics comes great responsibility","datePublished":"2018-06-27T20:22:21+00:00","mainEntityOfPage":{"@id":"https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/"},"wordCount":841,"image":{"@id":"https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/#primaryimage"},"thumbnailUrl":"https:\/\/world.edu\/wp-content\/uploads\/2018\/06\/zimm-stacks.jpg","keywords":["responsible metrics"],"articleSection":["Research"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/","url":"https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/","name":"With great metrics comes great responsibility - World leading higher education information and services","isPartOf":{"@id":"https:\/\/world.edu\/#website"},"primaryImageOfPage":{"@id":"https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/#primaryimage"},"image":{"@id":"https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/#primaryimage"},"thumbnailUrl":"https:\/\/world.edu\/wp-content\/uploads\/2018\/06\/zimm-stacks.jpg","datePublished":"2018-06-27T20:22:21+00:00","author":{"@id":"https:\/\/world.edu\/#\/schema\/person\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee"},"breadcrumb":{"@id":"https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/#primaryimage","url":"https:\/\/world.edu\/wp-content\/uploads\/2018\/06\/zimm-stacks.jpg","contentUrl":"https:\/\/world.edu\/wp-content\/uploads\/2018\/06\/zimm-stacks.jpg","width":680,"height":253},{"@type":"BreadcrumbList","@id":"https:\/\/world.edu\/with-great-metrics-comes-great-responsibility\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/world.edu\/"},{"@type":"ListItem","position":2,"name":"With great metrics comes great responsibility"}]},{"@type":"WebSite","@id":"https:\/\/world.edu\/#website","url":"https:\/\/world.edu\/","name":"World leading higher education information and services","description":"Global Higher Education Jobs, University jobs, Lecturer & tenure jobs.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/world.edu\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/world.edu\/#\/schema\/person\/42c1c1d5ff7ffbdf0bd50ed94c6ae6ee","name":"Kevin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g8ba4c352a279c450aa9435d770c1ebf4","url":"https:\/\/secure.gravatar.com\/avatar\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1e8d491a5f9b9bf3db67c9e46abbba7bb06b26b458bd5025c55e0a4b9b60ea8a?s=96&d=mm&r=g","caption":"Kevin"},"description":"Kevin is founder of the world.edu project. The past 28 years have been involved in publishing to the education sector in print and the internet. Kevin has a degree in Education and has a many years experience in developing companies and projects.","sameAs":["https:\/\/world.edu"],"url":"https:\/\/world.edu\/author\/kevin\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/posts\/44273","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/users\/33"}],"replies":[{"embeddable":true,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/comments?post=44273"}],"version-history":[{"count":0,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/posts\/44273\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/media\/44274"}],"wp:attachment":[{"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/media?parent=44273"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/categories?post=44273"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/tags?post=44273"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/world.edu\/wp-json\/wp\/v2\/coauthors?post=44273"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}