Workshop on The Use of Generative AI for Science of Science and Higher Education Studies (AISci)

Registration for the AISci workshop is open until the 27th of January.

Confirmation of registration will be sent to participants from w/c 20 January 2025.

Program

Wednesday 05.02.2025

13.30 Opening of the workshop (Benedetto Lepori)

14.00 Keynote speech 1. Generative AI in Social Sciences. State of the Art and future perspectives, Karsten Donnay, University of Zurich.

Discussant: Jens-Peter Andersen.

15.30 – 16.45 Higher Education Institutions Chair: Marek Kwiek

Finding ‘similar’ universities using ChatGPT. A large-scale comparison using ETER data (Benedetto Lepori, Lutz Bornmann and Mario Gay)

Using generative AI to evaluate educational innovations proposals in Higher Education (Andrea Kottmann and Renze Kolster)

17.00 – 18.00 Presentation of the book project “How to use LLMs for your research”, Vito Giordano, Pietro Cruciata, Catherine Beaudry

19.00 Workshop dinner

Thursday 06.02.2025

09.00 – 12.00 Generative AI for identifying and labelling research topics, Chair: Lutz Bornmann

Generative AI for automatic topic labelling (Diego Kozlowski, Carolina Pradier and Pierre Benz)

Exploring ChatGPT as a tool for automated classification of authors’ positivity perspectives on Russian topics (Carolina Coimbra Vieira, Elena Chechik and Victoria Di Césare)

Finding research topics related to specific authors with Retrieval-Augmented Generation (RAG) (Davide Pulizzotto, Anas Ramdani, Catherine Beaudry and Pietro Cruciata)

Predicting Citation Context Placement within IMRAD Structures Using Machine Learning (Francis Lareau)

12.30 Lunch

14.00 – 16.00 Technology and policy topics, Chair: Jens-Peter Andersen

The Impact of AI on SDG Classifications in Bibliometric Databases: A Conceptual Inquiry into Policy and Research Outcomes (Matteo Ottaviani and Stephan Stahlschmidt)

Understanding Future AI-Green Technology Directions from Past Technological Trajectories (Önder Nomaler, Tommaso Ciarli and Bart Verspagen)

Are large language models useful to support post-publication peer review? First empirical results of a research in progress study (Mengjia Wu, Yi Zhang, Robin Haunschild, Lutz Bornmann)

16.30-18.00 Public event. ChatGPT and research evaluation

Keynote speech 2. Evaluating societal impact of research with ChatGPT. Kayvan Kousha, University of Wolverhampton

Chair: Luca Maria Gambardella, Université della Svizzera italiana

Friday 07.02.2025

9.00 – 10.00 General topics, Chair: Frans Kaiser

Rise of Generative Artificial Intelligence in Science (Liangping Ding, Cornelia Lawson and Philip Shapira)

10.30 – 12.30 Generative AI for analyzing scientific careers, Chair: Benedetto Lepori

Research on Academic Careers and Generative AI: How Good Is ChatGPT in Providing Large-Scale Demographic and Career Data? (Lukasz Szymula and Marek Kwiek)

A new approach for collecting academic profession data: What can(not) ChatGPT yield? (Pinar Eldemir, Benedetto Lepori and Marek Kwiek)

Auditing Llama 3 for Academic Author Search (Daniele Barolo, Chiara Valentin, Fariba Karimi, Luis

Galarraga, Gonzalo Gabriel Mendez, Bruno Ribeiro and Lisette Espín-Noboa)

12.30 Lunch

13.30 – 15.00 Open discussion: good practices for the use of ChatGPT in science and technology studies: an overview, Chair: Benedetto Lepori.

Participants: Jean-Peter Andersen, Robin Haunschild, Frans Kaiser, Marek Kwiek.

15.00 Conclusion, Benedetto Lepori, Università della Svizzera italiana

15.30 Closure of the workshop

Scientific committee: Benedetto Lepori (Università della Svizzera italiana), Lutz Bornmann (Max Planck Society), Frans Kaiser (University of Twente), Wolfgang Glänzel (KU Leuven), Marek Kwiek (University of Poznan), Mike Thelwall (University of Sheffield), Marco Steenbergen (University of Zurich), Jens Peter Andersen (Aarhus University)

Organisation committee: Pınar Eldemir (Università della Svizzera italiana), Agata Lambrechts (Università della Svizzera italiana)

The organisers gratefully acknowledge the financial support provided by the Swiss National Science Foundation (SNSF) through the Scientific Exchanges scheme.