Skip to content
#Science

Quantifying large language model usage in scientific papers (Category Science)

#Science #AI #Whitepaper #Humor

Interesting. whitepaper About the use of LLM in writing scientific articles. I came across it through a mailing list for ACM members. (Association for Computing Machinery)where reference was made to news According to Science, approximately one-fifth of scientific papers in computer science include texts generated by AI. I decided to look into the original article to see what the methodology looked like and what the results were. In the end, it turned out about the following:

  • It was a massive analysis.1.1M preprints and published articles from January 2020 September 2024 (arXiv, bioRxiv, Nature journals) The analysis found a steady increase in the use of LLM, especially in the field of computer science. 22 % of articles contain AI-modified text But with mathematics, everything is not so smooth - AI modifications approximately in 2 lower Modified articles have common features – they are short, written in popular research domains, and their authors actively publish preprints. (no-review)

If you think about it, the consequences are like this.

  1. Scientific papers will appear faster, but will be like twins - authors using LLM, choose developed areas in which there are already many developments and it is easier to get articles similar to the real ones. As a result, I anticipate a drop in the signal/noise ratio in such domains.
  2. To increase trust, it is possible to legalize the use of AI by agreeing on rules for the use and markup of parts to which the LLM has attached its pens.
  3. Pressure on the publishing infrastructure will increase and manual publication reviewers will not be able to cope with the vanguyu, which will soon automate the review and entrust them to especially smart LLMs, which will be able to prepare analysis of new whitepapers in peer-reviewed scientific journals. At best, there will be scientists in the loop to revisit the review of agents-reviewers. And it may turn out that articles and reviews will be written by LLM agents:)

#Science #AI #Whitepaper #Humor