Contrasting Linguistic Patterns in Human and LLM-Generated News Text.

Alberto Mu��oz-Ortiz, Carlos G��mez-Rodr��guez, David Vilares
Author Information
  1. Alberto Mu��oz-Ortiz: Universidade da Coru��a, CITIC, Departamento de Ciencias de la Computaci��n y Tecnolog��as de la Informaci��n, Campus de Elvi��a s/n, A Coru��a, 15071 A Coru��a Spain.
  2. Carlos G��mez-Rodr��guez: Universidade da Coru��a, CITIC, Departamento de Ciencias de la Computaci��n y Tecnolog��as de la Informaci��n, Campus de Elvi��a s/n, A Coru��a, 15071 A Coru��a Spain.
  3. David Vilares: Universidade da Coru��a, CITIC, Departamento de Ciencias de la Computaci��n y Tecnolog��as de la Informaci��n, Campus de Elvi��a s/n, A Coru��a, 15071 A Coru��a Spain.

Abstract

We conduct a quantitative analysis contrasting Human-written English news text with comparable large language model (LLM) output from six different LLMs that cover three different families and four sizes in total. Our analysis spans several measurable linguistic dimensions, including morphological, syntactic, psychometric, and sociolinguistic aspects. The results reveal various measurable differences between Human and AI-generated texts. Human texts exhibit more scattered sentence length distributions, more variety of vocabulary, a distinct use of dependency and constituent types, shorter constituents, and more optimized dependency distances. Humans tend to exhibit stronger negative emotions (such as fear and disgust) and less joy compared to text generated by LLMs, with the toxicity of these models increasing as their size grows. LLM outputs use more numbers, symbols and auxiliaries (suggesting objective language) than Human texts, as well as more pronouns. The sexist bias prevalent in Human text is also expressed by LLMs, and even magnified in all of them but one. Differences between LLMs and Humans are larger than between LLMs.

Keywords

References

  1. PeerJ Comput Sci. 2021 Apr 6;7:e443 [PMID: 33954234]
  2. Phys Rev E. 2022 Jan;105(1-1):014308 [PMID: 35193296]
  3. Phys Life Rev. 2017 Jul;21:171-193 [PMID: 28624589]
  4. Phys Rev E Stat Nonlin Soft Matter Phys. 2004 Nov;70(5 Pt 2):056135 [PMID: 15600720]
  5. JMIR Med Educ. 2023 Dec 28;9:e48904 [PMID: 38153785]
  6. Proc Natl Acad Sci U S A. 2015 Aug 18;112(33):10336-41 [PMID: 26240370]
  7. Trends Cogn Sci. 2022 Dec;26(12):1153-1170 [PMID: 36253221]
  8. Behav Res Methods. 2010 May;42(2):381-92 [PMID: 20479170]

Word Cloud

Created with Highcharts 10.0.0LLMstextlanguagehumantextsanalysisLLMdifferentmeasurableHumanexhibitusedependencymodelsLinguisticconductquantitativecontrastinghuman-writtenEnglishnewscomparablelargemodeloutputsixcoverthreefamiliesfoursizestotalspansseverallinguisticdimensionsincludingmorphologicalsyntacticpsychometricsociolinguisticaspectsresultsrevealvariousdifferencesAI-generatedscatteredsentencelengthdistributionsvarietyvocabularydistinctconstituenttypesshorterconstituentsoptimizeddistancesHumanstendstrongernegativeemotionsfeardisgustlessjoycomparedgeneratedtoxicityincreasingsizegrowsoutputsnumberssymbolsauxiliariessuggestingobjectivewellpronounssexistbiasprevalentalsoexpressedevenmagnifiedoneDifferenceshumanslargerContrastingPatternsLLM-GeneratedNewsTextComputationallinguisticsLargebiasesMachine-generated

Similar Articles

Cited By (2)