Difference between revisions of "AI and Humans"

From GISAXS
Jump to: navigation, search
(→‎AI improves human work)
(→‎Creativity)
 
(32 intermediate revisions by the same user not shown)
Line 7: Line 7:
 
* 2023-11: [https://www.nature.com/articles/d41586-023-03507-3 ChatGPT has entered the classroom: how LLMs could transform education]
 
* 2023-11: [https://www.nature.com/articles/d41586-023-03507-3 ChatGPT has entered the classroom: how LLMs could transform education]
 
* 2025-04: [https://www.anthropic.com/news/anthropic-education-report-how-university-students-use-claude Anthropic Education Report: How University Students Use Claude]
 
* 2025-04: [https://www.anthropic.com/news/anthropic-education-report-how-university-students-use-claude Anthropic Education Report: How University Students Use Claude]
 +
* 2025-05: [https://www.nature.com/articles/s41599-025-04787-y The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a meta-analysis]
  
 
==AI improves learning/education==
 
==AI improves learning/education==
Line 18: Line 19:
 
* [https://arxiv.org/abs/2402.09809 Effective and Scalable Math Support: Evidence on the Impact of an AI- Tutor on Math Achievement in Ghana]
 
* [https://arxiv.org/abs/2402.09809 Effective and Scalable Math Support: Evidence on the Impact of an AI- Tutor on Math Achievement in Ghana]
 
* [https://doi.org/10.21203/rs.3.rs-4243877/v1 AI Tutoring Outperforms Active Learning]
 
* [https://doi.org/10.21203/rs.3.rs-4243877/v1 AI Tutoring Outperforms Active Learning]
−
* [https://blogs.worldbank.org/en/education/From-chalkboards-to-chatbots-Transforming-learning-in-Nigeria From chalkboards to chatbots: Transforming learning in Nigeria, one prompt at a time]
+
* [https://documents.worldbank.org/en/publication/documents-reports/documentdetail/099548105192529324 From chalkboards to chatbots: Transforming learning in Nigeria, one prompt at a time] ([https://blogs.worldbank.org/en/education/From-chalkboards-to-chatbots-Transforming-learning-in-Nigeria writeup])
 
** 6 weeks of after-school AI tutoring = 2 years of typical learning gains
 
** 6 weeks of after-school AI tutoring = 2 years of typical learning gains
 
** outperforms 80% of other educational interventions
 
** outperforms 80% of other educational interventions
Line 53: Line 54:
  
 
==Detection==
 
==Detection==
−
* [https://www.sciencedirect.com/science/article/pii/S2666920X24000109 Do teachers spot AI? Evaluating the detectability of AI-generated texts among student essays]
+
* 2024-06: [https://www.sciencedirect.com/science/article/pii/S2666920X24000109 Do teachers spot AI? Evaluating the detectability of AI-generated texts among student essays]
 
** GenAI can simulate student writing in a way that teachers cannot detect.
 
** GenAI can simulate student writing in a way that teachers cannot detect.
 
** AI essays are assessed more positively than student-written.
 
** AI essays are assessed more positively than student-written.
 
** Teachers are overconfident in their source identification.
 
** Teachers are overconfident in their source identification.
 
** Both novice and experienced teachers could not identify texts generated by ChatGPT vs. students
 
** Both novice and experienced teachers could not identify texts generated by ChatGPT vs. students
 +
* 2025-01: [https://arxiv.org/abs/2501.15654 People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text]
 
===AI Text Detectors Don't Work===
 
===AI Text Detectors Don't Work===
 
* 2024-05: [https://arxiv.org/abs/2405.07940 RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors]
 
* 2024-05: [https://arxiv.org/abs/2405.07940 RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors]
Line 84: Line 86:
  
 
===Creativity===
 
===Creativity===
 +
* See also: [[AI creativity]]
 
* 2023-07: [https://mackinstitute.wharton.upenn.edu/wp-content/uploads/2023/08/LLM-Ideas-Working-Paper.pdf Ideas Are Dimes A Dozen: Large Language Models For Idea Generation In Innovation]
 
* 2023-07: [https://mackinstitute.wharton.upenn.edu/wp-content/uploads/2023/08/LLM-Ideas-Working-Paper.pdf Ideas Are Dimes A Dozen: Large Language Models For Idea Generation In Innovation]
 
* 2023-09: [https://www.nature.com/articles/s41598-023-40858-3 Best humans still outperform artificial intelligence in a creative divergent thinking task]
 
* 2023-09: [https://www.nature.com/articles/s41598-023-40858-3 Best humans still outperform artificial intelligence in a creative divergent thinking task]
Line 95: Line 98:
 
** LLMs can be creative
 
** LLMs can be creative
 
* 2024-09: [https://docs.iza.org/dp17302.pdf Creative and Strategic Capabilities of Generative AI: Evidence from Large-Scale Experiments]
 
* 2024-09: [https://docs.iza.org/dp17302.pdf Creative and Strategic Capabilities of Generative AI: Evidence from Large-Scale Experiments]
 +
* 2025-06: [https://arxiv.org/abs/2506.00794 Predicting Empirical AI Research Outcomes with Language Models]
  
 
===Art===
 
===Art===
Line 100: Line 104:
 
* 2024-11: [https://www.astralcodexten.com/p/how-did-you-do-on-the-ai-art-turing How Did You Do On The AI Art Turing Test?]
 
* 2024-11: [https://www.astralcodexten.com/p/how-did-you-do-on-the-ai-art-turing How Did You Do On The AI Art Turing Test?]
  
−
===Marketing===
+
===Business & Marketing===
 
* 2023-11: [https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4597899 The power of generative marketing: Can generative AI create superhuman visual marketing content?]
 
* 2023-11: [https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4597899 The power of generative marketing: Can generative AI create superhuman visual marketing content?]
 +
* 2024-02: [https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4714776 Generative Artificial Intelligence and Evaluating Strategic Decisions]
  
 
===Professions===
 
===Professions===
 
* [https://agi.safe.ai/submit Humanity's Last Exam]
 
* [https://agi.safe.ai/submit Humanity's Last Exam]
 
** [https://x.com/alexandr_wang/status/1835738937719140440 Effort to build] a dataset of challenging (but resolvable) questions in specific domain areas, to act as a benchmark to test whether AIs are improving in these challenging topics.
 
** [https://x.com/alexandr_wang/status/1835738937719140440 Effort to build] a dataset of challenging (but resolvable) questions in specific domain areas, to act as a benchmark to test whether AIs are improving in these challenging topics.
 +
 +
====Coding====
 +
* 2025-02: [https://arxiv.org/abs/2502.06807 Competitive Programming with Large Reasoning Models]
  
 
====Medical====
 
====Medical====
Line 129: Line 137:
 
* 2025-04: [https://www.nature.com/articles/s41586-025-08866-7?linkId=13898052 Towards conversational diagnostic artificial intelligence]
 
* 2025-04: [https://www.nature.com/articles/s41586-025-08866-7?linkId=13898052 Towards conversational diagnostic artificial intelligence]
 
* 2025-04: [https://www.nature.com/articles/s41586-025-08869-4?linkId=13898054 Towards accurate differential diagnosis with large language models]
 
* 2025-04: [https://www.nature.com/articles/s41586-025-08869-4?linkId=13898054 Towards accurate differential diagnosis with large language models]
 +
 +
====Bio====
 +
* 2025-04: [https://www.virologytest.ai/vct_paper.pdf Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark]
 +
** Time: [https://time.com/7279010/ai-virus-lab-biohazard-study/ Exclusive: AI Outsmarts Virus Experts in the Lab, Raising Biohazard Fears]
 +
** AI Frontiers: [https://www.ai-frontiers.org/articles/ais-are-disseminating-expert-level-virology-skills AIs Are Disseminating Expert-Level Virology Skills]
  
 
====Therapy====
 
====Therapy====
Line 162: Line 175:
  
 
===Medical===
 
===Medical===
−
* 2025-03L: [https://www.medrxiv.org/content/10.1101/2025.02.28.25323115v1.full Medical Hallucination in Foundation Models and Their Impact on Healthcare]
+
* 2025-03: [https://www.medrxiv.org/content/10.1101/2025.02.28.25323115v1.full Medical Hallucination in Foundation Models and Their Impact on Healthcare]
 +
* 2025-03: [https://journals.lww.com/international-journal-of-surgery/fulltext/2025/03000/chatgpt_s_role_in_alleviating_anxiety_in_total.20.aspx ChatGPT’s role in alleviating anxiety in total knee arthroplasty consent process: a randomized controlled trial pilot study]
 +
* 2025-05: [https://openai.com/index/healthbench/ Introducing HealthBench]
  
 
===Translation===
 
===Translation===
Line 171: Line 186:
  
 
===Creativity===
 
===Creativity===
 +
* See also: [[AI creativity]]
 
* 2024-07: [https://www.science.org/doi/10.1126/sciadv.adn5290 Generative AI enhances individual creativity but reduces the collective diversity of novel content]
 
* 2024-07: [https://www.science.org/doi/10.1126/sciadv.adn5290 Generative AI enhances individual creativity but reduces the collective diversity of novel content]
 
* 2024-08: [https://www.nature.com/articles/s41562-024-01953-1 An empirical investigation of the impact of ChatGPT on creativity]
 
* 2024-08: [https://www.nature.com/articles/s41562-024-01953-1 An empirical investigation of the impact of ChatGPT on creativity]
 +
** 2024-08: Response: [https://www.nature.com/articles/s41562-024-01953-1 ChatGPT decreases idea diversity in brainstorming] ([https://www.nature.com/articles/s41562-025-02173-x.epdf?sharing_token=LA9NyDHj7y5WN8zvb5Qm49RgN0jAjWel9jnR3ZoTv0Nl8PrpXFkjZ93XvmUVBgB9Hlfro5Yo6YELr-pRqbpk3HaZENCvsfV8G1kwtTEj2oW1g87dSVT4BzrfCu3jS_606SLzmoDuDiALChY-MozVM4Pj1b4Vdf-YaIH5p3lfAnM%3D pdf])
 +
** 2025-05: Response: [https://www.nature.com/articles/s41562-025-02195-5 Reply to: ChatGPT decreases idea diversity in brainstorming]
 
* 2024-08: [https://doi.org/10.1287/orsc.2023.18430 The Crowdless Future? Generative AI and Creative Problem-Solving]
 
* 2024-08: [https://doi.org/10.1287/orsc.2023.18430 The Crowdless Future? Generative AI and Creative Problem-Solving]
 
* 2024-10: [https://arxiv.org/abs/2410.03703 Human Creativity in the Age of LLMs]
 
* 2024-10: [https://arxiv.org/abs/2410.03703 Human Creativity in the Age of LLMs]
−
* 2024-11: [https://conference.nber.org/conf_papers/f210475.pdf Artificial Intelligence, Scientific Discovery, and Product Innovation]: diffusion model increases "innovation" (patents), boosts the best performers, but also removes some enjoyable tasks.
+
* 2024-11: <strike>[https://conference.nber.org/conf_papers/f210475.pdf Artificial Intelligence, Scientific Discovery, and Product Innovation]</strike>: diffusion model increases "innovation" (patents), boosts the best performers, but also removes some enjoyable tasks.
 +
** 2025-05: Retraction: [https://economics.mit.edu/news/assuring-accurate-research-record Assuring an accurate research record]
 
* 2024-12: [https://doi.org/10.1080/10400419.2024.2440691 Using AI to Generate Visual Art: Do Individual Differences in Creativity Predict AI-Assisted Art Quality?] ([https://osf.io/preprints/psyarxiv/ygzw6 preprint]): shows that more creative humans produce more creative genAI outputs
 
* 2024-12: [https://doi.org/10.1080/10400419.2024.2440691 Using AI to Generate Visual Art: Do Individual Differences in Creativity Predict AI-Assisted Art Quality?] ([https://osf.io/preprints/psyarxiv/ygzw6 preprint]): shows that more creative humans produce more creative genAI outputs
 
* 2025-01: [https://arxiv.org/abs/2501.11433 One Does Not Simply Meme Alone: Evaluating Co-Creativity Between LLMs and Humans in the Generation of Humor]
 
* 2025-01: [https://arxiv.org/abs/2501.11433 One Does Not Simply Meme Alone: Evaluating Co-Creativity Between LLMs and Humans in the Generation of Humor]
Line 186: Line 205:
 
* 2025-03: [https://dam-prod2.media.mit.edu/x/2025/03/21/Randomized_Control_Study_on_Chatbot_Psychosocial_Effect.pdf How AI and Human Behaviors Shape Psychosocial Effects of Chatbot Use: A Longitudinal Controlled Study]
 
* 2025-03: [https://dam-prod2.media.mit.edu/x/2025/03/21/Randomized_Control_Study_on_Chatbot_Psychosocial_Effect.pdf How AI and Human Behaviors Shape Psychosocial Effects of Chatbot Use: A Longitudinal Controlled Study]
  
−
==AI worse==
+
==AI worse than humans==
−
tbd
+
* 2025-04: [https://spinup-000d1a-wp-offload-media.s3.amazonaws.com/faculty/wp-content/uploads/sites/27/2025/03/AI-debt-collection-20250331.pdf How Good is AI at Twisting Arms? Experiments in Debt Collection]
 +
* 2025-04: [https://arxiv.org/abs/2504.18919 Clinical knowledge in LLMs does not translate to human interactions]
 +
* 2025-05: [https://royalsocietypublishing.org/doi/10.1098/rsos.241776 Generalization bias in large language model summarization of scientific research]
  
 
==Human Perceptions of AI==
 
==Human Perceptions of AI==
Line 201: Line 222:
 
* 2024-07: [https://arxiv.org/abs/2407.08853 GPT-4 is judged more human than humans in displaced and inverted Turing tests]
 
* 2024-07: [https://arxiv.org/abs/2407.08853 GPT-4 is judged more human than humans in displaced and inverted Turing tests]
 
* 2025-03: [https://arxiv.org/abs/2503.23674 Large Language Models Pass the Turing Test]
 
* 2025-03: [https://arxiv.org/abs/2503.23674 Large Language Models Pass the Turing Test]
 +
* 2025-04: [https://www.sciencedirect.com/science/article/abs/pii/S0022103117303980 A Minimal Turing Test]
 +
 
'''Art'''
 
'''Art'''
 
* 2024-11: [https://www.astralcodexten.com/p/how-did-you-do-on-the-ai-art-turing How Did You Do On The AI Art Turing Test?] Differentiation was only slightly above random (60%). AI art was often ranked higher than human-made.
 
* 2024-11: [https://www.astralcodexten.com/p/how-did-you-do-on-the-ai-art-turing How Did You Do On The AI Art Turing Test?] Differentiation was only slightly above random (60%). AI art was often ranked higher than human-made.
Line 225: Line 248:
 
* 2025-02: [https://arxiv.org/abs/2502.09747 The Widespread Adoption of Large Language Model-Assisted Writing Across Society]: 10-25% adoption across a range of contexts
 
* 2025-02: [https://arxiv.org/abs/2502.09747 The Widespread Adoption of Large Language Model-Assisted Writing Across Society]: 10-25% adoption across a range of contexts
 
* 2025-02: [https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5078805 Local Heterogeneity in Artificial Intelligence Jobs Over Time and Space]
 
* 2025-02: [https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5078805 Local Heterogeneity in Artificial Intelligence Jobs Over Time and Space]
 +
* 2025-04: [https://andreyfradkin.com/assets/demandforllm.pdf Demand for LLMs: Descriptive Evidence on Substitution, Market Expansion, and Multihoming]
 +
* 2025-05: [https://civicscience.com/chatgpt-is-still-leading-the-ai-wars-but-google-gemini-is-gaining-ground/ ChatGPT Is Still Leading the AI Wars but Google Gemini Is Gaining Ground]
 +
* 2025-05: [https://www.nber.org/papers/w33777 Large Language Models, Small Labor Market Effects]
 +
** Significant uptake, but very little economic impact so far
 +
* 2025-05: [https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5136877 The Labor Market Effects of Generative Artificial Intelligence]
 +
** US worker usage of AI increasingly rapidly: 30% in 2024-12; 40% in 2025-05
 +
* 2025-05: [https://www.bondcap.com/report/pdf/Trends_Artificial_Intelligence.pdf Trends – Artificial Intelligence]
  
 
==Usage For==
 
==Usage For==
 
* 2024-12: [https://assets.anthropic.com/m/7e1ab885d1b24176/original/Clio-Privacy-Preserving-Insights-into-Real-World-AI-Use.pdf Clio: A system for privacy-preserving insights into real-world AI use] (Anthropic [https://www.anthropic.com/research/clio Clio])
 
* 2024-12: [https://assets.anthropic.com/m/7e1ab885d1b24176/original/Clio-Privacy-Preserving-Insights-into-Real-World-AI-Use.pdf Clio: A system for privacy-preserving insights into real-world AI use] (Anthropic [https://www.anthropic.com/research/clio Clio])
 +
* 2025-03: [https://learn.filtered.com/hubfs/The%202025%20Top-100%20Gen%20AI%20Use%20Case%20Report.pdf How People are Really Using Generative AI Now] ([https://hbr.org/2025/04/how-people-are-really-using-gen-ai-in-2025 writeup])
 +
* 2025-04: [https://www.anthropic.com/news/anthropic-education-report-how-university-students-use-claude Anthropic Education Report: How University Students Use Claude]
 +
 +
==Hiding Usage==
 +
* 2025-05: [https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5232910 Underreporting of AI use: The role of social desirability bias]
  
 
=Sentiment=
 
=Sentiment=
Line 239: Line 274:
 
* 2024-09: [https://www.science.org/doi/10.1126/science.adq1814 Durably reducing conspiracy beliefs through dialogues with AI]
 
* 2024-09: [https://www.science.org/doi/10.1126/science.adq1814 Durably reducing conspiracy beliefs through dialogues with AI]
 
* 2025-03: [https://www.pnas.org/doi/10.1073/pnas.2413443122 Scaling language model size yields diminishing returns for single-message political persuasion]
 
* 2025-03: [https://www.pnas.org/doi/10.1073/pnas.2413443122 Scaling language model size yields diminishing returns for single-message political persuasion]
 +
* 2025-04: [https://drive.google.com/file/d/1Eo4SHrKGPErTzL1t_QmQhfZGU27jKBjx/edit Can AI Change Your View? Evidence from a Large-Scale Online Field Experiment]
 +
** [https://www.404media.co/researchers-secretly-ran-a-massive-unauthorized-ai-persuasion-experiment-on-reddit-users/ Researchers Secretly Ran a Massive, Unauthorized AI Persuasion Experiment on Reddit Users]
 +
* 2025-05: [https://arxiv.org/abs/2505.09662 Large Language Models Are More Persuasive Than Incentivized Human Persuaders]
  
 
=Simulate Humans=
 
=Simulate Humans=
Line 250: Line 288:
 
* 2024-10: [https://www.pnas.org/doi/10.1073/pnas.2407639121 Large Language Models based on historical text could offer informative tools for behavioral science]
 
* 2024-10: [https://www.pnas.org/doi/10.1073/pnas.2407639121 Large Language Models based on historical text could offer informative tools for behavioral science]
 
* 2025-04: [https://arxiv.org/abs/2504.02234 LLM Social Simulations Are a Promising Research Method]
 
* 2025-04: [https://arxiv.org/abs/2504.02234 LLM Social Simulations Are a Promising Research Method]
 +
* 2025-04: [https://www.nber.org/papers/w33662 Measuring Human Leadership Skills with AI Agents]
 +
* 2025-04: [https://arxiv.org/abs/2504.10157 SocioVerse: A World Model for Social Simulation Powered by LLM Agents and A Pool of 10 Million Real-World Users]
  
 
=See Also=
 
=See Also=
 
* [https://www.google.com/books/edition/_/cKnYEAAAQBAJ?hl=en&gbpv=1&pg=PA2 UNESCO. Guidance for Generative AI in Education and Research]
 
* [https://www.google.com/books/edition/_/cKnYEAAAQBAJ?hl=en&gbpv=1&pg=PA2 UNESCO. Guidance for Generative AI in Education and Research]
 
* [[AI]]
 
* [[AI]]
 +
** [[AI predictions]]

Latest revision as of 19:52, 4 June 2025

AI in Education

Survey/study of

AI improves learning/education

AI harms learning

Software/systems

LLMs

Individual tools

Systems

AI for grading

Detection

AI Text Detectors Don't Work

AI/human

Capabilities

Writing

AI out-performs humans

Tests

Creativity

Art

Business & Marketing

Professions

  • Humanity's Last Exam
    • Effort to build a dataset of challenging (but resolvable) questions in specific domain areas, to act as a benchmark to test whether AIs are improving in these challenging topics.

Coding

Medical

Bio

Therapy

Financial

AI improves human work

Coding

Forecasting

Finance

Law

Medical

Translation

Customer service

  • 2023-11: Generative AI at Work: Improvements for workers and clients (though also a ceiling to improvement)

Creativity

Equity

Counter loneliness

AI worse than humans

Human Perceptions of AI

AI passes Turing Test

Text Dialog

Art

Psychological Effects of AI Usage

Uptake

Usage For

Hiding Usage

Sentiment

Persuasion

(AI can update beliefs, change opinions, tackle conspiracy theories, etc.)

Simulate Humans

See Also

Retrieved from "http://gisaxs.com/index.php?title=AI_and_Humans&oldid=7879"