LibGuides: Electrical Engineering & Computer Science: Data sources for ML and AI

Additional guides

Other tools

label errors
A project from MIT to identify and label errors in ML test sets and software to validate test sets.

General sources for datasets related to ML and AI

Papers with Code
Growing resource of papers, code and datasets related to machine learning and AI.
UCI Machine learning repository
A wide variety of types of data specifically for ML research.
Google dataset search
Google's dataset search for finding a variety of datasets across fields

Sources for linguistic, textual and bibliographic data

The libraries supports access to APIs from many sources and publishers for textual data and bibliographic metadata. Use this page to find out more. Different publishers will have different use restrictions and procedures for accessing this data. In addition, we support access to several linguistic corpora. Licenses may also vary for re-use and machine learning use. If you have questions about using any of these sources for NLP or machine learning projects or other work, please feel free to contact me with the details of your project.

COCA
Largest structured corpus of American English composed of more than 450 million words in 189,431 texts, including 20 million words each year from 1990-2012. The corpus is equally divided among spoken, fiction, popular magazines, newspapers, and academic texts. [restricted to MIT]
COHA
Corpus of Historical American English (COHA) is composed of more than 400 million words of text in more than 100,000 individual texts. [restricted to MIT]
Linguistics data consortium (LDC)
A variety of datasets of spoken and written language. MIT Libraries provides access to LDC data corpora published in 2016 and after. See our guide at https://libguides.mit.edu/ldc for information on how to access. [restricted to MIT]
WordNet
A lexical database for English.
OPUS translation data
Open collection of translated texts from the web
bibliographic data
For bibliographic data (metadata about scientific articles and books, or journal article full-text) see this guide to specific publisher APIs, and contact us with the details of your project if you are looking for a specific dataset.

Sources for other specific types of data

Moments in Time
A large-scale dataset for recognizing and understanding action in videos, produced at MIT.
Physionet
Physionet, produced at MIT, is a repository for a large variety of types of medical data, from many sources; access restrictions vary by dataset.
CORD19
CORD19 is a dataset of papers about Covid-19.
MIT Supercloud dataset
An HPC cluster dataset for the research and development of innovative AI/ML techniques to that will facilitate the development of tools to detect, explain, recover, and potentially predict outliers and ultimately mitigate system failure in complex multi-sensor systems.
Objaverse
A dataset of 3D objects, with annotations.
Social Media Archive
Archive of datasets from various social media platforms, hosted by ICPSR

EECS Librarian

Phoebe Ayers

Contact:

psayers@mit.edu
Room 10-500
617.253.4442

How can Phoebe help you?