DuckDuckGo

What if we applied the techniques Google applied to index the internet back in 1998 to the world of open source software? That's exactly the thought Andrew Nesbitt had in 2014 which lead to the creation of Libraries.io, an open source project for indexing other open source projects. This month Libraries.io released metadata on over 25 million open source projects. You can download it right now from Zenodo, but what can you do with it? To understand what is contained within this dataset, I'll take a quick look at how it's collected. Everything in Libraries.io begins with package managers. We index project metadata from 33 package managers, filling in gaps from their source repositories where we can. We parse project manifests—a gemfile, package.json, or similar—that includes code from other projects and stores the links between them...