Cloning the Entire Drupal Repository to a Local Server

If you need to clone every project from Drupal.org you can do so using a quick one-liner and drush!

Although it’s not required, it is a good idea to start by installing a base install of Drupal using the latest project code from Drupal.org.

You will also need to have drush installed on your machine as drush will be used to download each project.

Once you have your clean install ready, navigate to your site directory and use enter the following command:

curl http://drupal.org/project/usage | perl -nle 'print "$2" if /(<a href=.*usage\/(.*)">.*<\/a>)/' >project_list.txt

This will create a file called project_list.txt, which will contain a list of every project for every contributed repository.

This file will need to be edited to remove the first line, which is the core project for Drupal. Failure to do so will result in error.

Once the file has been edited and the line containing Drupal has been removed, you will then need to use the following command to download all the projects:

drush --no dl `cat project_list.txt` -y

Depending on the version of Drupal your base install, Drush will download all supported releases for your installed version.

If you wish to update the modules, you could either repeat the process or use the drush up command.

At the time of creating this post, the list contained 11,286 projects including both themes and modules. Although some entries will be skipped due to non-supported releases and non-compatible modules, but make no mistake the download takes time and it will use a decent amount of space on your hard disk.

I would NOT recommend performing this operation on a live site, and only performed in testing environments.

Thanks for reading and Happy Drupaling!

About the author

Will works as a technical lead for Kanopi Studios and provides Drupal support for a wide range of amazing projects. He current resides in South Carolina along with his wife, son and two dogs. When not working on tech projects, Will enjoys spending time with family and photographing the stars.