Sunday, February 1, 2015

Download all python package source from pypi mirror

I can not find any ftp or rsync server to download all the packages. So I have compiled a list. The list does not consist all the packages because some packages is linked to their own website, not in the pypi mirror.

Download the list and extract to a directory and use to the following script to download all that packages.


#!/usr/bin/python
import os
import re
plist = open("packagelist","r").read()

pl = plist.split("\n")
mirror1 = "http://pypi.python.org/packages/source/"
downlist = os.listdir("./")
n = 0
for i in pl:
  if i != "":
    iname = i.split("/")
    iname = iname[len(iname) - 1]
    if iname in downlist:
        n += 1
    else:
      try:
          os.system("wget " + mirror1 + newthing)
         
      except:
          pass
      finally:
          pass

print n




you can also change the mirror site that you can good speed.

http://www.pypi-mirrors.org/

No comments:

Post a Comment