Tuesday, May 31, 2016

setup your own python pypi repository with bandersnatch and apache2

1- install apache2 and bandersnatch
$ sudo apt-get install bandersnatch
$ sudo pip install --upgrade bandersntach

2- configure bandersnatch by update file /etc/bandersnatch.conf as below
[mirror]
directory = /srv/pypi
master = https://pypi.python.org
timeout = 10
workers = 3
stop-on-error = false
delete-packages = true
[statistics]
access-log-pattern = /var/log/nginx/*.pypi.python.org*access*


3- create directory for storing and run the bandersnatch
$ sudo mkdir -p /srv/pypi
$ sudo bandersnatch mirror

4- wait until bandersnatch to finish, it might take very long time, because the whole repository can be exceed 400Gb. now we configure apache2 serve as a repository to client

- add the following line to /etc/apache2/apache2.conf
<Directory /srv>
        Options Indexes FollowSymLinks
        AllowOverride None
        Require all granted
</Directory>


- add s vhost /etc/apache2/sites-enabled/pypi.conf
<VirtualHost *:80>
    ServerName simple.pypi.repo
    DocumentRoot  /srv/pypi/web/
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
    <Directory  /srv/pypi/web/>
        Options Indexes FollowSymLinks MultiViews
                AllowOverride All
        Order allow,deny
        Allow from all
    </Directory>
</VirtualHost>


make sure thtat simple.pypi.repo is resolve to this server ip address

5- restart apache2 service and you get a pypi repository running
$ sudo /etc/init.d/apache2 restart

6- on client site we have to create ~/.pip/pip.conf to configure it to use our local repository
so create a directory ~/.pip and and file ~./.php/pip.conf with the following content
[global]
index-url = http://simple.pypi.repo/simple
trusted-host = simple.pypi.repo





so when you install new python module using pip it will no longer use pypi.python.org it will use simple.pypi.repo instead

- example install yolk
$ sudo pip install yolk
Collecting yolk
  Downloading http://simple.pypi.repo/packages/2b/c0/73510e50668b70f3fad25328c58d7a29a300a2e452058dfdcb7c7538af7b/yolk-0.4.3.tar.gz (86kB)
    100% |████████████████████████████████| 92kB 50.9MB/s
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/local/lib/python2.7/dist-packages (from yolk)
Building wheels for collected packages: yolk
  Running setup.py bdist_wheel for yolk ... done
  Stored in directory: /root/.cache/pip/wheels/33/ed/3b/20bf0f9285c9d267edca4ccadba42651156539643181e323c1
Successfully built yolk
Installing collected packages: yolk
Successfully installed yolk-0.4.3


No comments:

Post a Comment