Friday, July 23, 2010

debmirroring

+++++++++++++++++++++++++++++++++++
DISTS="hardy,hardy-backports,hardy-security,hardy-updates"
SECTION="main,multiverse,restricted,universe"
ARCH="i386"
debmirror --ignore-release-gpg --verbose --method=ftp --host=mirror.its.ac.id \
--dist=$DISTS --section=$SECTION \
--arch=$ARCH --nosource --root=ubuntu \
--diff=mirror --ignore-small-errors /hardy
+++++++++++++++++++++++++++++++++++

DISTS="lucid,lucid-backports,lucid-security,lucid-updates"
SECTION="main,multiverse,restricted,universe"
ARCH="i386"
debmirror --verbose --method=http --host=kambing.ui.ac.id --dist=$DISTS --section=$SECTION --arch=$ARCH --nosource --root=ubuntu --ignore-release-gpg /home/globus/mnt/3/lucid
+++++++++++++++++++++++++++++++++++

save one of those on the file then execute it

4 comments:

  1. To build a mirror with debmirror follow these steps

    This is not covering all use cases but it will allow you experiment. To do this safely 8.2 gigs of space are needed to mirror main source and binaries. This number grows a lot as time progresses.

    Alternative Consideration

    If you want to download all the apt binaries for all boxes onto one box on a network then maybe apt-cache or a transparent proxy (eg http-replicator is all you need.

    Another method is to use rsync. The Rsyncmirror page goes into greater detail on this method. Rsync can create a full mirror of a site and selects files indiscriminately. Debmirror, however, does allow for better control over which packages are mirrored and can be easier on the amount of hard drive space required.

    Start The Mirror Build Process

    Open gnome-terminal by clicking Applications -> Accessories -> Terminal.
    If your sources list has universe enabled, skip the next step.
    Open your sources list (sudo sensible-editor /etc/apt/sources.list), and add the following line
    deb http://cc.archive.ubuntu.com/ubuntu/ dapper universe
    Then save and return to a prompt.
    In the example above, cc is your country's code (e.g., au means Australia), and dapper is replaced with the release you are using.

    Update the package lists (sudo apt-get update) and then install debmirror (sudo apt-get install debmirror).
    Now we have to create a directory to save the files to. The approximate size of the components of the mirror (i386 only) are: # Changes over time.

    feisty


    ~27G


    main, restricted, universe, multiverse

    dapper


    15G


    main, restricted, universe, multiverse

    Please ensure where ever you make the mirror has room for the packages you want to download!
    (If you are more advanced you can make different parts of the mirror on different volumes, but that is not something for this howto).
    We will make our mirror on /home/ with

    sudo mkdir /home/UbuntuMirror

    Create the file `mirrorbuild.sh`

    Now to create mirrorbuild.sh
    Open your prefered editor thus

    sudo sensible-editor /usr/local/bin/mirrorbuild.sh

    Copy the text from the box below into the editor.
    You may omit the lines starting with # (EXCEPTION: the line that starts #!/bin/bash must be left in.)

    ReplyDelete
  2. Contents of the mirror script, please adapt it to your needs

    #### Start script to automate building of Ubuntu mirror #####
    ## THE NEXT LINE IS NEEDED THE REST OF THE LINES STARTING WITH A # CAN BE DELETED

    #!/bin/bash

    ## Setting variables with explanations.

    #
    # Don't touch the user's keyring, have our own instead
    #
    export GNUPGHOME=/home/mirrorkeyring

    # Arch= -a # Architecture. For Ubuntu can be i386, powerpc or amd64.
    # sparc, only starts in dapper, it is only the later models of sparc.
    #
    arch=i386

    # Minimum Ubuntu system requires main, restricted
    # Section= -s # Section (One of the following - main/restricted/universe/multiverse).
    # You can add extra file with $Section/debian-installer. ex: main/debian-installer,universe/debian-installer,multiverse/debian-installer,restricted/debian-installer
    #
    section=main,restricted,universe,multiverse

    # Release= -d # Release of the system (Dapper, Edgy, Feisty, Gutsy, Hardy, Intrepid), and the -updates and -security ( -backports can be added if desired)
    #
    release=dapper,dapper-security,dapper-updates

    # Server= -h # Server name, minus the protocol and the path at the end
    # CHANGE "*" to equal the mirror you want to create your mirror from. au. in Australia ca. in Canada.
    # This can be found in your own /etc/apt/sources.list file, assuming you have Ubuntu installed.
    #
    server=cc.archive.ubuntu.com

    # Dir= -r # Path from the main server, so http://my.web.server/$dir, Server dependant
    #
    inPath=/ubuntu

    # Proto= -e # Protocol to use for transfer (http, ftp, hftp, rsync)
    # Choose one - http is most usual the service, and the service must be avaialbe on the server you point at.
    #
    proto=http

    # Outpath= # Directory to store the mirror in
    # Make this a full path to where you want to mirror the material.
    #
    outPath=/home/UbuntuMirror

    # The --nosource option only downloads debs and not deb-src's
    # The --progress option shows files as they are downloaded
    # --source \ in the place of --no-source \ if you want sources also.
    # --nocleanup Do not clean up the local mirror after mirroring is complete. Use this option to keep older repository
    # Start script
    #
    debmirror -a $arch \
    --no-source \
    -s $section \
    -h $server \
    -d $release \
    -r $inPath \
    --progress \
    -e $proto \
    $outPath


    #### End script to automate building of Ubuntu mirror ####

    [..continue..]

    ReplyDelete
  3. How to use the file

    * Save the file and exit.
    * Now change the permissions and ownership on the mirror. The group name is your username:

    sudo chown -R root:username /home/UbuntuMirror
    sudo chmod -R 571 /home/UbuntuMirror

    * Set up your keyring to correctly verify Release signatures

    for a mirror host running feisty:

    sudo mkdir /home/mirrorkeyring
    gpg --no-default-keyring --keyring /home/mirrorkeyring/pubring.gpg --import /usr/share/keyrings/ubuntu-archive-keyring.gpg

    for a mirror host running gutsy:

    sudo mkdir /home/mirrorkeyring
    gpg --no-default-keyring --keyring /home/mirrorkeyring/trustedkeys.gpg --import /usr/share/keyrings/ubuntu-archive-keyring.gpg

    * And the final part of setting up the mirror is to download the files:

    mirrorbuild.sh

    Now walk away. Your machine has a lot of downloading to do! Seriously... it'll take hours.

    Set up the mirror

    We need to install Apache2. You could choose Apache version 1.3 but that is beyond the scope of this document.
    We can do this with sudo apt-get install apache2

    Then (for simplicity sake) lets make a link from /home/UbuntuMirror to /var/www/ubuntu.

    ln -s /home/UbuntuMirror /var/www/ubuntu

    This means when you go to download from your mirror, you will visit http://mirrorbox/ubuntu/.

    Right! So we have all the packages, and a working server. Now we need to set up the clients.

    Add a cronjob to keep the mirror updated
    [..continue..]

    ReplyDelete
  4. If you want to keep your mirror fresh, you will want to stick your mirror script in cron. To edit root's crontab, do:

    sudo crontab -e

    And now add the following line:

    2 */12 * * * /usr/local/bin/mirrorbuild.sh

    Which will run two minutes after the hour every twelve hours. MAKE SURE that there is a blank line after the line you added, otherwise it will never run. If you want to be paranoid, restart cron:

    sudo /etc/init.d/cron restart

    Set up a client system

    Open a terminal and enter

    cd /etc/apt
    sudo mv sources.list sources.list.orig
    sudo sensible-editor sources.list

    Now in your editor, put the following lines:

    # Local network mirror sources.
    deb http://mirrorbox/ubuntu dapper main restricted universe multiverse
    deb http://mirrorbox/ubuntu dapper-updates main restricted universe multiverse
    deb http://mirrorbox/ubuntu dapper-security main restricted universe multiverse

    Then save and exit
    If you then run

    sudo apt-get update
    sudo apt-get dist-upgrade

    you should be updating from your new server!

    ReplyDelete