Comments on: How do you migrate a 20GB site?

By: Load Philippines

Load Philippines — Fri, 12 Feb 2010 07:56:25 +0000

I just adore stumbleupon for being able to show me this web page. instant fan now. I have viewed some definitely poor websites on stumble and when I discovered this, I immediately knew it was excellent! This blogpost is informative specifically for somebody like me haha. I’ve been attempting to read a lot more these days and being able to discover a web page like this can be excellent for my education. Love what you are doing, keep it up!

By: hoop

hoop — Fri, 20 Jan 2006 10:39:06 +0000

Ah the ever reliable SSH, nice to know it can handle that much data without any corruption or termination… this is a tip that will come in handy in the future

By: bimbo

bimbo — Wed, 18 Jan 2006 09:55:45 +0000

well it certainly took its time!

By: yuga

yuga — Wed, 18 Jan 2006 07:41:20 +0000

What we did was to backup the whole site via ssh (./scripts/pkgacct) which resulted into a tarball of about 17GB file. Copying took about 5 hours.

Once in the ne server, we did a manual restore via SSH again (./scripts/restorepkg) and the un-packaging took about 2 hours and the restore another 4 hours.

During this period, if your own net connection blips for one second, the process aborts, which happened once.

By: markku

markku — Wed, 18 Jan 2006 06:53:47 +0000

There are FTP clients that can do FTP-to-FTP transfers. I’m sure that would take some time, but possibly the best solution right now, without requiring you to work on the site immediately.

By: hoop

hoop — Mon, 16 Jan 2006 19:48:44 +0000

I usually run optimize table in phpmyadmin to defragment the tables and reclaim some unused disk space. This might help to bring that DB down further

Maybe you should work with the smaller tables first and see how much it brings down the file size…

I’d suggest preventing access to the DB while optimizing if incase you go down this path to prevent corruption of the DB.

then once you’ve uploaded the DB to the new server you can dump the old DB so that all you have to worry about is the 1gb classifieds and 1.5 photofiles

By: kzap

kzap — Mon, 16 Jan 2006 09:22:27 +0000

how bout clean it up first, im sure theres a lot of unneeded stuff there

By: demonhale

demonhale — Mon, 16 Jan 2006 02:04:47 +0000

Hmmm… that is a problem really, post about your solutions will ya?

Anyways heres an idea, I worked with a big site before (I just dont want to mention their URL coz their a private internal site) .. Which had their own hosting, but the Outer parts for public visitors are hosted by an external host located cross-country away. I issued an order to the dept. to send a request to the external host to send us a collection of the backup files written in discs and labeled appropriately, boxed and sent via a freight service. We paid for the overhead cost as well as service fee (affordable considering all the work that needs to be done) …

By: yuga

yuga — Sun, 15 Jan 2006 20:18:35 +0000

Here’s a quick breakdown: 500MB DB, 4GB forum files, 1GB classified ads files, 1.5GB photo gallery files, then there are the attachments and other core site files.

By: hoop

hoop — Sun, 15 Jan 2006 20:02:04 +0000

1. maybe you can try exporting the tablespaces piecemeal for that particular DB via FTP if it’s below 2 GB per table.

2. Use phpmyadmin to export the tables individually

unless it’s not the tablespaces/DB that comprise the entire 20GB

By: Bong

Bong — Sun, 15 Jan 2006 18:32:18 +0000

all the best and good luck!