Register Members List Search Today's Posts Mark Forums Read

Reply
 
Mod Options
  #1  
Old 03 Feb 2008, 16:17
blackvborg blackvborg is offline
 
Join Date: Dec 2006
Replicate attachments/images among servers

Hello,
ho do you handle the replication of images/attachments among multiple servers to achive HA/LB ?

If you've multiple frontend web servers for your forum you've the problem of "sharing" attachments. Using rsync could be a good solution at the beginning but it's not efficient with hundred of thousand of files, moreover some webservers haven't the new images until the replication is complete.

I've read that some of you use a dedicated attachment server, but it's a single point of failure and for load balancing has the same problem as mentioned above (the lag before sync).
Any idea? I think a SAN/NAS is not a very affordable solution, NFS has other drawbacks

Thank you for sharing your solutions!
Reply With Quote
Comments
  #2  
Old 03 Feb 2008, 16:43
Brandon Sheley's Avatar
Brandon Sheley Brandon Sheley is offline
 
Join Date: Mar 2005
Real name: Brandon
Why would you want to duplicate your attachments ?
__________________

Email me for website help: brandon[at]sheley[dot]org
Reply With Quote
  #3  
Old 03 Feb 2008, 16:59
nexialys
Guest
 
did you check with your host about Mirroring your server content?!
Reply With Quote
  #4  
Old 04 Feb 2008, 17:37
mute mute is offline
 
Join Date: Dec 2002
Here's the route we took with our multi-webserver setup:

1) Initially we tried rsync, but found that it sort of sucked for users when they'd upload an avatar or attachment and the link would be broken for some people until the rsync happened (1 minute is still too long to lag IMO).

2) We setup FASD or whatever it is called to monitor directories for changes and rsync based on activity. This worked OK, but it was sort of prone to problems, and still lagged a wee bit.

3) We decided to add a bunch of webservers, so we moved to a setup where we have an image server (running thttpd) that serves all the images and most of the static content (js, etc) on the site. This server also runs NFSd, and serves our webroot out to all of our webservers. The webservers mount the webroot via NFS, have rw access to avatars and attachments but only ro access to everything else.

#3 has been working well for us for a couple of years now. Sure, it's a single point of failure, but it has the least amount of headache compared to some of the other solutions I've seen (hacking code so image uploads go to a specific server, etc).

In the near future I'd like to start thinking about provisioning a better disaster recovery system where the webservers rsync a copy off the NFS to their local disks, and a new virtualhost so in the event of an imageserver failure, I can point the images url at our loadbalancer as opposed to the images server and move the local webroots into place and the webservers will pick up the slack until our images server is back online.
Reply With Quote
  #5  
Old 06 Feb 2008, 19:10
blackvborg blackvborg is offline
 
Join Date: Dec 2006
Hello mute, thank you for your post; I've evauleated all your points and the best (in theory) seems the 2nd, but it's not much reliable (eg the notify of file change happens when the write starts and not when it finish, moreover you've some headeache with the replication of new directories)

about NFS yes, it a SPOF. have you some figure about the performance of apache running against a local filesystem vs a network filesystem like NFS? in some benchmarks I've seen that the performance are much lower. for sure it's the easiest implementation.

About HA and NFS I think you could rsync the nfs content on a second passive server (or just one of your webserver) and in case of failure change the NFS server to the backup server

BTW I don't like none of them

for Loco.M: I'd replicate attachments bacause I need to have more webservers

for nexialys: I've not found much to date; I'm looking for realtime mirroring software; not found much except network filesystem like redhat gfs (poor performance for me)

any other advice?

Last edited by blackvborg; 06 Feb 2008 at 19:19.
Reply With Quote
  #6  
Old 06 Feb 2008, 19:25
mute mute is offline
 
Join Date: Dec 2002
Originally Posted by blackvborg View Post
Hello mute, thank you for your post; I've evauleated all your points and the best (in theory) seems the 2nd, but it's not much reliable (eg the notify of file change happens when the write starts and not when it finish, moreover you've some headeache with the replication of new directories)

about NFS yes, it a SPOF. have you some figure about the performance of apache running against a local filesystem vs a network filesystem like NFS? in some benchmarks I've seen that the performance are much lower. for sure it's the easiest implementation.

About HA and NFS I think you could rsync the nfs content on a second passive server (or just one of your webserver) and in case of failure change the NFS server to the backup server

BTW I don't like none of them

for Loco.M: I'd replicate attachments bacause I need to have more webservers

for nexialys: I've not found much to date; I'm looking for realtime mirroring software; not found much except network filesystem like redhat gfs (poor performance for me)

any other advice?
Well, as far as NFS vs locally hosted content, I'm sure there's a performance hit, but I don't have any hard numbers. One thing to remember is that if you are using a PHP cache like Xcache, APC, etc, then a lot of your dynamic content (the only thing we serve via apache) is cached in memory, so the amount of traffic over the NFS is actually a lot less than you'd expect.
Reply With Quote
  #7  
Old 07 Feb 2008, 02:16
EricGT EricGT is offline
 
Join Date: Jan 2004
I just use an NFS mount for the front-end servers and keep an rsync'ed copy of the mount on another server. My servers utilize RAID arrays, so while I have lost hard drives occasionally over the years, I've never lost an array or had any unexpected downtime due to hard drive failure, nor have I ever suffered any data loss. If there is a performance hit for setting things up this way, I haven't noticed it. Eric
Reply With Quote
  #8  
Old 09 Feb 2008, 13:00
blackvborg blackvborg is offline
 
Join Date: Dec 2006
I've not found any benchmark/tests about NFS vs local; have you found any of it?
Reply With Quote
Reply



Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Mod Options

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


New To Site? Need Help?

All times are GMT. The time now is 18:49.

Layout Options | Width: Wide Color: