PHP Development
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me

The Shed is going Social! Join us on FaceBook and Twitter and chime in on the conversation.

Go Back   Dev Shed ForumsProgramming LanguagesPHP Development

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old February 27th, 2013, 10:03 PM
Hairboy Hairboy is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jan 2010
Location: Adelaide
Posts: 6 Hairboy User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 2 h 3 m 26 sec
Reputation Power: 0
Best method for remote file read/write

The scenario:
  • Two dedicated servers
  • Server A: Generates XML files, at a high CPU load for approximately 5 minutes out of every 15 as it generates these files. It also makes database updates to Server B during this process, but this is not relevant to the question.
  • Server B: A "Web Server". Does little or no processing, primarily reading from databases and compiling PHP pages at user request
  • XML files are approx 500kb each
  • While processing is underway, an XML file is stored approximately every 5 seconds
  • Up to 5,000 of these files will be created daily
  • Files will remain on the server for approximately a week and then be deleted by a cleanup CRON job.

The dilemma:
  • The best method for the XML files created on Server A to be delivered to users on request in as timely a manner as possible

The possibilities
  1. Server A stores XML locally, Server B loads them remotely on demand from users
  2. Server A outputs the files remotely to Server B (FTP? FWRITE?), which can then serve them on request

From what I understand, option 2 would be the best solution for the purpose of a user requesting a file... however my fear is that the large number of writes in a short period of time make Server A grind to a halt on a regular basis?

Is that fear justified??

Reply With Quote
  #2  
Old February 28th, 2013, 12:56 AM
requinix's Avatar
requinix requinix is offline
Still alive
Dev Shed God 16th Plane (12500 - 12999 posts)
 
Join Date: Mar 2007
Location: Washington, USA
Posts: 12,877 requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)  Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1
Time spent in forums: 5 Months 1 Week 5 Days 8 h 8 sec
Reputation Power: 8977
Send a message via AIM to requinix Send a message via MSN to requinix Send a message via Yahoo to requinix Send a message via Google Talk to requinix
How about making a network share on one of the servers then mounting it on the other? Keeping it on A and mounting on B is probably better since B does less work and less often.

Reply With Quote
  #3  
Old February 28th, 2013, 01:36 AM
Hairboy Hairboy is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jan 2010
Location: Adelaide
Posts: 6 Hairboy User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 2 h 3 m 26 sec
Reputation Power: 0
Quote:
Originally Posted by requinix
How about making a network share on one of the servers then mounting it on the other? Keeping it on A and mounting on B is probably better since B does less work and less often.

Thanks for the suggestion.

Although this provides a nice method of addressing the files and abstracts it nicely from PHP, and makes sure that "B" is not interrupted at any time.....does it do anything to help "B" read the file more quickly from "A", even when "A" is at high CPU load?

Reply With Quote
  #4  
Old February 28th, 2013, 01:48 AM
requinix's Avatar
requinix requinix is offline
Still alive
Dev Shed God 16th Plane (12500 - 12999 posts)
 
Join Date: Mar 2007
Location: Washington, USA
Posts: 12,877 requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)requinix User rank is General 120th Grade (Above 100000 Reputation Level)  Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1Folding Points: 417516 Folding Title: Super Ultimate Folder - Level 1
Time spent in forums: 5 Months 1 Week 5 Days 8 h 8 sec
Reputation Power: 8977
Send a message via AIM to requinix Send a message via MSN to requinix Send a message via Yahoo to requinix Send a message via Google Talk to requinix
I can't imagine sharing taking anything more than a negligible amount of CPU time.

Reply With Quote
Reply

Viewing: Dev Shed ForumsProgramming LanguagesPHP Development > Best method for remote file read/write

Developer Shed Advertisers and Affiliates



Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump

Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 


Powered by: vBulletin Version 3.0.5
Copyright ©2000 - 2013, Jelsoft Enterprises Ltd.

© 2003-2013 by Developer Shed. All rights reserved. DS Cluster - Follow our Sitemap