|
|
|||||||||
|
|||||||||
| |||||||||
|
|
|
| |||||||||
![]() |
|
|
«
Previous Thread
|
Next Thread
»
|
Thread Tools | Search this Thread | Rate Thread | Display Modes |
|
#1
|
|||
|
|||
|
Suggestions for Automatic Data Transfer?
Can anyone recommend a product (software/hardware) for automatic data transfer from (a specified location in) RAID (where the specification comes from another PC) to a laptop PC, without requiring any human involvement and/or Internet connection? (i.e., no wireless/Ethernet etc required) It would need to work reliably for a large volume of data, on the order of tens of thousand of numbers per second.
Also, what is the fastest mode for this: serial, USB, parallel port, etc? Which of these does the suggested product work with? |
|
#2
|
|||
|
|||
|
(^^;?(Though I'm not familiar with it and have no necessary enviroment to experiment.)
I think you'd better use Gigabit Ether cross cable with new 2 NIC attached rather than those. And you introduce clustering service. |
|
#3
|
|||
|
|||
|
I'm not sure. I'd follow Megumi's NIC interface recommendation and investigate if backup software can do what you want. If you set automated backup software to dump all the data in your selected directories at your predefined schedule you may get what you need. ArcServeIT is software I've used before but there may be other ones you should look at too.
|
|
#4
|
|||
|
|||
|
Clarification
Here is a clarification, in case it helps: Computer1 will need to run algorithms to process data, which comes from a specified location in RAID on Computer2. It would have to be a rapid, very secure transfer of data from RAID to Computer1 (ie, 802.11 is NOT secure enough – an Ethernet link between the machines, without Internet, may be ok) which occurs as soon as RAID has the data (Computer1 will be notified of this, and of the location in RAID, by a separate device). NO person should have to be involved for any of this to work (ie, the person does not have to click to transfer data, it happens automatically upon notification between the machines) and the method of transfer would have to be quick (volume of data = tens of thousands of numbers per second).
What would be the consensus on this, i.e., what approach do you think would be best? The suggestions I have received include: -Laplink -Ethernet between the two machines, with network share, then a software package like MirrorFolder http://www.techsoftpl.com/backup/index.htm -“you'd better use Gigabit Ether cross cable with new 2 NIC attached rather than those. And you introduce clustering service.” (am new to this, so am not sure how this would be implemented) -“set automated backup software to dump all the data in your selected directories at your predefined schedule you may get what you need. ArcServeIT is software I've used before but there may be other ones you should look at too.” -USB/Firewire via http://www.wdc.com/en/products/products.asp?DriveID=75 Do you have any guidance on how this would be set up, or how difficult it would be to do with any of these approaches? |
|
#5
|
|||
|
|||
|
That's quite a clarification. I don't think any of the solutions suggested so far will meet your requirements.
A LIMS or a SCADA system is close to what you described but not exactly. You could probably have a SCADA system customized to meet your requirements. A SCADA system typically takes data generated on a client PC and sends it to the server PC for processing. SCADAs usually work with PLCs but data is data. |
|
#6
|
|||
|
|||
|
(^^;?(Yes, too much requirement. It would needs the expert.)
Slow cables is excluded automatically even if configured, USB became fast and convenient but still have problems. Clusterring is in a word CPU or Computer RAID, though the HDD RAID would be still unreliable from the start, it should be able to cooperate as striping and also to synchronize as mirroring as if it is one computer, but security issue is another matter. SCADA is also new to me, I had a look at a few pages of them, though it's hard to get a conception, but the key word seems to be "redundancy" (the word itself is somewhat vague?) as is expected. It seems, in a word, to be clustering + measuring devices RAID. It should be noted that performance and security is incompatible with each other in their essence. In addition, reliability also does not cooperate with either of them. This may explain why SP2 damages hardwares. |
|
#7
|
|||
|
|||
|
Clarification 2
What I am looking for is the easiest, most secure and quick (the input data is created on the order of hundreds of thousands to millions of numbers per second) means of getting data from short-term data storage, to a laptop which runs algorithms, and then outputting it to a long-term storage device. The below is an outline, the referenced “OurPart” is a laptop with software that must operate in such a larger system (it must get data from a shared folder, and output it to another shared folder).
What I am looking for is a Windows-based software tool (or code/tool that can be added in MS Visual Studio or Borland C++ Builder, in a C/C++ based project as the actual algorithms will be in C/C++) that would help with the data file-retrieval/file deletion/file creation on a shared folder on a different PC. No user participation should be required (i.e., transferring a file does not require a person to actually click, etc). Your reply may therefore mostly consist of offering product suggestions and instructions, as appropriate. For the following scenario, assume the machines are networked with an Ethernet connection between the machines (but no Internet, etc) 1. There is a shared folder (“OurPartInput”, Windows type) on the short-term storage (RAID) 2. DataCreatorDevice writes an “alert_computerInfo4.txt” file to “OurPartInput” (the OurPart laptop constantly scans for the presence of this file in the “OurPartInput” folder). The alert file tells OurPart that new data is written to the file “Input_Info1_Info2_Info3_Info4.txt”, and that a key data value is X. 3. OurPart fetches the alert file to the OurPart laptop (and deletes it from the source, as it is already fetched) 4. OurPart reads the info from the alert file and fetches the input data file “Input_Info1_Info2_Info3_Info4.txt” from the shared “OurPartInput” folder to the OurPart laptop. (Note: it would help if OurPart could also have the capability to periodically delete the old input files from the “OurPartInput” folder.) 4. OurPart reads the input data from this file, runs the algorithms/analysis, and creates an output data file “Output_Info1_Info2_Info3_Info4.txt”, which OurPart writes to a (shared, Windows-type) folder on a different (output) device. 5. OurPart keeps a log on our hard-drive (in a shared folder, “OurPartBackup”), which the system can access anytime to retrieve stored input/output data. |
|
#8
|
|||
|
|||
|
This is the wrong forum for your post. Maybe if you posted in a developers' forum on this site they could help you more.
|
|
#9
|
|||
|
|||
|
(~~;( Yeah, Windows cannot stand such a precise architecture from the start.)
Though I don't know what you are thinking, Windows is the OS which is particularly apt for beginners, user friendly but mostly meaninglessly GUI based, hence heavy and resource eating one. There's no reason why you must choose Windows above all in stead of no need of attendance of man's hand. It is completely wrong idea to expect Windows such reliably precise task and high performance from the begining. Even highly costed product must (or so much) consider user friendliness rather than performance in Windows plathome. Your better choice should lie in other light and flexible CUI based OS plathomes. If you don't understand what I said thus far, you'd better change your business. You are not apt for computer absolutely, since even just an amateur as me can understand this. |
![]() |
| Viewing: Dev Shed Forums > Computer Hardware > Computer Hardware > Suggestions for Automatic Data Transfer? |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
|
|