Zhang Weiwu wrote:
Dear list
Recently I need to start a small project to distribute about 100GB megabytes of audio files to local university campus. I'd like to post my requirement hoping I can get some insightful recommendation on what software/technology to use to distribute these files.
1. It must be accessible from Windows linearly (e.g. FTP protocol), better also random access from Windows is also supported (e.g. samba). We must not require Windows user to install for-fee software in order to get the content. 2. It must be accessible from Linux randomly, with no client side tool (e.g. nfs) or free (as in beer) client side tool. 3. Both Linux and Windows user should be able to batch-download, e.g. download a whole folder and its sub-folders; 4. It must stand relatively heavy load; 5. It better can provide user a search feature so that user can search with keyword in file name, or even better, in mp3/ogg tag. 6. It better handle character-set difference in nice manner. e.g. HTTP can handle character-set difference because charset info is in HTTP header and charset conversion is done automatically; FTP cannot handle character-set difference in very nice manner, user have to configure their FTP client to do charset conversion, and most Windows FTP client software I know of do not support charset conversion.
I think you are rather hoping you can set it up and leave it.... if people start using it and it becomes popular they will need support and unless you have plenty of time it will be wise to consider who has access, how you monitor that access, how you stop the resource being compromised (security), and how you are going to assist the user community. There are other issues, your network support people may not be too happy if your archive stuffs the network if it gets popular, you may need to look into things like multi-casting and QoS with them. You may and your users may not be to happy if it the server collapses under the load. Your solution needs to take into account how many people are expected to use the resource, how often, and from where. While 100G may not seem a lot, 100 people accessing 100G is an awful lot of data moving around wires.
I am thinking about possible solutions:
1. FTP -> can handle heavy load, can do bath upload, not random-accessible, auto-charset conversion not supported;
Hmm. usually hard work for the user. PUTTY in the Windows world does offer a fairly simply command interface. Using cygwin on windows machines to setup the machines up as a X terminal is a further route.
2. apache -> batch download not easy for users, handle charset conversion nicely, not random access
Web is really down to how you setup the web access, it is up to you how easy for the users to access the data and how it is presented. External access becomes a viable option. Plenty of search options, and support pages can be setup. Probably easiest solution because you will have minimal security concerns, and only one thing to look after.
3. NFS -> I don't know any free-as-in-beer Windows client software for it and I don't know if that client software can do charset conversion; for Linux clients it's perfect;
NFS within a university environment is a security no-brainer. I believe NFS can be made to work under cygwin though I have not tried this myself.
4. Samba -> I don't know if charset conversion is easy with it. If a SuSE client connects to it, can suse client select which charset to use without forcing user to use commandline? And how about windows, can windows connect to the samba share and do charset conversion automatically?
For raw file store access within the institution is OK. External access usually a no-no. Sorting out authentication may be an interesting experience if you are within an AD environment. Samba performs most of things a domain server, you can set up the server end to use specific character sets but the interaction with client may a bit odd if client is configured for something different.
5. DC++ -> looks very nice for charset conversion, I also tried it, nice. But I don't know if there are Linux server-end software. Need to check.
If it is peer to peer access is what is required e-Mule/e-Donkey is another option, and there are other options. Personally do not use and do not recommend P2P, security is down to the weakest link and P2P is somewhat like unprotected sex... you never no what you are going to catch. P2P solutions do need careful thought about security.
I am thinking perhaps combine two solutions together might be the best, e.g. setting up FTP server for Windows users and set up NFS server for Linux users. Still I am not sure what's the best solution, and there might be better solutions than what I listed that I never heard of.
* In all above discussion "charset conversion" means charset conversion for file names, not the content. Content is in mp3/ogg format.
Thanks for any comments!
The thing to remember in the University environment you have a lot of talented individuals, and will regard any shared resource as fair game to so you need to think paranoid. It may be a war zone in the outside internet world but it can be like living in the middle of armageddon within a University.