[opensuse] best file distribution technology for my case?
Dear list Recently I need to start a small project to distribute about 100GB megabytes of audio files to local university campus. I'd like to post my requirement hoping I can get some insightful recommendation on what software/technology to use to distribute these files. 1. It must be accessible from Windows linearly (e.g. FTP protocol), better also random access from Windows is also supported (e.g. samba). We must not require Windows user to install for-fee software in order to get the content. 2. It must be accessible from Linux randomly, with no client side tool (e.g. nfs) or free (as in beer) client side tool. 3. Both Linux and Windows user should be able to batch-download, e.g. download a whole folder and its sub-folders; 4. It must stand relatively heavy load; 5. It better can provide user a search feature so that user can search with keyword in file name, or even better, in mp3/ogg tag. 6. It better handle character-set difference in nice manner. e.g. HTTP can handle character-set difference because charset info is in HTTP header and charset conversion is done automatically; FTP cannot handle character-set difference in very nice manner, user have to configure their FTP client to do charset conversion, and most Windows FTP client software I know of do not support charset conversion. I am thinking about possible solutions: 1. FTP -> can handle heavy load, can do bath upload, not random-accessible, auto-charset conversion not supported; 2. apache -> batch download not easy for users, handle charset conversion nicely, not random access 3. NFS -> I don't know any free-as-in-beer Windows client software for it and I don't know if that client software can do charset conversion; for Linux clients it's perfect; 4. Samba -> I don't know if charset conversion is easy with it. If a SuSE client connects to it, can suse client select which charset to use without forcing user to use commandline? And how about windows, can windows connect to the samba share and do charset conversion automatically? 5. DC++ -> looks very nice for charset conversion, I also tried it, nice. But I don't know if there are Linux server-end software. Need to check. I am thinking perhaps combine two solutions together might be the best, e.g. setting up FTP server for Windows users and set up NFS server for Linux users. Still I am not sure what's the best solution, and there might be better solutions than what I listed that I never heard of. * In all above discussion "charset conversion" means charset conversion for file names, not the content. Content is in mp3/ogg format. Thanks for any comments! -- Zhang Weiwu Real Softservice http://www.realss.com +86 592 2091112 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Zhang Weiwu wrote:
Dear list
Recently I need to start a small project to distribute about 100GB megabytes of audio files to local university campus. I'd like to post my requirement hoping I can get some insightful recommendation on what software/technology to use to distribute these files.
1. It must be accessible from Windows linearly (e.g. FTP protocol), better also random access from Windows is also supported (e.g. samba). We must not require Windows user to install for-fee software in order to get the content. 2. It must be accessible from Linux randomly, with no client side tool (e.g. nfs) or free (as in beer) client side tool. 3. Both Linux and Windows user should be able to batch-download, e.g. download a whole folder and its sub-folders; 4. It must stand relatively heavy load; 5. It better can provide user a search feature so that user can search with keyword in file name, or even better, in mp3/ogg tag. 6. It better handle character-set difference in nice manner. e.g. HTTP can handle character-set difference because charset info is in HTTP header and charset conversion is done automatically; FTP cannot handle character-set difference in very nice manner, user have to configure their FTP client to do charset conversion, and most Windows FTP client software I know of do not support charset conversion.
I think you are rather hoping you can set it up and leave it.... if people start using it and it becomes popular they will need support and unless you have plenty of time it will be wise to consider who has access, how you monitor that access, how you stop the resource being compromised (security), and how you are going to assist the user community. There are other issues, your network support people may not be too happy if your archive stuffs the network if it gets popular, you may need to look into things like multi-casting and QoS with them. You may and your users may not be to happy if it the server collapses under the load. Your solution needs to take into account how many people are expected to use the resource, how often, and from where. While 100G may not seem a lot, 100 people accessing 100G is an awful lot of data moving around wires.
I am thinking about possible solutions:
1. FTP -> can handle heavy load, can do bath upload, not random-accessible, auto-charset conversion not supported;
Hmm. usually hard work for the user. PUTTY in the Windows world does offer a fairly simply command interface. Using cygwin on windows machines to setup the machines up as a X terminal is a further route.
2. apache -> batch download not easy for users, handle charset conversion nicely, not random access
Web is really down to how you setup the web access, it is up to you how easy for the users to access the data and how it is presented. External access becomes a viable option. Plenty of search options, and support pages can be setup. Probably easiest solution because you will have minimal security concerns, and only one thing to look after.
3. NFS -> I don't know any free-as-in-beer Windows client software for it and I don't know if that client software can do charset conversion; for Linux clients it's perfect;
NFS within a university environment is a security no-brainer. I believe NFS can be made to work under cygwin though I have not tried this myself.
4. Samba -> I don't know if charset conversion is easy with it. If a SuSE client connects to it, can suse client select which charset to use without forcing user to use commandline? And how about windows, can windows connect to the samba share and do charset conversion automatically?
For raw file store access within the institution is OK. External access usually a no-no. Sorting out authentication may be an interesting experience if you are within an AD environment. Samba performs most of things a domain server, you can set up the server end to use specific character sets but the interaction with client may a bit odd if client is configured for something different.
5. DC++ -> looks very nice for charset conversion, I also tried it, nice. But I don't know if there are Linux server-end software. Need to check.
If it is peer to peer access is what is required e-Mule/e-Donkey is another option, and there are other options. Personally do not use and do not recommend P2P, security is down to the weakest link and P2P is somewhat like unprotected sex... you never no what you are going to catch. P2P solutions do need careful thought about security.
I am thinking perhaps combine two solutions together might be the best, e.g. setting up FTP server for Windows users and set up NFS server for Linux users. Still I am not sure what's the best solution, and there might be better solutions than what I listed that I never heard of.
* In all above discussion "charset conversion" means charset conversion for file names, not the content. Content is in mp3/ogg format.
Thanks for any comments!
The thing to remember in the University environment you have a lot of talented individuals, and will regard any shared resource as fair game to so you need to think paranoid. It may be a war zone in the outside internet world but it can be like living in the middle of armageddon within a University.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Friday 2007-04-06 at 14:07 +0100, G.T.Smith wrote:
If it is peer to peer access is what is required e-Mule/e-Donkey is another option, and there are other options. Personally do not use and do not recommend P2P, security is down to the weakest link and P2P is somewhat like unprotected sex... you never no what you are going to catch. P2P solutions do need careful thought about security.
bittorrent does have security: the initial seed or link inlcudes checksums.
Thanks for any comments!
The thing to remember in the University environment you have a lot of talented individuals, and will regard any shared resource as fair game to so you need to think paranoid. It may be a war zone in the outside internet world but it can be like living in the middle of armageddon within a University.
X'-) - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFGFoh9tTMYHG2NR9URApfDAJ9zs71Z15Utku3GOtdCo+xyBLykzACfbLSi gOq3K0faIBa+wiBDnUI+BF4= =9R+Z -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Friday 06 April 2007 19:50:51 Carlos E. R. wrote:
The Friday 2007-04-06 at 14:07 +0100, G.T.Smith wrote:
If it is peer to peer access is what is required e-Mule/e-Donkey is another option, and there are other options. Personally do not use and do not recommend P2P, security is down to the weakest link and P2P is somewhat like unprotected sex... you never no what you are going to catch. P2P solutions do need careful thought about security.
bittorrent does have security: the initial seed or link inlcudes checksums.
Indeed. You can be absolutely sure that the virus you receive is identical to the virus that was sent A checksum isn't security, it is fault protection -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Friday 2007-04-06 at 19:55 +0200, Anders Johansson wrote:
bittorrent does have security: the initial seed or link inlcudes checksums.
Indeed. You can be absolutely sure that the virus you receive is identical to the virus that was sent
A checksum isn't security, it is fault protection
Should I remind you that SuSE/Novell uses torrent to distribute the iso images of the distribution? Indeed, the "virus" that SuSE distributes is the one I have installed in my system, alive and running - it is called "opensuse linux"! :-P - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFGFpMUtTMYHG2NR9URAnbYAKCFl7+/abn+vNjJA2CPGclRZ6f0dgCghBP+ 17qsOzXAljT44NHvegRjfE0= =v0JG -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Friday 06 April 2007 13:35, Carlos E. R. wrote:
Indeed, the "virus" that SuSE distributes is the one I have installed in my system, alive and running - it is called "opensuse linux"! ... preach it bubba... !
-- Kind regards, M Harris <>< -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Friday 06 April 2007 20:35:44 Carlos E. R. wrote:
The Friday 2007-04-06 at 19:55 +0200, Anders Johansson wrote:
bittorrent does have security: the initial seed or link inlcudes checksums.
Indeed. You can be absolutely sure that the virus you receive is identical to the virus that was sent
A checksum isn't security, it is fault protection
Should I remind you that SuSE/Novell uses torrent to distribute the iso images of the distribution? Indeed, the "virus" that SuSE distributes is the one I have installed in my system, alive and running - it is called "opensuse linux"!
That doesn't change the fact that bittorrent in itself doesn't have security. It also doesn't change the fact that a checksum is not a security feature. It only helps you ensure that what you get is what the other side sent. In the end, you're still stuck with the question "do I trust the sender". Bittorrent doesn't help you with that -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Anders Johansson wrote:
On Friday 06 April 2007 20:35:44 Carlos E. R. wrote:
The Friday 2007-04-06 at 19:55 +0200, Anders Johansson wrote:
bittorrent does have security: the initial seed or link inlcudes checksums.
Indeed. You can be absolutely sure that the virus you receive is identical to the virus that was sent
A checksum isn't security, it is fault protection
Should I remind you that SuSE/Novell uses torrent to distribute the iso images of the distribution? Indeed, the "virus" that SuSE distributes is the one I have installed in my system, alive and running - it is called "opensuse linux"!
That doesn't change the fact that bittorrent in itself doesn't have security. It also doesn't change the fact that a checksum is not a security feature. It only helps you ensure that what you get is what the other side sent. In the end, you're still stuck with the question "do I trust the sender". Bittorrent doesn't help you with that
Which protocol does that, I'd like to know... In the end, you have to trust to source, right? Unless it's source code, then you can check out the code for yourself. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Friday 06 April 2007 20:42:47 Jan Karjalainen wrote:
Which protocol does that, I'd like to know...
None, that's the point.
In the end, you have to trust to source, right?
Exactly
Unless it's source code, then you can check out the code for yourself.
If you want to get really picky, that doesn't help either -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Friday 06 April 2007 11:42, Jan Karjalainen wrote:
...
That doesn't change the fact that bittorrent in itself doesn't have security. It also doesn't change the fact that a checksum is not a security feature. It only helps you ensure that what you get is what the other side sent. In the end, you're still stuck with the question "do I trust the sender". Bittorrent doesn't help you with that
Which protocol does that, I'd like to know... In the end, you have to trust to source, right?
Of course. But there's a separate issue, and that is the matter of knowing that the provider is who them claim to be. Piggybacking malware on the name of a trusted source is a viable means for injecting an exploit, if the distribution system does not preclude such misrepresentations. That's what cryptographic identity certificates are for. One would hope that if BitTorrent is going to be widely used to distribute critical resources such as software it would be endowed with the ability to propagate and verify these signatures. Or does BitTorrent already incorporate certificate validation?
Unless it's source code, then you can check out the code for yourself.
True, if you're a good enough programmer and have the time. For practical purposes, virtually all users must trust someone else to certify that a given piece of software if free of deliberately added vulnerabilities. And that does not reflect bugs with security impacts--they're a separate issue--nor does such a professional certification constitute a guarantee. C'est la vie. Randall Schulz -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Friday 2007-04-06 at 11:59 -0700, Randall R Schulz wrote:
That's what cryptographic identity certificates are for. One would hope that if BitTorrent is going to be widely used to distribute critical resources such as software it would be endowed with the ability to propagate and verify these signatures.
Or does BitTorrent already incorporate certificate validation?
Tell me, when I download opensuse, using http, for instance, do I get such cryptographic certificates? I believe not. Not even if download from the novell site. However, you can publish the torrent initial link in a secure webserver (https), which means that you get the download site links and checksums from a certified source. The ensuing torrent download is thus certified. To duplicate that feat with http you require all mirror servers to use https. And FTP? No way. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFGFp6gtTMYHG2NR9URAqT6AJ9Y0W/lIDT3nFkEX7JNwIT1ngkisQCffjIx EfSqj+TgVkIYaxsy4u250qU= =wEte -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Carlos, On Friday 06 April 2007 12:25, Carlos E. R. wrote:
The Friday 2007-04-06 at 11:59 -0700, Randall R Schulz wrote:
That's what cryptographic identity certificates are for. One would hope that if BitTorrent is going to be widely used to distribute critical resources such as software it would be endowed with the ability to propagate and verify these signatures.
Or does BitTorrent already incorporate certificate validation?
Tell me, when I download opensuse, using http, for instance, do I get such cryptographic certificates? I believe not. Not even if download from the novell site.
That's the point. It seems like something that needs to be incorporated into file distribution software in order to secure our on-line software distribution networks.
However, you can publish the torrent initial link in a secure webserver (https), which means that you get the download site links and checksums from a certified source. The ensuing torrent download is thus certified.
So the answer is that security virtually identical to what could be achieved by directly incorporating certificate support into BitTorrent itself can be achieved with existing mechanisms. That's good.
To duplicate that feat with http you require all mirror servers to use https. And FTP? No way.
Don't get me wrong: I'm not suggesting there's anything in any way superior to BitTorrent, at least for popular downloads (below a certain threshold of demand, BitTorrent is slower 'cause there aren't enough copies to satisfy retrieval requests in a timely manner and direct retrieval is preferable for the end user). I was merely addressing the point that running someone else's software is an act of trust. Such trust must be based on true identities and not something forgeable.
-- Cheers, Carlos E. R.
Randall Schulz -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Friday 2007-04-06 at 12:41 -0700, Randall R Schulz wrote: ...
I was merely addressing the point that running someone else's software is an act of trust. Such trust must be based on true identities and not something forgeable.
But the OP wants to distribute audio files, not programs. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFGFq/0tTMYHG2NR9URAtSsAJ47BdMHPc0pkxF4jyJCChxCJEl3DgCcD4U1 XhZvsNEaz+yVb0D3MibOF9Q= =1S7w -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Friday 06 April 2007 13:39, Carlos E. R. wrote:
The Friday 2007-04-06 at 12:41 -0700, Randall R Schulz wrote:
...
I was merely addressing the point that running someone else's software is an act of trust. Such trust must be based on true identities and not something forgeable.
But the OP wants to distribute audio files, not programs.
So? That does not fundamentally alter the security analysis. If the audio is a message telling you where to meet to hand off your copy of some top-secret documents you're leaking, you probably want to know you're getting the right message. If it's your new favorite mix tape, then I don't think it's a very big deal if someone swaps their's for yours.
-- Cheers, Carlos E. R.
Randall Schulz -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
From: Jan Karjalainen To: opensuse@opensuse.org Subject: Re: [opensuse] best file distribution technology for my case? Date: Fri, 06 Apr 2007 20:42:47 +0200
Which protocol does that, I'd like to know... In the end, you have to trust to source, right? Unless it's source code, then you can check out the code for yourself. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Greetings, use Bittorrent for the transfer but use zip, rar or other tool to password protect your package, so your package is protected against curious people and by the integrity check you're certain that you have received your package. Its a weird discussion, your should know =/ Carlos A. PS: Unless you know that there's a way to change a package without modifying the integrity of these (MD5SUM), is that possible? _________________________________________________________________ FREE pop-up blocking with the new MSN Toolbar - get it now! http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Ryouga Hibiki wrote:
PS: Unless you know that there's a way to change a package without modifying the integrity of these (MD5SUM), is that possible?
I *think* it's been shown that it's possible to create two different files that have the same MD5 checksum. Exploiting this would require creating a *meaningful* file with the same checksum as the original, though, which is much more difficult. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Sunday 2007-04-08 at 23:43 -0700, David Brodbeck wrote:
Ryouga Hibiki wrote:
PS: Unless you know that there's a way to change a package without modifying the integrity of these (MD5SUM), is that possible?
I *think* it's been shown that it's possible to create two different files that have the same MD5 checksum.
Curious! I was thinking of that the other day while falling sleep. It is obviously possible: if it weren't, then we could use the checksum instead of the original file as a brutally effective compression technique. There will be then several (many?) files of the same size having the same checksum.
Exploiting this would require creating a *meaningful* file with the same checksum as the original, though, which is much more difficult.
Not knowing the in depth mathematical analysis of checksums, my educated guess is that a checksum protects against the chance corruption of a file in transmission, affecting one or many, but not all, of its bytes. It will not protect against the deliberate attempt to generate a file of the same size and checksum; but generating one such file that is a valid file of the same format I imagine could be an herculean task. In the case of the SuSE iso images, the task would be terrible difficult: each rpm inside the iso has also checksums, plus a pgp signature. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFGGf5dtTMYHG2NR9URAuysAKCMl9zILcGYrrmrS3HDS/OoM8FAaQCeOOD+ yBdiXBGKKBLNLZq2j+gqGso= =lSj3 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Carlos E. R. wrote:
The Sunday 2007-04-08 at 23:43 -0700, David Brodbeck wrote:
Ryouga Hibiki wrote:
PS: Unless you know that there's a way to change a package without modifying the integrity of these (MD5SUM), is that possible? I *think* it's been shown that it's possible to create two different files that have the same MD5 checksum.
Curious!
I was thinking of that the other day while falling sleep. It is obviously possible: if it weren't, then we could use the checksum instead of the original file as a brutally effective compression technique. There will be then several (many?) files of the same size having the same checksum.
Exploiting this would require creating a *meaningful* file with the same checksum as the original, though, which is much more difficult.
Not knowing the in depth mathematical analysis of checksums, my educated guess is that a checksum protects against the chance corruption of a file in transmission, affecting one or many, but not all, of its bytes. It will not protect against the deliberate attempt to generate a file of the same size and checksum; but generating one such file that is a valid file of the same format I imagine could be an herculean task.
In the case of the SuSE iso images, the task would be terrible difficult: each rpm inside the iso has also checksums, plus a pgp signature.
Bear in mind an md5sum is only 128 bits. It is impossible for there to be only one file that results in that sum, given that a file can be any size, with any value in each of the bytes. However, it's virtually impossible to change a file so that it has the same md5sum and is still sensible in the intended application. A small change in the original file makes a big change in the md5sum. -- Use OpenOffice.org http://www.openoffice.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Monday 2007-04-09 at 07:28 -0400, James Knott wrote:
Bear in mind an md5sum is only 128 bits. It is impossible for there to be only one file that results in that sum, given that a file can be any size, with any value in each of the bytes. However, it's virtually impossible to change a file so that it has the same md5sum and is still sensible in the intended application. A small change in the original file makes a big change in the md5sum.
Exactly. But it will also be possible to find several files of the same size that produce the same checksum. I have no idea how many, but there will be a number that I think could be calculated. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFGGipWtTMYHG2NR9URAqKhAJ4m0GKdgUA0tYlU3mpQNQAAg81+fgCfceon PB4X3bUiR680cBRmp7lBgn0= =KuzZ -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
The Monday 2007-04-09 at 07:28 -0400, James Knott wrote:
Bear in mind an md5sum is only 128 bits. It is impossible for there to be only one file that results in that sum, given that a file can be any size, with any value in each of the bytes. However, it's virtually impossible to change a file so that it has the same md5sum and is still sensible in the intended application. A small change in the original file makes a big change in the md5sum.
Exactly.
But it will also be possible to find several files of the same size that produce the same checksum. I have no idea how many, but there will be a number that I think could be calculated. Of course not, if I calculated it right there are 20370359763344860862684456884093781610514683936659362506361404493543\ 81299763336706183397376 (a 32 char long string, which contains a-f (6) and 0-9 (10), so 32^60, right?)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Am Montag, 9. April 2007 13:57 schrieb Carlos E. R.: possible md5sums, so sure it'll be possible to have two files with the same, the point is, that it is almost impossible to make use of it to attack something, since the file with the same md5sum must be valid and contains the destructive code and this will be rather difficult. Cheers Michael -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) iD8DBQFGGkHucHwbW/zlOZoRAs+HAKCU2r2vURni4QfdZ6m/aUzlGFwfmgCgiunb LtMCdg6Gw2elHc1PJmQMFDY= =pTcH -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Michael Skiba wrote:
something, since the file with the same md5sum must be valid and contains the destructive code and this will be rather difficult.
not so sure. it's usually possible to include in any file dummy code (at the end, by example), never executed, but this allow to adjust a file. it's also possible to delete some part of the code, most application have very rarely used code one can delete without notice so, in theory, it's possible to change an application working and adapt to have the same md5sum. but I nevre heard of such thing so this may be very difficult to acheive. if it's not possible to have a calculation of the missing code (ie md5sum1 - md5sum2 gives the code to add), finding the necessary code may be impossible jdd -- http://www.dodin.net Lucien Dodin, inventeur http://lucien.dodin.net/index.shtml -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
jdd wrote:
Michael Skiba wrote:
something, since the file with the same md5sum must be valid and contains the destructive code and this will be rather difficult.
not so sure.
it's usually possible to include in any file dummy code (at the end, by example), never executed, but this allow to adjust a file.
it's also possible to delete some part of the code, most application have very rarely used code one can delete without notice
so, in theory, it's possible to change an application working and adapt to have the same md5sum.
but I nevre heard of such thing so this may be very difficult to acheive.
if it's not possible to have a calculation of the missing code (ie md5sum1 - md5sum2 gives the code to add), finding the necessary code may be impossible
jdd
Making any change to the original file causes a significant change to the md5sum. Adding or deleting a small amount of code would be noticable. You'd have to come up with some method of retaining the same md5sum, while encorporating those changes. Perhaps you should read up on md5sum and hashing to understand why it's so difficult to sneak in any change. -- Use OpenOffice.org http://www.openoffice.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Michael Skiba wrote:
...sure it'll be possible to have two files with the same, the point is, that it is almost impossible to make use of it to attack something, since the file with the same md5sum must be valid and contains the destructive code and this will be rather difficult.
Right. On the other hand, if someone has access to the web server to plant their malicious files, they also have access to the files that hold the checksums. So in practice checksums are good protection against files corrupted in transit, but rather weak protection against malicious modifications. To check for that, you'd use PGP and get the public key from a keyserver or some other source, *not* from the webserver you downloaded the file from. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Monday 09 April 2007, David Brodbeck wrote:
Michael Skiba wrote:
...sure it'll be possible to have two files with the same, the point is, that it is almost impossible to make use of it to attack something, since the file with the same md5sum must be valid and contains the destructive code and this will be rather difficult.
Right. On the other hand, if someone has access to the web server to plant their malicious files, they also have access to the files that hold the checksums. So in practice checksums are good protection against files corrupted in transit, but rather weak protection against malicious modifications. To check for that, you'd use PGP and get the public key from a keyserver or some other source, *not* from the webserver you downloaded the file from.
Unless the checksum's are signed, getting the pgp key will do you no good. If you suppose that the web site can be easily compromised, why not order a CDrom? As is usual for this board, the entire topic has now spiraled out of control into a fit of paranoia, fear, and suspicion. Perhaps there are people who should not use computers at all. -- _____________________________________ John Andersen -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
John Andersen wrote:
Unless the checksum's are signed, getting the pgp key will do you no good.
Creating a checksum, then signing it, is an unnecessary extra step. GPG can generate a signature for a file all by itself. It's pretty common for sites to include signature files for downloads that are security-related. Many package managers automatically check signatures on files they download, making the whole thing automatic and painless.
If you suppose that the web site can be easily compromised, why not order a CDrom?
That's an option, too. But why ship physical media around when you can download bits? :)
As is usual for this board, the entire topic has now spiraled out of control into a fit of paranoia, fear, and suspicion.
It's not paranoia if they're really out to get you. ;) Debian had a server compromised a year or two ago. It happens. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
<snip>
As is usual for this board, the entire topic has now spiraled out of control into a fit of paranoia, fear, and suspicion.
It's not paranoia if they're really out to get you. ;) Debian had a server compromised a year or two ago. It happens. --
Greetings, Debian problem was because of a vulnerability and compromised account. http://news.com.com/Debian+locks+out+developers+after+server+hack/2100-7349_... The weakest point of the system is the human factor =/ But also its the strongest point. =) Then, finally, Bittorrent can accomplish the task safely, right? Carlos A. _________________________________________________________________ Express yourself instantly with MSN Messenger! Download today it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Ryouga Hibiki wrote:
Then, finally, Bittorrent can accomplish the task safely, right?
Sure. Bittorrent is as safe as the original file. ;) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Monday 2007-04-09 at 14:21 -0800, John Andersen wrote:
As is usual for this board, the entire topic has now spiraled out of control into a fit of paranoia, fear, and suspicion.
I disagree. We are just considering the theoretical posibilities of compromising checksums. I'm the one that started this thread and I'm not afraid: just curious. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFGGsUqtTMYHG2NR9URAjl6AJ9/yEAYnRlvASUZpqJJtba7szPqugCdHp75 eS+VH+D1XrkK/2bFn3ijncY= =N8Nw -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Friday 2007-04-06 at 20:41 +0200, Anders Johansson wrote:
Should I remind you that SuSE/Novell uses torrent to distribute the iso images of the distribution? Indeed, the "virus" that SuSE distributes is the one I have installed in my system, alive and running - it is called "opensuse linux"!
That doesn't change the fact that bittorrent in itself doesn't have security. It also doesn't change the fact that a checksum is not a security feature. It only helps you ensure that what you get is what the other side sent. In the end, you're still stuck with the question "do I trust the sender". Bittorrent doesn't help you with that
And that's way more than what ftp does: I normally get what the other side sent, with no integrity check. The same as any other file transfer protocol, be it ftp, http, samba, nfs... you name it, I have to trust what the other side sends. With torrent at least integrity is checked. You are missing the point: torrent, in the way that Novell uses it to distribute opensuse, is as secure as can be. It is they who post the link with the checksums, and it is they who put the seeds. We don get those from out there in the wild. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD4DBQFGFpe6tTMYHG2NR9URAs3wAKCEgV4OeyGS9j4dtRkRObeQomuoVwCY2Csp APpPpFLTwi45TeRFcVXVog== =Y1UX -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Carlos E. R. wrote:
The Friday 2007-04-06 at 20:41 +0200, Anders Johansson wrote:
Should I remind you that SuSE/Novell uses torrent to distribute the iso images of the distribution? Indeed, the "virus" that SuSE distributes is the one I have installed in my system, alive and running - it is called "opensuse linux"! That doesn't change the fact that bittorrent in itself doesn't have security. It also doesn't change the fact that a checksum is not a security feature. It only helps you ensure that what you get is what the other side sent. In the end, you're still stuck with the question "do I trust the sender". Bittorrent doesn't help you with that
And that's way more than what ftp does: I normally get what the other side sent, with no integrity check. The same as any other file transfer protocol, be it ftp, http, samba, nfs... you name it, I have to trust what the other side sends. With torrent at least integrity is checked.
You are missing the point: torrent, in the way that Novell uses it to distribute opensuse, is as secure as can be. It is they who post the link with the checksums, and it is they who put the seeds. We don get those from out there in the wild.
Bittorrent relies on replication on mutliple source servers so that the client can obtain data from multiple sources. To some extent it sidesteps the bandwidth and server load issues, but there is the potentially dangerous assumption that the source servers concerned are securely maintained by people of good intention. (There are also a few domestic router/modems that choke under the number of open connections that bittorrent can accumulate but that is a separate issue). Checksums as it has been already pointed out provide no security, only a guarantee of the integrity of the source files, and as such are essential for technologies such as bittorrent to work. However, checksum + datasource checks can be gimmicked (though in the instance of bittorrent such gimmickry is unlikely to work). If should also be noted Novell have resources that most individuals do not have to monitor the distribution and flag possible problems.
On Friday 06 April 2007, G.T.Smith wrote:
Checksums as it has been already pointed out provide no security, only a guarantee of the integrity of the source files, and as such are essential for technologies such as bittorrent to work. However, checksum + datasource checks can be gimmicked (though in the instance of bittorrent such gimmickry is unlikely to work).
So which is it Graham ? You seem to want to come down simultaneously on every side of the issue. If you cut/paste your checksums from the web page and they match the bittorrent downloaded ISO, it proves that all of the contributing servers from which your bittorrent was served, were secure enough for the job at hand within the accuracy of the checkum methodology. Each packet (or what ever the data block is called) is check summed in the bittorrent client, and the whole iso can (and should) be checked. There is virtually no opportunity to insert a rogue data block (with a virus) that was not on the original iso and get away with it. -- _____________________________________ John Andersen -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
John Andersen wrote:
On Friday 06 April 2007, G.T.Smith wrote:
Checksums as it has been already pointed out provide no security, only a guarantee of the integrity of the source files, and as such are essential for technologies such as bittorrent to work. However, checksum + datasource checks can be gimmicked (though in the instance of bittorrent such gimmickry is unlikely to work).
So which is it Graham ? You seem to want to come down simultaneously on every side of the issue.
If you cut/paste your checksums from the web page and they match the bittorrent downloaded ISO, it proves that all of the contributing servers from which your bittorrent was served, were secure enough for the job at hand within the accuracy of the checkum methodology.
Each packet (or what ever the data block is called) is check summed in the bittorrent client, and the whole iso can (and should) be checked.
There is virtually no opportunity to insert a rogue data block (with a virus) that was not on the original iso and get away with it.
John, Why should there be sides? If in an intelligent discussion Limes become Limen (frontiers become walls) one would question whether the discussion is still intelligent! Both sides have valid points. and BTW have you ever heard of a dialectic.... ? The packet based checksum used by bittorrent is determined on publication and included in the initial download response, which also points to initial known mirrors. However, one can extract the contents without reference to the checksum (this seems to be implied on their website), and mirrors can point to mirrors of mirrors (and so on ad infinitum). This mechanism is possibly a bit vulnerable, not from bad guys injecting bad code into the image (initially) but bad guys inserting bad packets into mirror copies in a form of DOS attack. Such a checksum approach is only valuable it is usable, and if someone can render it unusable ...... BTW I did initially attempt to download with the SuSE images with bittorrent and abandoned the attempt when I discovered that I had something like 60 open connections to various locations and the data throughput was crawling with a completion estimate of a couple of days! This could be due to bad datalinks, bad data or bandwidth throttling by the ISP, but it was beginning to have an impact on using the machine for other purposes.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Saturday 2007-04-07 at 10:14 +0100, G.T.Smith wrote: ...
BTW I did initially attempt to download with the SuSE images with bittorrent and abandoned the attempt when I discovered that I had something like 60 open connections to various locations and the data throughput was crawling with a completion estimate of a couple of days! This could be due to bad datalinks, bad data or bandwidth throttling by the ISP, but it was beginning to have an impact on using the machine for other purposes.
It is only fast at the beginning, when there are a lot of torrent downloaders, and the available "classic" mirrors are also too busy. As to the open connections, you can configure a lower limit. For instance, I used: btdownloadcurses --spew 1 --max_upload_rate 25 --max_allow_in 12 file.torrent which allows only 12 incoming connections, as my output bandwidth is only 30, and those 60 would get only a droplet from me. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFGF3G1tTMYHG2NR9URAj7/AJ4xOcBtIoqHcPhsMKVWKR5WVtTYNACeJXnw 2ee6pMdATaRFgZd3KGwU6ec= =oKCP -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
G.T.Smith 写道:
I think you are rather hoping you can set it up and leave it.... if people start using it and it becomes popular they will need support and unless you have plenty of time it will be wise to consider who has access, how you monitor that access, how you stop the resource being compromised (security), and how you are going to assist the user community.
Unfortunately (for me) G. T. Smith said the truth: I even didn't realize myself that I was thinking how to save MY time doing this. I always want to have students who are interested to help administration, hoping that would reduce the work. This small project have little funds and we need to use existing resource. I googled a lot and found I am outdated: FTP protocol can do encoding conversion. There is a new RFC2640 specified how to do this. I think vsftpd can save me a lot of maintenance for being secure and simple, but I found vsftp does not follow this RFC. After I read the RFC I go to vsftpd source, features.c and add one line of code to make it complaint with the new RFC (so much of goodness of opensource). So now if Windows user use standard complaint FTP client (FileZilla or smartftp, later I didn't test only heard of), they can get the filenames in correct encoding. Security: as long as the server is not compromised, the file integrity is not my concern, because other campus services also have this problem (integrity) for years and they even offering .exe file for downloading (from a Windows server, oh, let's just hope they have patched everything), here I only got audio files, I am sure there will not be so much compliance. The solution would be offering vsftpd server as well as NFS client. Most users will only use FTP, so I guess it's easy to control access (plenty of options in vsftpd.conf for controlling access). The Linux library and laboratory computers can use NFS share. I am pretty sure NFS share will not be a traffic problem because Linux users are still only a very very small percentage.
There are other issues, your network support people may not be too happy if your archive stuffs the network if it gets popular, you may need to look into things like multi-casting and QoS with them. You may and your users may not be to happy if it the server collapses under the load. Your solution needs to take into account how many people are expected to use the resource, how often, and from where. While 100G may not seem a lot, 100 people accessing 100G is an awful lot of data moving around wires.
I am thinking about possible solutions:
1. FTP -> can handle heavy load, can do bath upload, not random-accessible, auto-charset conversion not supported;
Hmm. usually hard work for the user. PUTTY in the Windows world does offer a fairly simply command interface. Using cygwin on windows machines to setup the machines up as a X terminal is a further route.
2. apache -> batch download not easy for users, handle charset conversion nicely, not random access
Web is really down to how you setup the web access, it is up to you how easy for the users to access the data and how it is presented. External access becomes a viable option. Plenty of search options, and support pages can be setup. Probably easiest solution because you will have minimal security concerns, and only one thing to look after.
3. NFS -> I don't know any free-as-in-beer Windows client software for it and I don't know if that client software can do charset conversion; for Linux clients it's perfect;
NFS within a university environment is a security no-brainer. I believe NFS can be made to work under cygwin though I have not tried this myself.
4. Samba -> I don't know if charset conversion is easy with it. If a SuSE client connects to it, can suse client select which charset to use without forcing user to use commandline? And how about windows, can windows connect to the samba share and do charset conversion automatically?
For raw file store access within the institution is OK. External access usually a no-no. Sorting out authentication may be an interesting experience if you are within an AD environment. Samba performs most of things a domain server, you can set up the server end to use specific character sets but the interaction with client may a bit odd if client is configured for something different.
5. DC++ -> looks very nice for charset conversion, I also tried it, nice. But I don't know if there are Linux server-end software. Need to check.
If it is peer to peer access is what is required e-Mule/e-Donkey is another option, and there are other options. Personally do not use and do not recommend P2P, security is down to the weakest link and P2P is somewhat like unprotected sex... you never no what you are going to catch. P2P solutions do need careful thought about security.
I am thinking perhaps combine two solutions together might be the best, e.g. setting up FTP server for Windows users and set up NFS server for Linux users. Still I am not sure what's the best solution, and there might be better solutions than what I listed that I never heard of.
* In all above discussion "charset conversion" means charset conversion for file names, not the content. Content is in mp3/ogg format.
Thanks for any comments!
The thing to remember in the University environment you have a lot of talented individuals, and will regard any shared resource as fair game to so you need to think paranoid. It may be a war zone in the outside internet world but it can be like living in the middle of armageddon within a University.
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Fri, 2007-04-06 at 19:13 +0800, Zhang Weiwu wrote:
Recently I need to start a small project to distribute about 100GB megabytes of audio files to local university campus. I'd like to post my requirement hoping I can get some insightful recommendation on what software/technology to use to distribute these files.
Unless somebody comes up with a killer app that does everything, I think I'd separate the search requirements from the distribution requirements. And sadly, you may need to separate the character set requirements as well - I hope somebody knows of a complete solution. I don't have many thoughts on the search or character set issues. On the distribution side, you might also want to look at rsync and bittorrent. Cheers, Dave -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (14)
-
Anders Johansson -
Carlos E. R. -
Dave Howorth -
David Brodbeck -
G.T.Smith -
James Knott -
Jan Karjalainen -
jdd -
John Andersen -
M Harris -
Michael Skiba -
Randall R Schulz -
Ryouga Hibiki -
Zhang Weiwu