[opensuse] vsftpd issues
Oh boy! I almost don't want to start this discussion because I feel like I am wading into a real can of worms and could start a firestorm of opinions! In my previous thread, Per Jessen asked me to keep the list in the loop about what I am pursuing with vsftpd. So here goes... For those who do not want to do the long read, the short version of this posting is how do I replicate the code and build environment of vsftpd as it was done for the release of OpenSuSE Leap 15.0? This is NOT an easy question with a easy answer either, I tried the obvious and it failed. I have decided to poke into vsftpd because I (and many others if you Google the particular problem that got me started) have encountered an issue with what appears to be a problem in the client/server relationship, with vsftpd being the common denominator. I have not yet discovered what is the exact problem yet, mostly because I do not yet fully grok what the expectations are in the client/server protocols of FTP servers and with SSL/TLS encryption procedures, let alone the fact that I do not yet fully grok the vsftpd code either! ;-) In a nutshell, and again this is mostly speculation, it appears that when the SSL/TLS negotiations begin between a client (such as Filezilla) and vsftpd something can go wrong. I am apparently experiencing such a situation, and from the vsftpd log messages I was able to track down where in the vsftpd code that it first detected a problem. The error message I was seeing was extremely bad and unhelpful (IMHO) - "not a normal exit in vsf_sysutil_wait_get_exitcode" but at least it told me where to look in the code for a clue. I downloaded the source code (more about this later) and began studying it and it appears that this error message is wrapped up (prefaced with "500 OOPS: ") in such a way that it gets sent to the client in a desperate attempt to communicate the problem back to the user. (God only knows what a client/user is expected to do about it, let alone that it will be meaningless to most users!) I have not yet dug into the FTP protocols enough to know if a client is expected to show these messages to the user or not, but it appears (and again I am speculating) that a client such as Filezilla responds instead by telling the user that it received an unexpected TLS packet - "GnuTLS error -15: An unexpected TLS packet was received" Google that to see how much confusion it has caused users over the years! Yet another near useless error message (made worse by hiding the contents of the TLS packet that was received) with no guidance to a solution for the poor user, but that's Filezilla's problem. At this point I decided to explore in the opposite direction and see who is calling this function vsf_sysutil_wait_get_exitcode and I could not find any references to it in the rest of the vsftpd code. I am not undaunted by that fact however, there are other ways to call a function other than through a direct reference, so I decided to compile and build vsftpd so that I could instrument the code myself and see what is going on. I followed the instructions at https://en.opensuse.org/Source_code to get the "Exact source code", unpacked the tar file of source code, and applied two additional patch files that were indicated as being later than the released version of source code. One of the patch files contained changes that fixed a security hole which is nice but not necessary/relevant to the issue I am trying to address nor does it affect any of the code I am looking at. The other patch file fixed an external script, used by Makefile, so that the SSL/TLS libraries could be correctly located and installed if needed. I don't feel it was a good solution, and have a better proposal I could talk about later, but for now it does what is needed. BUT there are two issues that I would like to bring up - The first issue is that in order to build vsftpd I had to figure out what all libraries I needed to install on my system. That is not an easy process, and I had to do some research tracking down which library handled each un-referenced item errors that the build process complained about. This actually surprised me because it wasn't just developer packages and I would have thought that the dependency requirements of a package, such as for vsftpd, would have insured that all other necessary libraries are installed. But apparently there are different dependencies for source code development, and it wasn't just the developer packages, so if that is the case why can't the release developers simply include a list of libraries needed for each source package as well? (Maybe they are and I am not finding it? Or maybe/probably this is related to and caused by the second issue I talk about below... ) I particularly worry about version levels, that were used during the build/release process of vsftpd, of each library package. I just guessed and went with the latest but that may lead to having a version of a program that is different than what was installed by the release. I personally think it would be better if source packages were released like any other package, with a dependency list/tree behind it that insured all necessary libraries, including development packages, were installed. Didn't YaST and Zypper use to do that? The second and more serious issue is that the source code I downloaded is different from what was used in building vsftpd for the OpenSuSE 15.0 release. I know that the version of vsftpd distributed in Leap 15.0 had support for SSL/TLS built in. Yet I had to go into the source code that I downloaded and reconfigure a #define declaration in order to include SSL/TLS support. This is the first of a couple of indications that the source code I downloaded does NOT match up with what was used in the Leap 15.0 distribution. The second indication that the source code is different, came about when I finally managed to build and install vsftpd because I am no longer able to replicate the initial error with it, that got me started on this in the first place! I am apparently getting a different sort of error which I will continue to chase but not the one I wanted to go after. In fact with the new version of vsftpd that I built, there is no calls being made to the aforementioned function vsf_sysutil_wait_get_exitcode. So this is another strong indication, that the source code I downloaded is different, (probably through other configuration #defines or assignments) than what was used during the build of vsftpd for Leap 15.0. Without the patches, and making some changes to configuration parameters, there is no way that the source code I downloaded is the same as what was used during the build of Leap 15.0. And even after having made the changes I made, in order to get vsftpd to compile and build, I have no confidence now that I built and replicated vsftpd as it came with the distro, and for debugging purposes it is extremely important that I be able to do so. There are a lot more issues I could go into, coding standards, code reviews, IDE support to name a few. I know that open source software development is constrained by resources but these issues should also be discussed/addressed at some point. Addressing these could help people like myself (an experienced software developer) jump in and rapidly learn/grok code, such as I am doing for vsftpd. I would love to assist with debug and development, but don't want to spend a lot of time overcoming barriers. And that would make it easier for a project such as vsftpd to get more assistance/support. But for the moment, I wonder if the OpenSuSE development team could address the two issues I raised above and change the release process so that the source code and build environment could be managed better and focus on enabling an engineer to be able to replicate what gets released? Marc.. -- Linux Counter -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
06.02.2019 1:29, Marc Chamberlin пишет:
on. I followed the instructions at https://en.opensuse.org/Source_code to get the "Exact source code", unpacked the tar file of source code, and applied two additional patch files that were indicated as being later than the released version of source code.
vsftpd SRPM has 30 patches. If you applied just two of them you do not build vsftpd as provided by openSUSE. If you *manually* (gasp) apply patches you also do not build vsftpd as provided by openSUSE.
One of the patch files contained changes that fixed a security hole which is nice but not necessary/relevant to the issue I am trying to address nor does it affect any of the code I am looking at. The other patch file fixed an external script, used by Makefile, so that the SSL/TLS libraries could be correctly located and installed if needed. I don't feel it was a good solution, and have a better proposal I could talk about later, but for now it does what is needed.
BUT there are two issues that I would like to bring up -
The first issue is that in order to build vsftpd I had to figure out what all libraries I needed to install on my system. That is not an easy process,
Which is why everyone else either installs SRPM using zypper (which also installs needed dependencies) or uses osc/OBS to build (which installs dependencies automatically when initializing build environment). The latter is much preferred because it builds in clean room environment so eliminates risk of silently picking the wrong configuration during package build.
The second and more serious issue is that the source code I downloaded is different from what was used in building vsftpd for the OpenSuSE 15.0 release.
Link you referenced describes several ways to obtain source code, and in your 100 pages treatise you never bothered to say which one you followed. But if you did not even apply all patches - of course source code will be different.
I know that the version of vsftpd distributed in Leap 15.0 had support for SSL/TLS built in. Yet I had to go into the source code that I downloaded and reconfigure a #define declaration in order to include SSL/TLS support.
Huh? Casual glance in SPEC file shows that it builds vsftpd with SSL support. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Thanks Andrei, I appreciate your trying to help me out, and it appears I may be going about getting the source code incorrectly. But I remain confused and will reply to your comments below - On 02/05/2019 08:22 PM, Andrei Borzenkov wrote:
on. I followed the instructions at https://en.opensuse.org/Source_code to get the "Exact source code", unpacked the tar file of source code, and applied two additional patch files that were indicated as being later than the released version of source code. vsftpd SRPM has 30 patches. If you applied just two of them you do not build vsftpd as provided by openSUSE. If you *manually* (gasp) apply
06.02.2019 1:29, Marc Chamberlin пишет: patches you also do not build vsftpd as provided by openSUSE. After I download the source code, as described in further detail below, I saw a tar file containing the source code labeled -
vsftpd-3.0.3.tar Actually it was originally a zip file - vsftpd-3.0.3.tar.gz. The label gave me the impression that this contained all the patches that had version numbers below 3.0.3 so I simply applied patches that were labeled 3.0.3. There was no documentation that gave me reason to believe otherwise or even explained the patch model so I had to make an educated guess. And yes making assumptions are not always a good idea but then without good documentation one doesn't always know if/when one is making a bad assumption either... If this tar file had been labeled vsftpd-1.0.0 then I would have known that it was the original source code and that all the patches had to be applied... I am sorry but I would have to say this is confusing! And yeah, I would hate to have to manually apply 30 some patches, and if I had to I would have made a different complaint... Please forgive me for having made the assumption, about the 3.0.3 label, that someone had gone to the trouble of saving me from the pain of applying all the other patches, by bundling them all up in that tar file.
One of the patch files contained changes that fixed a security hole which is nice but not necessary/relevant to the issue I am trying to address nor does it affect any of the code I am looking at. The other patch file fixed an external script, used by Makefile, so that the SSL/TLS libraries could be correctly located and installed if needed. I don't feel it was a good solution, and have a better proposal I could talk about later, but for now it does what is needed.
BUT there are two issues that I would like to bring up -
The first issue is that in order to build vsftpd I had to figure out what all libraries I needed to install on my system. That is not an easy process, Which is why everyone else either installs SRPM using zypper (which also installs needed dependencies) or uses osc/OBS to build (which installs dependencies automatically when initializing build environment). The latter is much preferred because it builds in clean room environment so eliminates risk of silently picking the wrong configuration during package build.
OK, but this is NOT explained on the web page that I was directed to, to download source files. Please explain this in more detail, or provide me with a link to better documentation, because I am unfamiliar with how OpenSuSE expects developers to download source code. I didn't ask, up front, how to acquire the source code because then I would likely get told to RTFM, so instead I did RTFM first and apparently that has gotten me into trouble... Sigh...
The second and more serious issue is that the source code I downloaded is different from what was used in building vsftpd for the OpenSuSE 15.0 release. Link you referenced describes several ways to obtain source code, and in your 100 pages treatise you never bothered to say which one you followed. But if you did not even apply all patches - of course source code will be different.
But I think I did say which method I used, it was described on the web page at https://en.opensuse.org/Source_code in the section titled "Exact source code" which I referenced in my treatise (though I guess not as explicitly as I could have) using quotes even. I will copy/paste here. Exact source code To retrieve the exact source code of a single package that is installed on your system, query the package first for it unique source package url: rpm -q --qf '%{disturl}\n' INSTALLED_PACKAGE_NAME and you will receive a result of the following form: obs://build.opensuse.org/PROJECT_NAME/RELEASE_HEX-PACKAGE_NAME Then, you can use this information to check out the sources with the osc command line command: osc co -r RELEASE_HEX PROJECT_NAME PACKAGE_NAME I am sorry Andrie for having written a 100 page treatise, but I cannot read developers minds, nor can they read mine, we can only read each other's written documentation. And I know, from past experience that when I communicate with other developers I must try to be as clear and as precise as I can be. And sometimes that takes a lot of words and a lot of effort! ;-)
I know that the version of vsftpd distributed in Leap 15.0 had support for SSL/TLS built in. Yet I had to go into the source code that I downloaded and reconfigure a #define declaration in order to include SSL/TLS support. Huh? Casual glance in SPEC file shows that it builds vsftpd with SSL support.
I agree that the installed version of vsftpd is built with SSL support. But the source code I downloaded, using the documented procedures I found, along with the assumptions I made, as described above, the resulting code was NOT configured to build in SSL support. I had to manually change a #define in order to do so. That alone would not have bothered me nor prompted me to write my treatise, but combined with other observed differences, it did because I do not (did not) understand why the source code I got differs in behavior from what was installed in the released version... Marc... -- Linux Counter -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 02/06/2019 01:11 AM, Marc Chamberlin wrote:
I agree that the installed version of vsftpd is built with SSL support. But the source code I downloaded, using the documented procedures I found, along with the assumptions I made, as described above, the resulting code was NOT configured to build in SSL support.
Don't know if this is related to your SSL/TLS, but bits and pieces of TLS have already been deprecated. (I use vsftpd internally without SSL/TLS for copier scan to server) I haven't paid exacts attention to when all the various apps will pull early and general TLS, but here are a couple of links (just google 'tls deprecated') Are You Ready for 30 June 2018? Saying Goodbye to SSL/early TLS https://blog.pcisecuritystandards.org/are-you-ready-for-30-june-2018-sayin-g... Chrome, Edge, IE, Firefox, and Safari to disable TLS 1.0 and TLS 1.1 ... https://www.zdnet.com/article/chrome-edge-ie-firefox-and-safari-to-disable-t... So on top of simply building, you may need to verify whatever is trying to access vsftp doesn't have TLS disabled. Andrei is being kind with the srpm hint. Just install the srpm (as user not root) and install the rpm-build package, and after install you will have ~/rpmbuild/SPECS/vsftp.spec spec-file in your SPECS folder. Then simply: rpmbuild -ba ~/rpmbuild/SPECS/vsftp.spec (to -ba build-all, source rpms and binaries, or -bb just build binaries) and watch the magic happen (or fix what croaks, then watch it happen) After the rpm-build install and your first srpm, you will have the following in you ~/rpmbuild directory: l1 ~/rpmbuild/ BUILD BUILDROOT OTHER RPMS SOURCES SPECS SRPMS Where the sources and patches will be in SOURCES, .specs in SPECS, completed .srpm and binaries in SRPMS and RPMS with build files and miscellaneous others in the rest. Good luck -- David C. Rankin, J.D.,P.E. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Wed, Feb 6, 2019 at 10:35 AM David C. Rankin <drankinatty@suddenlinkmail.com> wrote:
Just install the srpm (as user not root) and install the rpm-build package, and after install you will have ~/rpmbuild/SPECS/vsftp.spec spec-file in your SPECS folder. Then simply:
rpmbuild -ba ~/rpmbuild/SPECS/vsftp.spec
That is rather bad advice if intent is to replicate distribution package. It is executed in environment of your currently installed system. Which means that as soon as your environment differs from the one used to build packages (and it always differs), you risk that scripts in your package pick up different libraries, dependencies etc. They may simply skip what is not found. The order of search may be so that they find different implementation (as example GNU TLS vs. OpenSSL). They may use different compiler options. Etc etc etc. That is why I mentioned osc and/or OBS that do it in clean environment that always matches expectations of package maintainer.
(to -ba build-all, source rpms and binaries, or -bb just build binaries)
Or rpmbuild -bp to simply extract sources and apply all patches (and do whatever maintainer considered necessary to prepare sources for building) if intention is to simply look at it. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 02/05/2019 11:35 PM, David C. Rankin wrote:
Don't know if this is related to your SSL/TLS, but bits and pieces of TLS have already been deprecated. (I use vsftpd internally without SSL/TLS for copier scan to server) I haven't paid exacts attention to when all the various apps will pull early and general TLS, but here are a couple of links (just google 'tls deprecated')
Are You Ready for 30 June 2018? Saying Goodbye to SSL/early TLS https://blog.pcisecuritystandards.org/are-you-ready-for-30-june-2018-sayin-g...
Chrome, Edge, IE, Firefox, and Safari to disable TLS 1.0 and TLS 1.1 ... https://www.zdnet.com/article/chrome-edge-ie-firefox-and-safari-to-disable-t...
David - Yeah I have disabled early SSL/TLS protocol handling on all my servers, Apache, James, VSFTPD etc., since it has been long known there are security holes in those protocols. But thanks for mentioning this... When I mentioned I had to enable SSL in VSFTPD I assume the author of VSFTPD is using the $define SSL configuration macro in the common sense that includes TLS also... So yeah to be clear I am only focused on supporting TLS 1.2 on... The Filezilla client that I am using for testing does not even allow for SSL protocols, just TLS but it is unclear what version of TLS it is using. As I dig into VSFTPD I will double check to see what it is using in the way of TLS (and also if SSL is disabled) but I kinda suspect that is more in the realm of the openSSL libraries purview and/or perhaps constrained by the certificates... Marc.. -- Linux Counter -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (3)
-
Andrei Borzenkov
-
David C. Rankin
-
Marc Chamberlin