[opensuse] Unwanted DNS Queries
Hi all: I have a question about some DNS querys/responses that continue after the application that originally requested them is closed. I am running Tumbleweed (20190426 snapshot)on x86_64, Firefox version 66.0.3. The Setup: run Firefox and browse multiple sites. Close Firefox. Open Wireshark. Note multiple DNS querys/responses are still being made from/to some process (Firefox does not appear in a ps query) related to the sites just visited. In a recent test, I surfed to seven different websites in three minutes, closed Firefox, opened Wireshark, and noticed 32 DNS queries and 32 responses, all related to sites I just visited (and their "partners"). Letting Wireshark run, this same sequence was repeated every few seconds for 10 minutes, when I closed Wireshark. I have no idea how long this process will continue. I have also tried this same test on Fedora and OpenBSD, with the same version of Firefox, with the same configuration, and the issue does not present. I don't happen to prefer this behaviour, so I have two questions: 1. Why is this happening on OpenSuSE? 2. How do I disable it? All constructive replies are greatly appreciated! -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 29/04/2019 18.14, Dutch Ingraham wrote:
Hi all:
I have a question about some DNS querys/responses that continue after the application that originally requested them is closed.
I am running Tumbleweed (20190426 snapshot)on x86_64, Firefox version 66.0.3.
The Setup: run Firefox and browse multiple sites. Close Firefox. Open Wireshark. Note multiple DNS querys/responses are still being made from/to some process (Firefox does not appear in a ps query) related to the sites just visited.
In a recent test, I surfed to seven different websites in three minutes, closed Firefox, opened Wireshark, and noticed 32 DNS queries and 32 responses, all related to sites I just visited (and their "partners").
Letting Wireshark run, this same sequence was repeated every few seconds for 10 minutes, when I closed Wireshark. I have no idea how long this process will continue.
I have also tried this same test on Fedora and OpenBSD, with the same version of Firefox, with the same configuration, and the issue does not present.
I don't happen to prefer this behaviour, so I have two questions:
1. Why is this happening on OpenSuSE? 2. How do I disable it?
All constructive replies are greatly appreciated!
Wild guess. There is some DNS cache app running, which has an entry for "somesite" that has a short lifetime. Maybe that DNS cache app refreshes the IP before the timeout in order to have it ready for any application requesting it. -- Cheers / Saludos, Carlos E. R. (from openSUSE, Leap 15.1 x86_64 (ssd-test)) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 4/29/19 11:19 AM, Carlos E. R. wrote:
On 29/04/2019 18.14, Dutch Ingraham wrote:
Hi all:
I have a question about some DNS querys/responses that continue after the application that originally requested them is closed.
I am running Tumbleweed (20190426 snapshot)on x86_64, Firefox version 66.0.3.
The Setup: run Firefox and browse multiple sites. Close Firefox. Open Wireshark. Note multiple DNS querys/responses are still being made from/to some process (Firefox does not appear in a ps query) related to the sites just visited.
In a recent test, I surfed to seven different websites in three minutes, closed Firefox, opened Wireshark, and noticed 32 DNS queries and 32 responses, all related to sites I just visited (and their "partners").
Letting Wireshark run, this same sequence was repeated every few seconds for 10 minutes, when I closed Wireshark. I have no idea how long this process will continue.
I have also tried this same test on Fedora and OpenBSD, with the same version of Firefox, with the same configuration, and the issue does not present.
I don't happen to prefer this behaviour, so I have two questions:
1. Why is this happening on OpenSuSE? 2. How do I disable it?
All constructive replies are greatly appreciated!
Wild guess.
There is some DNS cache app running, which has an entry for "somesite" that has a short lifetime. Maybe that DNS cache app refreshes the IP before the timeout in order to have it ready for any application requesting it.
Thanks, Carlos. I'm sure you are correct, but identifying that cache app is the harder part. I took an uninformed chance that it may be systemd-resolved, which caches by default, and provides for turning the caching off, but that had no apparent effect on the issue. BTW - I started wireshark about 4 hours later, and the same DNS queries were still being made. There must be some kind of timeout, but hard to say what it is set to until the offending daemon is properly identified. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 30/04/2019 18.04, Dutch Ingraham wrote:
On 4/29/19 11:19 AM, Carlos E. R. wrote:
On 29/04/2019 18.14, Dutch Ingraham wrote:
Wild guess.
There is some DNS cache app running, which has an entry for "somesite" that has a short lifetime. Maybe that DNS cache app refreshes the IP before the timeout in order to have it ready for any application requesting it.
Thanks, Carlos. I'm sure you are correct, but identifying that cache app is the harder part. I took an uninformed chance that it may be systemd-resolved, which caches by default, and provides for turning the caching off, but that had no apparent effect on the issue.
BTW - I started wireshark about 4 hours later, and the same DNS queries were still being made. There must be some kind of timeout, but hard to say what it is set to until the offending daemon is properly identified.
It could be dnsmasq, nscd... The later is active by default. -- Cheers / Saludos, Carlos E. R. (from openSUSE, Leap 15.1 x86_64 (ssd-test)) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Hi, On Leap 15 (and earlier), it's nscd (Name Service Caching Daemon). Once you make a request, it tries to keep the queried information fresh in the cache, so it re-queries the information when it expires. The only way I've found to stop this is to shut the daemon down. Brendan On 30/04/2019 17:26, Carlos E. R. wrote:
On 30/04/2019 18.04, Dutch Ingraham wrote:
On 4/29/19 11:19 AM, Carlos E. R. wrote:
On 29/04/2019 18.14, Dutch Ingraham wrote:
Wild guess.
There is some DNS cache app running, which has an entry for "somesite" that has a short lifetime. Maybe that DNS cache app refreshes the IP before the timeout in order to have it ready for any application requesting it.
Thanks, Carlos. I'm sure you are correct, but identifying that cache app is the harder part. I took an uninformed chance that it may be systemd-resolved, which caches by default, and provides for turning the caching off, but that had no apparent effect on the issue.
BTW - I started wireshark about 4 hours later, and the same DNS queries were still being made. There must be some kind of timeout, but hard to say what it is set to until the offending daemon is properly identified.
It could be dnsmasq, nscd... The later is active by default.
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 30/04/2019 20.35, Brendan McKenna wrote:
Hi,
On Leap 15 (and earlier), it's nscd (Name Service Caching Daemon). Once you make a request, it tries to keep the queried information fresh in the cache, so it re-queries the information when it expires. The only way I've found to stop this is to shut the daemon down.
You can simple disable the DNS part of it. /etc/nscd.conf: enable-cache hosts no But that will make your machine "internet" slower, unless you have a DNS server in the same machine or in your local network and it responds fast. -- Cheers / Saludos, Carlos E. R. (from openSUSE, Leap 15.1 x86_64 (ssd-test)) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2019-04-30 5:19 p.m., Carlos E. R. wrote:
On 30/04/2019 20.35, Brendan McKenna wrote:
Hi,
On Leap 15 (and earlier), it's nscd (Name Service Caching Daemon). Once you make a request, it tries to keep the queried information fresh in the cache, so it re-queries the information when it expires. The only way I've found to stop this is to shut the daemon down.
You can simple disable the DNS part of it.
/etc/nscd.conf:
enable-cache hosts no
How does nscd interact with dnsmasq> -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 01/05/2019 00.47, Anton Aylward wrote:
On 2019-04-30 5:19 p.m., Carlos E. R. wrote:
On 30/04/2019 20.35, Brendan McKenna wrote:
Hi,
On Leap 15 (and earlier), it's nscd (Name Service Caching Daemon). Once you make a request, it tries to keep the queried information fresh in the cache, so it re-queries the information when it expires. The only way I've found to stop this is to shut the daemon down.
You can simple disable the DNS part of it.
/etc/nscd.conf:
enable-cache hosts no
How does nscd interact with dnsmasq>
They don't. However, I think nscd asks the configured DNS server, which can be dnsmask or any other, local or not, about host names. -- Cheers / Saludos, Carlos E. R. (from openSUSE, Leap 15.1 x86_64 (ssd-test)) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2019-04-30 8:32 p.m., Carlos E. R. wrote:
On 01/05/2019 00.47, Anton Aylward wrote:
On 2019-04-30 5:19 p.m., Carlos E. R. wrote:
On 30/04/2019 20.35, Brendan McKenna wrote:
Hi,
On Leap 15 (and earlier), it's nscd (Name Service Caching Daemon). Once you make a request, it tries to keep the queried information fresh in the cache, so it re-queries the information when it expires. The only way I've found to stop this is to shut the daemon down.
You can simple disable the DNS part of it.
/etc/nscd.conf:
enable-cache hosts no
How does nscd interact with dnsmasq>
They don't.
However, I think nscd asks the configured DNS server, which can be dnsmask or any other, local or not, about host names.
It probably matters on whether or not dnsmasq caches. As far as I can tell it can be configured either way. I'm running default # Set the cachesize here. #cache-size=150 I suppose I could increase that, or set it to zero to disable caching. There are sooooo many things that dnsmasq can do! The thing is, when an application makes a DNS query, what is the logical path of calls that it follows? There are just so many possibilities that come to mind "A guide for the perplexed" with a decision chart diagram, branching mindmap or something would be nice :-) -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 01/05/2019 04.40, Anton Aylward wrote:
On 2019-04-30 8:32 p.m., Carlos E. R. wrote:
On 01/05/2019 00.47, Anton Aylward wrote:
On 2019-04-30 5:19 p.m., Carlos E. R. wrote:
On 30/04/2019 20.35, Brendan McKenna wrote:
Hi,
On Leap 15 (and earlier), it's nscd (Name Service Caching Daemon). Once you make a request, it tries to keep the queried information fresh in the cache, so it re-queries the information when it expires. The only way I've found to stop this is to shut the daemon down.
You can simple disable the DNS part of it.
/etc/nscd.conf:
enable-cache hosts no
How does nscd interact with dnsmasq>
They don't.
However, I think nscd asks the configured DNS server, which can be dnsmask or any other, local or not, about host names.
It probably matters on whether or not dnsmasq caches. As far as I can tell it can be configured either way.
AFAIK, all DNS servers caches. Whatever they ask from outside they keep if they can. What would be the point of a distributed network of DN servers, if for every question they have to ask upstream? The master servers would suffer the compound load of the entire world.
I'm running default # Set the cachesize here. #cache-size=150 I suppose I could increase that, or set it to zero to disable caching.
There are sooooo many things that dnsmasq can do!
The thing is, when an application makes a DNS query, what is the logical path of calls that it follows? There are just so many possibilities that come to mind "A guide for the perplexed" with a decision chart diagram, branching mindmap or something would be nice :-)
I guess the query is first answered or intercepted by nscd, then it goes to the configured DNS. Ah, wait, first it goes to the host file, which I think is cached by nscd. -- Cheers / Saludos, Carlos E. R. (from openSUSE, Leap 15.1 x86_64 (ssd-test)) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2019-05-01 6:51 a.m., Carlos E. R. wrote:
The thing is, when an application makes a DNS query, what is the logical path of calls that it follows? There are just so many possibilities that come to mind "A guide for the perplexed" with a decision chart diagram, branching mindmap or something would be nice :-)
I guess the query is first answered or intercepted by nscd, then it goes to the configured DNS. Ah, wait, first it goes to the host file, which I think is cached by nscd.
Perhaps I didn't make myself clear. When an application such as Firefox make a library call "gethostbyname()" (or is that obsolete now?) what happens? I may be wrong but I think the first thing that happens is that there is a look-see to /etc/nsswitch.conf to see what should be invoked. Mine says: hosts: files dns which, I take it, means check /etc/hosts first and then use the configired DNS resolver if the address isn't found in the hosts file. Two things occur to me at this point. The first is that dnsmasq slurps up /etc/hosts anyway so the 'files' entry is redundant. The second is I don't see how things get to dnsmasq. Can someone help me here, please. Sorry Carlos, "Intercepted' is just too vague a concept. Somehow, somewhere there is a decision-in-software, an 'if" statement based on a config file entry, and not a beady-eyed raptor waiting to swoop down on an unsuspecting rodent. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 01/05/2019 13.22, Anton Aylward wrote:
On 2019-05-01 6:51 a.m., Carlos E. R. wrote:
The thing is, when an application makes a DNS query, what is the logical path of calls that it follows? There are just so many possibilities that come to mind "A guide for the perplexed" with a decision chart diagram, branching mindmap or something would be nice :-)
I guess the query is first answered or intercepted by nscd, then it goes to the configured DNS. Ah, wait, first it goes to the host file, which I think is cached by nscd.
Perhaps I didn't make myself clear. When an application such as Firefox make a library call "gethostbyname()" (or is that obsolete now?) what happens?
I may be wrong but I think the first thing that happens is that there is a look-see to /etc/nsswitch.conf to see what should be invoked. Mine says: hosts: files dns which, I take it, means check /etc/hosts first and then use the configired DNS resolver if the address isn't found in the hosts file.
Two things occur to me at this point. The first is that dnsmasq slurps up /etc/hosts anyway so the 'files' entry is redundant.
Before that point (before reading from /etc/hosts) is the nscd intercept.
The second is I don't see how things get to dnsmasq. Can someone help me here, please.
Sorry Carlos, "Intercepted' is just too vague a concept. Somehow, somewhere there is a decision-in-software, an 'if" statement based on a config file entry, and not a beady-eyed raptor waiting to swoop down on an unsuspecting rodent.
nscd is not in the "official" loop, IMHO, so instead it puts itself in the middle intercepting some library calls and giving an answer without the library reaching its designed goal of reading the hosts file or querying a DNS. I can not explain in detail because I do not know, I simply guess. The getting to dnsmasq is the same thing as getting to bind, or getting to the caching server in your router. No difference. It is just a DNS server somewhere in the entire world, just that it happens to sit locally. -- Cheers / Saludos, Carlos E. R. (from openSUSE, Leap 15.1 x86_64 (ssd-test)) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2019-05-01 7:47 a.m., Carlos E. R. wrote:
On 01/05/2019 13.22, Anton Aylward wrote:
Perhaps I didn't make myself clear. When an application such as Firefox make a library call "gethostbyname()" (or is that obsolete now?) what happens?
I may be wrong but I think the first thing that happens is that there is a look-see to /etc/nsswitch.conf to see what should be invoked. Mine says: hosts: files dns which, I take it, means check /etc/hosts first and then use the configired DNS resolver if the address isn't found in the hosts file.
Two things occur to me at this point. The first is that dnsmasq slurps up /etc/hosts anyway so the 'files' entry is redundant.
Before that point (before reading from /etc/hosts) is the nscd intercept.
You mean that -- somehow, some unspecified manner -- that happens before the decision as to whether to use files or DNS or something else as determined by the nsswitch.conf file is accessed? As I said, I'm using dnsmasq and NOT nscd. The decision path still has to be there.
The second is I don't see how things get to dnsmasq. Can someone help me here, please.
Sorry Carlos, "Intercepted' is just too vague a concept. Somehow, somewhere there is a decision-in-software, an 'if" statement based on a config file entry, and not a beady-eyed raptor waiting to swoop down on an unsuspecting rodent.
nscd is not in the "official" loop, IMHO, so instead it puts itself in the middle intercepting some library calls
HOW? Right now you are talking the realm of magic and I don't accept it. This is a computer. there HAS to be a deterministic, repeatable, identifiable, documented mechanism that works for everyone, even those of us living in mana-poor areas of the world. "Intercept" is too vague a concept. There HAS to be a decision path arising from a configuration file setting. This beady-eyed raptor waiting to whoop down and 'intercept' is too asynchronous, to inconsistent and unreliable.
and giving an answer without the library reaching its designed goal of reading the hosts file or querying a DNS.
I can not explain in detail because I do not know, I simply guess.
I too guess, but then I investigate and look for documentation. If I can't find it I discard that guess and try antoehr one. I've found that dnsmasq wants to listen locally so there should be an entry in the /etc/resolv.conf file of 127.0.0.1 Now I need to figure out how "gethostentrybyname()" ends up at /etc/resolv.conf
The getting to dnsmasq is the same thing as getting to bind, or getting to the caching server in your router. No difference. It is just a DNS server somewhere in the entire world, just that it happens to sit locally.
-- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 01/05/2019 14.15, Anton Aylward wrote:
On 2019-05-01 7:47 a.m., Carlos E. R. wrote:
On 01/05/2019 13.22, Anton Aylward wrote:
Perhaps I didn't make myself clear. When an application such as Firefox make a library call "gethostbyname()" (or is that obsolete now?) what happens?
I may be wrong but I think the first thing that happens is that there is a look-see to /etc/nsswitch.conf to see what should be invoked. Mine says: hosts: files dns which, I take it, means check /etc/hosts first and then use the configired DNS resolver if the address isn't found in the hosts file.
Two things occur to me at this point. The first is that dnsmasq slurps up /etc/hosts anyway so the 'files' entry is redundant.
Before that point (before reading from /etc/hosts) is the nscd intercept.
You mean that -- somehow, some unspecified manner -- that happens before the decision as to whether to use files or DNS or something else as determined by the nsswitch.conf file is accessed?
No, after the decision. If the decision is files, it is nscd who answers what the files say. If the decision is bind, it is nscd who answers what the bind says. Or, maybe it intercepts, find out he doesn't know the answer, lets the call to the library continue, and when there is an answer it stores the answer in the cache for the next time.
As I said, I'm using dnsmasq and NOT nscd.
Did you notice that nscd also caches other services, and that you can disable just dns?
The decision path still has to be there.
But it seems that nscd is not part of the design. It is an afterthought by someone (maybe the same people, I do not know). man: «nscd provides caching for accesses of the passwd(5), group(5), hosts(5) services(5) and netgroup databases through standard libc interfaces, such as getpwnam(3), getpwuid(3), getgrnam(3), getgrgid(3), gethostbyname(3), and others.» I don't know if it intercepts the calls to those functions, or if it replaces them in full. I don't know the exact mechanism.
The second is I don't see how things get to dnsmasq. Can someone help me here, please.
Sorry Carlos, "Intercepted' is just too vague a concept. Somehow, somewhere there is a decision-in-software, an 'if" statement based on a config file entry, and not a beady-eyed raptor waiting to swoop down on an unsuspecting rodent.
nscd is not in the "official" loop, IMHO, so instead it puts itself in the middle intercepting some library calls
HOW? Right now you are talking the realm of magic and I don't accept it. This is a computer. there HAS to be a deterministic, repeatable, identifiable, documented mechanism that works for everyone, even those of us living in mana-poor areas of the world.
So?
"Intercept" is too vague a concept. There HAS to be a decision path arising from a configuration file setting. This beady-eyed raptor waiting to whoop down and 'intercept' is too asynchronous, to inconsistent and unreliable.
Didn't you study the interrupt intercept concept used in MsDOS? It is a similar thing. ISR? Only that I don't know how exactly it is done in Linux. Suppose there is a table somewhere that says calls to "gethostbyname() are done to this address". Suppose nscd replaces the address with his own. Intercept. Or something vaguely similar, I don't know the exact mechanism in Linux. You sure can work with vague concepts and incomplete information, that's a human feature.
and giving an answer without the library reaching its designed goal of reading the hosts file or querying a DNS.
I can not explain in detail because I do not know, I simply guess.
I too guess, but then I investigate and look for documentation. If I can't find it I discard that guess and try antoehr one.
Well, I have read programming guides in Linux but this detail I have not found.
I've found that dnsmasq wants to listen locally so there should be an entry in the /etc/resolv.conf file of 127.0.0.1
Or not. cat /etc/resolv.conf: nameserver 192.168.1.16 nameserver 80.58.61.250 nameserver 80.58.61.254 192.168.1.16 is dnsmasq on another computer. As far as your computer knows, it is simply asking a DN server on some machine, that in your case happens to be the same one, and it happens that the service is done by dnsmasq. There is no difference. -- Cheers / Saludos, Carlos E. R. (from openSUSE, Leap 15.1 x86_64 (ssd-test)) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2019-05-01 8:44 a.m., Carlos E. R. wrote:
Before that point (before reading from /etc/hosts) is the nscd intercept. You mean that -- somehow, some unspecified manner -- that happens before the decision as to whether to use files or DNS or something else as determined by the nsswitch.conf file is accessed?
No, after the decision.
So you are saying that nsswitch.conf is accessed then ..
If the decision is files, it is nscd who answers what the files say.
HOW? you are doing the 'by magic' (aka unspecified) again. "if the decision is 'files' ..." well its not about 'decision. if the refile reads: hosts: files dns that means 'try files first". That means READING a file. presumably /etc/hosts. NOT invoking a routine or calling a network connection.
If the decision is bind, it is nscd who answers what the bind says.
More magic. Its not about a 'decision', it is IF AND ONLY IF the 'files' returns STATUS = notfound And where does 'bind' come into this? I'm running dnsmasq, not Bind9. I think you mean, reading the MAN(nsswitch.conf) where it says "The order of the services on the line determines the order in which those services will be queried, in turn, until a result is found." So if the answer isn't find in "files" then it uses "dns". But how does that get to invoke Bind9, nscd or dnsmasq? In a deterministic, procedural manner? As determined by some configuration!
Or, maybe it intercepts,
No, it's the beady-eyed raptor that intercepts ...
find out he doesn't know the answer,
You mean "its not in the cache". I can see how that logic applies to both nscd and dnsmasq, but it still doesn't say how it got to them in the first place.
lets the call to the library continue, and when there is an answer it stores the answer in the cache for the next time.
I think that "library continues" needs a LOT more clarification. And yes, Bind9 and dnsmasq can be configured to do their own caching. At which point the relevance of nscd becomes apropos. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 01/05/2019 15.34, Anton Aylward wrote:
On 2019-05-01 8:44 a.m., Carlos E. R. wrote:
Before that point (before reading from /etc/hosts) is the nscd intercept. You mean that -- somehow, some unspecified manner -- that happens before the decision as to whether to use files or DNS or something else as determined by the nsswitch.conf file is accessed?
No, after the decision.
So you are saying that nsswitch.conf is accessed then ..
If the decision is files, it is nscd who answers what the files say.
HOW? you are doing the 'by magic' (aka unspecified) again.
Where is the magic? I don't see it. It is you who first used the word "decision". If "/etc/host.conf" says: order files, bind then the "decision" is first to ask the local files, ie, /etc/hosts.
"if the decision is 'files' ..." well its not about 'decision. if the refile reads:
hosts: files dns
that means 'try files first". That means READING a file. presumably /etc/hosts.
NOT invoking a routine or calling a network connection.
Who said different?
If the decision is bind, it is nscd who answers what the bind says.
More magic.
No magic.
Its not about a 'decision', it is IF AND ONLY IF the 'files' returns STATUS = notfound
And where does 'bind' come into this? I'm running dnsmasq, not Bind9.
Because the wording in "/etc/host.conf" is bind, not dns. See "man host.conf": order This keyword specifies how host lookups are to be performed. It should be followed by one or more lookup methods, separated by commas. Valid methods are bind, hosts, and nis.
I think you mean, reading the MAN(nsswitch.conf) where it says "The order of the services on the line determines the order in which those services will be queried, in turn, until a result is found."
So if the answer isn't find in "files" then it uses "dns". But how does that get to invoke Bind9, nscd or dnsmasq? In a deterministic, procedural manner? As determined by some configuration!
Again, it does not decide what daemon to ask. It ask 127.0.0.1 or 192.168.0.1. The address is written in hosts. Who ever answers. Not its business to know what daemon actually answers. If nscd is running, then the library is hijacked, intercepted or however you want to call it, to ask nscd first.
Or, maybe it intercepts,
No, it's the beady-eyed raptor that intercepts ...
find out he doesn't know the answer,
You mean "its not in the cache".
Yes, of course. Did you think it is a person, a magician? Of course I mean "its not in the cache". But I did not use that wording.
I can see how that logic applies to both nscd and dnsmasq, but it still doesn't say how it got to them in the first place.
nscd: intercept the library call. dnsmasq: make an internet call, that happens in your case to be on the same computer.
lets the call to the library continue, and when there is an answer it stores the answer in the cache for the next time.
I think that "library continues" needs a LOT more clarification.
But not from me.
And yes, Bind9 and dnsmasq can be configured to do their own caching. At which point the relevance of nscd becomes apropos.
nscd is not at the same level as the rest. -- Cheers / Saludos, Carlos E. R. (from openSUSE, Leap 15.1 x86_64 (ssd-test)) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Anton Aylward wrote:
On 2019-05-01 7:47 a.m., Carlos E. R. wrote:
On 01/05/2019 13.22, Anton Aylward wrote:
Perhaps I didn't make myself clear. When an application such as Firefox make a library call "gethostbyname()" (or is that obsolete now?) what happens?
I may be wrong but I think the first thing that happens is that there is a look-see to /etc/nsswitch.conf to see what should be invoked. Mine says: hosts: files dns which, I take it, means check /etc/hosts first and then use the configired DNS resolver if the address isn't found in the hosts file.
Two things occur to me at this point. The first is that dnsmasq slurps up /etc/hosts anyway so the 'files' entry is redundant.
Before that point (before reading from /etc/hosts) is the nscd intercept.
You mean that -- somehow, some unspecified manner -- that happens before the decision as to whether to use files or DNS or something else as determined by the nsswitch.conf file is accessed?
As I said, I'm using dnsmasq and NOT nscd. The decision path still has to be there.
When an application calls getaddrinfo(), the first thing it does is check with nscd over a unix socket. nscd is a standard part of libc.
I've found that dnsmasq wants to listen locally so there should be an entry in the /etc/resolv.conf file of 127.0.0.1 Now I need to figure out how "gethostentrybyname()" ends up at /etc/resolv.conf
That is done by the resolver library. If nscd is not available or has no useful answer, the resolver continues by looking at nsswitch.ch, then it proceeds according to that config. -- Per Jessen, Zürich (16.9°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 01/05/2019 15.55, Per Jessen wrote:
Anton Aylward wrote:
On 2019-05-01 7:47 a.m., Carlos E. R. wrote:
On 01/05/2019 13.22, Anton Aylward wrote:
Before that point (before reading from /etc/hosts) is the nscd intercept.
You mean that -- somehow, some unspecified manner -- that happens before the decision as to whether to use files or DNS or something else as determined by the nsswitch.conf file is accessed?
As I said, I'm using dnsmasq and NOT nscd. The decision path still has to be there.
When an application calls getaddrinfo(), the first thing it does is check with nscd over a unix socket. nscd is a standard part of libc.
Ok, so an intercept in source code :-P Ok, ok, not an intercept then. Designed that way as an afterthought. :-D -- Cheers / Saludos, Carlos E. R. (from openSUSE, Leap 15.1 x86_64 (ssd-test)) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. wrote:
nscd is not in the "official" loop, IMHO, so instead it puts itself in the middle intercepting some library calls and giving an answer without the library reaching its designed goal of reading the hosts file or querying a DNS.
It is the resolver library that asks nscd. It's perfectly "official". nscd listens on /var/run/nscd/socket. -- Per Jessen, Zürich (16.2°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2019-05-01 9:42 a.m., Per Jessen wrote:
Carlos E. R. wrote:
nscd is not in the "official" loop, IMHO, so instead it puts itself in the middle intercepting some library calls and giving an answer without the library reaching its designed goal of reading the hosts file or querying a DNS.
It is the resolver library that asks nscd. It's perfectly "official". nscd listens on /var/run/nscd/socket.
That makes, sense, sort of. It clears up HOW to communicate with nscd. But where do we get the reference to use /var/run/nscd/socket? I understand listening on localhost:53 Its a 'well known' address. I can even grok listening on *:53 when I'm in a generous mood. But something has to refer to /var/run/nscd/socket As opposed to some other socket In order to talk to nscd via that oscket they have to have found the address /var/run/nscd/socket somewhere. So WHERE? # find /var -type s -print Or try that from "/" if you want to be overwhelmed. I'm running dnsmasq not nscd but I find this interesting # find / -type s -print | grep dns /run/dovecot/dns-client /var/run/dovecot/dns-client Now dnsmasq listens on the well known port 53 (and yes that's configurable: # grep -C 2 "53" /etc/dnsmasq.conf # Listen on this specific port instead of the standard DNS port # (53). Setting this to zero completely disables DNS function, # leaving only DHCP and/or TFTP. #port=5353 ) How do we get there? Well the man pages for configuring dnsmasq say to set up /etc/resolve.conf to have ONLY ONE entry: nameserver 127.0.0.1 The man page says In order to configure dnsmasq to act as cache for the host on which it is running, put "name-server 127.0.0.1" in /etc/resolv.conf to force local processes to send queries to dnsmasq. And, as I mentioned earlier, dnsmasq can do its own caching. Now nscd is a cache server and not a resolver. I can see setting dnsmasq's own cache to zero and using nascd, but that seems a bit complicated. And if that were to be done, it still doesn't clear up a few HOW. I must admit, in a single user system a lightweight dns resolver makes sense. (MaraDNS might make even more sense since I'm not using DHCP) (https://www.linuxjournal.com/content/localhost-dns-cache) For email & web browsing, caching addresses makes sense - to a degree. but how much does the application do the caching? Realistically, my Thunderbird needs to cache imap.mail.antonaylward.com smtp.mail.antonaylward.com That's the theory. Actually its all one "mail.antonaylward.com" imap.gmail.com smtp.gmail.com Yes, Firefox is more demanding :-) BUT there is little point in caching 'user credentials' from /etc/passwd on a single user system. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
* Anton Aylward <opensuse@antonaylward.com> [05-01-19 11:26]:
On 2019-05-01 9:42 a.m., Per Jessen wrote:
Carlos E. R. wrote:
nscd is not in the "official" loop, IMHO, so instead it puts itself in the middle intercepting some library calls and giving an answer without the library reaching its designed goal of reading the hosts file or querying a DNS.
It is the resolver library that asks nscd. It's perfectly "official". nscd listens on /var/run/nscd/socket.
That makes, sense, sort of. It clears up HOW to communicate with nscd. But where do we get the reference to use /var/run/nscd/socket?
I understand listening on localhost:53 Its a 'well known' address. I can even grok listening on *:53 when I'm in a generous mood. But something has to refer to /var/run/nscd/socket As opposed to some other socket In order to talk to nscd via that oscket they have to have found the address /var/run/nscd/socket somewhere. So WHERE?
# find /var -type s -print
things change :) find /run -type s -print |grep nscd -- (paka)Patrick Shanahan Plainfield, Indiana, USA @ptilopteri http://en.opensuse.org openSUSE Community Member facebook/ptilopteri Registered Linux User #207535 @ http://linuxcounter.net Photos: http://wahoo.no-ip.org/piwigo paka @ IRCnet freenode -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Anton Aylward wrote:
On 2019-05-01 9:42 a.m., Per Jessen wrote:
Carlos E. R. wrote:
nscd is not in the "official" loop, IMHO, so instead it puts itself in the middle intercepting some library calls and giving an answer without the library reaching its designed goal of reading the hosts file or querying a DNS.
It is the resolver library that asks nscd. It's perfectly "official". nscd listens on /var/run/nscd/socket.
That makes, sense, sort of. It clears up HOW to communicate with nscd. But where do we get the reference to use /var/run/nscd/socket?
Well, we don't. That is the job of the resolver.
I understand listening on localhost:53 Its a 'well known' address. I can even grok listening on *:53 when I'm in a generous mood. But something has to refer to /var/run/nscd/socket As opposed to some other socket In order to talk to nscd via that oscket they have to have found the address /var/run/nscd/socket somewhere. So WHERE?
It's hardcoded in libc. (no doubt configurable at build time)
I'm running dnsmasq not nscd but I find this interesting
They don't preclude oneanother, they actually complement each other. nscd isn't necessary, but presumably does speed up things.
Now dnsmasq listens on the well known port 53 (and yes that's configurable: # grep -C 2 "53" /etc/dnsmasq.conf
# Listen on this specific port instead of the standard DNS port # (53). Setting this to zero completely disables DNS function, # leaving only DHCP and/or TFTP. #port=5353 )
How do we get there? Well the man pages for configuring dnsmasq say to set up /etc/resolve.conf to have ONLY ONE entry: nameserver 127.0.0.1
Yup. And that is exactly how you get there. You point the resolver to the nameserver you wish to use.
Now nscd is a cache server and not a resolver. I can see setting dnsmasq's own cache to zero and using nscd, but that seems a bit complicated. And if that were to be done, it still doesn't clear up a few HOW.
nscd caches hosts, i.e. A and AAAA records, for apps using e.g. getaddrinfo(), nothing else. Any app doing direct dns lookups will be cached by dnsmasq, according to DNS ttl values.
For email & web browsing, caching addresses makes sense - to a degree. but how much does the application do the caching?
Any sensible application will leave the majority of cacheing to the system. In your code you might cache for the duration of a loop or a session, but otherwise you just do the lookup again.
Realistically, my Thunderbird needs to cache imap.mail.antonaylward.com smtp.mail.antonaylward.com That's the theory. Actually its all one "mail.antonaylward.com" imap.gmail.com smtp.gmail.com
Your Thunderbird should be doing a lookup every time it needs one of those. If it caches, any round-robin'ing will be negated, for instance. -- Per Jessen, Zürich (18.7°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 01/05/2019 18.01, Per Jessen wrote:
Anton Aylward wrote:
On 2019-05-01 9:42 a.m., Per Jessen wrote:
Carlos E. R. wrote:
I understand listening on localhost:53 Its a 'well known' address. I can even grok listening on *:53 when I'm in a generous mood. But something has to refer to /var/run/nscd/socket As opposed to some other socket In order to talk to nscd via that oscket they have to have found the address /var/run/nscd/socket somewhere. So WHERE?
It's hardcoded in libc. (no doubt configurable at build time)
I'm running dnsmasq not nscd but I find this interesting
They don't preclude oneanother, they actually complement each other. nscd isn't necessary, but presumably does speed up things.
My main install in this machine has nscd and bind. My laptop has nscd and dnsmasq. I would not stop nscd, at most I would configure it to not cache internet names. But nothing bad happens if you don't, it is the default setting. The documentation of dnsmasq doesn't say to disable nscd, AFAIR. -- Cheers / Saludos, Carlos E. R. (from openSUSE, Leap 15.1 x86_64 (ssd-test)) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Wed, 1 May 2019 13:47:32 +0200 "Carlos E. R." <robin.listas@telefonica.net> wrote:
On 01/05/2019 13.22, Anton Aylward wrote:
Sorry Carlos, "Intercepted' is just too vague a concept. Somehow, somewhere there is a decision-in-software, an 'if" statement based on a config file entry, and not a beady-eyed raptor waiting to swoop down on an unsuspecting rodent.
nscd is not in the "official" loop, IMHO, so instead it puts itself in the middle intercepting some library calls and giving an answer without the library reaching its designed goal of reading the hosts file or querying a DNS.
I can not explain in detail because I do not know, I simply guess.
I just typed 'linux how does dns resolution work' into u-no-hoo and got this result: https://zwischenzugs.com/2018/06/08/anatomy-of-a-linux-dns-lookup-part-i/ I haven't read it all, so it might be a waste of time, but it starts off well .... -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Anton Aylward wrote:
On 2019-05-01 6:51 a.m., Carlos E. R. wrote:
The thing is, when an application makes a DNS query, what is the logical path of calls that it follows? There are just so many possibilities that come to mind "A guide for the perplexed" with a decision chart diagram, branching mindmap or something would be nice :-)
I guess the query is first answered or intercepted by nscd, then it goes to the configured DNS. Ah, wait, first it goes to the host file, which I think is cached by nscd.
Perhaps I didn't make myself clear. When an application such as Firefox make a library call "gethostbyname()" (or is that obsolete now?) what happens?
I may be wrong but I think the first thing that happens is that there is a look-see to /etc/nsswitch.conf to see what should be invoked. Mine says: hosts: files dns which, I take it, means check /etc/hosts first and then use the configired DNS resolver if the address isn't found in the hosts file.
The resolver library first attempts to contact nscd, then it proceeds with the above.
Two things occur to me at this point. The first is that dnsmasq slurps up /etc/hosts anyway so the 'files' entry is redundant.
By default dnsmasq is configured to do so, but nscd also reads nsswitch.ch.
The second is I don't see how things get to dnsmasq. Can someone help me here, please.
On my system (when I occasionally use dnsmasq), localhost:53. (by way of resolv.conf). -- Per Jessen, Zürich (16.6°C) http://www.cloudsuisse.com/ - your owncloud, hosted in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 4/30/19 1:35 PM, Brendan McKenna wrote:
Hi,
On Leap 15 (and earlier), it's nscd (Name Service Caching Daemon). Once you make a request, it tries to keep the queried information fresh in the cache, so it re-queries the information when it expires. The only way I've found to stop this is to shut the daemon down.
Brendan
On 30/04/2019 17:26, Carlos E. R. wrote:
On 30/04/2019 18.04, Dutch Ingraham wrote:
On 4/29/19 11:19 AM, Carlos E. R. wrote:
On 29/04/2019 18.14, Dutch Ingraham wrote:
Wild guess.
There is some DNS cache app running, which has an entry for "somesite" that has a short lifetime. Maybe that DNS cache app refreshes the IP before the timeout in order to have it ready for any application requesting it.
Thanks, Carlos. I'm sure you are correct, but identifying that cache app is the harder part. I took an uninformed chance that it may be systemd-resolved, which caches by default, and provides for turning the caching off, but that had no apparent effect on the issue.
BTW - I started wireshark about 4 hours later, and the same DNS queries were still being made. There must be some kind of timeout, but hard to say what it is set to until the offending daemon is properly identified.
It could be dnsmasq, nscd... The later is active by default.
Carlos and Brendan: dnsmasq wasn't active on my system, and it was in fact nscd. I disabled it and the dns queries disappeared, with no immediate noticeable negative effect. Thank you both! -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 30/04/2019 23.24, Dutch Ingraham wrote:
On 4/30/19 1:35 PM, Brendan McKenna wrote:
Hi,
On Leap 15 (and earlier), it's nscd (Name Service Caching Daemon). Once you make a request, it tries to keep the queried information fresh in the cache, so it re-queries the information when it expires. The only way I've found to stop this is to shut the daemon down.
Brendan
On 30/04/2019 17:26, Carlos E. R. wrote:
On 30/04/2019 18.04, Dutch Ingraham wrote:
On 4/29/19 11:19 AM, Carlos E. R. wrote:
On 29/04/2019 18.14, Dutch Ingraham wrote:
Wild guess.
There is some DNS cache app running, which has an entry for "somesite" that has a short lifetime. Maybe that DNS cache app refreshes the IP before the timeout in order to have it ready for any application requesting it.
Thanks, Carlos. I'm sure you are correct, but identifying that cache app is the harder part. I took an uninformed chance that it may be systemd-resolved, which caches by default, and provides for turning the caching off, but that had no apparent effect on the issue.
BTW - I started wireshark about 4 hours later, and the same DNS queries were still being made. There must be some kind of timeout, but hard to say what it is set to until the offending daemon is properly identified.
It could be dnsmasq, nscd... The later is active by default.
Carlos and Brendan: dnsmasq wasn't active on my system, and it was in fact nscd. I disabled it and the dns queries disappeared, with no immediate noticeable negative effect.
Close firefox. Open it and open certain page, and time the process. Close it. Open it again, same thing, time again. Start nscd, and repeat the above. Notice that nscd also caches the password and group accesses, services and netgroup, whatever those are. But machines are so powerful now that things are harder to notice. -- Cheers / Saludos, Carlos E. R. (from openSUSE, Leap 15.1 x86_64 (ssd-test)) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 4/29/2019 9:14 AM, Dutch Ingraham wrote:
Hi all:
I have a question about some DNS querys/responses that continue after the application that originally requested them is closed.
--- Someone else may have already answered this, but you do you have name resolution set to on in wireshark? It will generate dns queries and sometimes, depending on the query, your dns system may generate some looking for more glue pieces and more authoritative answers. It is another example of observing something interfering with what is observed. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 02/05/2019 11.48, L A Walsh wrote:
On 4/29/2019 9:14 AM, Dutch Ingraham wrote:
Hi all:
I have a question about some DNS querys/responses that continue after the application that originally requested them is closed.
--- Someone else may have already answered this, but you do you have name resolution set to on in wireshark? It will generate dns queries and sometimes, depending on the query, your dns system may generate some looking for more glue pieces and more authoritative answers.
It is another example of observing something interfering with what is observed.
and ntop. Interestingly, I don't find it on Leap 15.1, or the package has different name. -- Cheers / Saludos, Carlos E. R. (from openSUSE, Leap 15.1 x86_64 (ssd-test)) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 5/2/2019 3:04 AM, Carlos E. R. wrote:
On 02/05/2019 11.48, L A Walsh wrote:
name resolution ... in wireshark?
and ntop.
Interestingly, I don't find it on Leap 15.1, or the package has different name.
--- Don't find which one - ntop? Not in tumbleweed either. I last see it in 11.3. I remember it generating alot of output for interesting graphs, but of questionable usefulness given the diskspace... -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 03/05/2019 23.10, L A Walsh wrote:
On 5/2/2019 3:04 AM, Carlos E. R. wrote:
On 02/05/2019 11.48, L A Walsh wrote:
name resolution ... in wireshark?
and ntop.
Interestingly, I don't find it on Leap 15.1, or the package has different name.
--- Don't find which one - ntop? Not in tumbleweed either. I last see it in 11.3. I remember it generating alot of output for interesting graphs, but of questionable usefulness given the diskspace...
Oh, then I got the name wrong. Yes, I have used ntop in the past, but that was not the package I wanted. I'm on a test partition and I don't have easy access to all my notes. iftop... no. iptraf... yes, this one. It can convert ip addresses to names if asked. -- Cheers / Saludos, Carlos E. R. (from openSUSE, Leap 15.1 x86_64 (ssd-test)) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Fri, 03 May 2019 14:10:25 -0700 L A Walsh <suse@tlinx.org> wrote:
On 5/2/2019 3:04 AM, Carlos E. R. wrote:
On 02/05/2019 11.48, L A Walsh wrote:
name resolution ... in wireshark?
and ntop.
Interestingly, I don't find it on Leap 15.1, or the package has different name.
--- Don't find which one - ntop? Not in tumbleweed either.
https://software.opensuse.org/package/ntop It seems to be listed as an official package for tumbleweed? A more recent version is listed as experimental in server:monitoring for all releases. wireshark is also there as experimental in all and official in some.
I last see it in 11.3. I remember it generating alot of output for interesting graphs, but of questionable usefulness given the diskspace...
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
* Dave Howorth <dave@howorth.org.uk> [05-03-19 17:25]:
On Fri, 03 May 2019 14:10:25 -0700 L A Walsh <suse@tlinx.org> wrote:
On 5/2/2019 3:04 AM, Carlos E. R. wrote:
On 02/05/2019 11.48, L A Walsh wrote:
name resolution ... in wireshark?
and ntop.
Interestingly, I don't find it on Leap 15.1, or the package has different name.
--- Don't find which one - ntop? Not in tumbleweed either.
https://software.opensuse.org/package/ntop
It seems to be listed as an official package for tumbleweed?
A more recent version is listed as experimental in server:monitoring for all releases.
both seem to be excluded/missing -- (paka)Patrick Shanahan Plainfield, Indiana, USA @ptilopteri http://en.opensuse.org openSUSE Community Member facebook/ptilopteri Registered Linux User #207535 @ http://linuxcounter.net Photos: http://wahoo.no-ip.org/piwigo paka @ IRCnet freenode -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (8)
-
Anton Aylward -
Brendan McKenna -
Carlos E. R. -
Dave Howorth -
Dutch Ingraham -
L A Walsh -
Patrick Shanahan -
Per Jessen