[opensuse-programming] Back to basics question about opening devices on Linux
I feel funny even asking the following, but I have a situation that I need to resolve. 1. When opening a serial port on Linux, the very first read() will only return data that arrived from the point of the open. That is, any data in kernel buffers will not be seen. Right? Just to be sure, we do a flush after the open: tcflush(serialPort, TCIOFLUSH); So, is there any chance we see data from before the open/flush() call? 2. Similar question about opening a TCP/IP socket. Could we expect to see any data that arrived before the connect() call? Is there any need to flush the socket? The reason I ask is that I have two devices that provide time tagged data. One is a Trimble GPS receiver on a serial port. The other is a data collection device (time-synced with its own GPS receiver) on a TCP/IP socket. It is expected that the two, when opened, should be reporting rather similar times. Unfortunately, the two differ by up to 20 seconds. It seems that the trend is that the network device starts out 20 seconds in the future relative to the serial port device. My suspicion is that the data collection device is not syncing properly with it's own GPS receiver. Before I complain to the manufacturer, I want to be certain I have not missed something. Yours sincerely, Roger Oberholtzer Ramböll RST / Systems Office: Int +46 10-615 60 20 Mobile: Int +46 70-815 1696 roger.oberholtzer@ramboll.se ________________________________________ Ramböll Sverige AB Krukmakargatan 21 P.O. Box 17009 SE-104 62 Stockholm, Sweden www.rambollrst.se -- To unsubscribe, e-mail: opensuse-programming+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-programming+owner@opensuse.org
Roger Oberholtzer <roger@opq.se> wrote:
I feel funny even asking the following, but I have a situation that I need to resolve.
1. When opening a serial port on Linux, the very first read() will only return data that arrived from the point of the open. That is, any data in kernel buffers will not be seen. Right?
Just to be sure, we do a flush after the open:
tcflush(serialPort, TCIOFLUSH);
So, is there any chance we see data from before the open/flush() call?
2. Similar question about opening a TCP/IP socket. Could we expect to see any data that arrived before the connect() call? Is there any need to flush the socket?
The reason I ask is that I have two devices that provide time tagged data. One is a Trimble GPS receiver on a serial port. The other is a data collection device (time-synced with its own GPS receiver) on a TCP/IP socket. It is expected that the two, when opened, should be reporting rather similar times. Unfortunately, the two differ by up to 20 seconds. It seems that the trend is that the network device starts out 20 seconds in the future relative to the serial port device.
My suspicion is that the data collection device is not syncing properly with it's own GPS receiver. Before I complain to the manufacturer, I want to be certain I have not missed something.
Roger, For rs-232 there is no kernel level concept of a connection. I don't know what the receiver does if data comes in without a open file handle, but the sending side can only send it or queue it up. If you are using rts/cts hardware flow control the data could easily be sitting in the sending device until you open the tty and raise cts. You need to some how flush the output queue of the sender in that case. If you have a rs-232 monitor you should use it to see exactly what is going on. (I did tons of rs-232 work in the '90s, but nothing in the last 10 years.) For sockets, there are not any kernel level queues on either side until the connection is established. Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. -- To unsubscribe, e-mail: opensuse-programming+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-programming+owner@opensuse.org
On Thu, 2013-01-17 at 08:50 -0500, Greg Freemyer wrote:
Roger Oberholtzer <roger@opq.se> wrote:
I feel funny even asking the following, but I have a situation that I need to resolve.
1. When opening a serial port on Linux, the very first read() will only return data that arrived from the point of the open. That is, any data in kernel buffers will not be seen. Right?
Just to be sure, we do a flush after the open:
tcflush(serialPort, TCIOFLUSH);
So, is there any chance we see data from before the open/flush() call?
2. Similar question about opening a TCP/IP socket. Could we expect to see any data that arrived before the connect() call? Is there any need to flush the socket?
The reason I ask is that I have two devices that provide time tagged data. One is a Trimble GPS receiver on a serial port. The other is a data collection device (time-synced with its own GPS receiver) on a TCP/IP socket. It is expected that the two, when opened, should be reporting rather similar times. Unfortunately, the two differ by up to 20 seconds. It seems that the trend is that the network device starts out 20 seconds in the future relative to the serial port device.
My suspicion is that the data collection device is not syncing properly with it's own GPS receiver. Before I complain to the manufacturer, I want to be certain I have not missed something.
Roger,
For rs-232 there is no kernel level concept of a connection. I don't know what the receiver does if data comes in without a open file handle, but the sending side can only send it or queue it up.
If you are using rts/cts hardware flow control the data could easily be sitting in the sending device until you open the tty and raise cts.
Hard to guess what the GPS sender does in this respect. What I can say is that there are no gaps at the start of the data stream. If the sender's FIFO fills up and I get that data initially, I would expect it to have some older time stamp from when the data went in the FIFO. Then, while the FIFO was full (presumably waiting for my connection), data would have to be discarded, causing a gap.
You need to some how flush the output queue of the sender in that case.
No such control is allowed. The GPS simply sends NMEA data strings on port. You do not turn it on or off.
If you have a rs-232 monitor you should use it to see exactly what is going on. (I did tons of rs-232 work in the '90s, but nothing in the last 10 years.)
We are going to see if the GPS continues to send data when there is no reader. Perhaps the flow control lines are not used. I suspect that this is the case.
For sockets, there are not any kernel level queues on either side until the connection is established.
As I suspected. Yours sincerely, Roger Oberholtzer Ramböll RST / Systems Office: Int +46 10-615 60 20 Mobile: Int +46 70-815 1696 roger.oberholtzer@ramboll.se ________________________________________ Ramböll Sverige AB Krukmakargatan 21 P.O. Box 17009 SE-104 62 Stockholm, Sweden www.rambollrst.se -- To unsubscribe, e-mail: opensuse-programming+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-programming+owner@opensuse.org
Roger Oberholtzer wrote:
We are going to see if the GPS continues to send data when there is no reader. Perhaps the flow control lines are not used. I suspect that this is the case.
This is often the case with very simple serial devices - you have to get the baudrate right (ie. no auto-negotiate), and flow-control (soft or hard) is not implemented. -- Per Jessen, Zürich (-4.5°C) http://www.dns24.ch/ - free DNS hosting, made in Switzerland. -- To unsubscribe, e-mail: opensuse-programming+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-programming+owner@opensuse.org
On Thu, 2013-01-17 at 08:50 -0500, Greg Freemyer wrote:
Roger Oberholtzer <roger@opq.se> wrote:
I feel funny even asking the following, but I have a situation that I need to resolve. So, is there any chance we see data from before the open/flush() call?
Yes, there is a chance.
For rs-232 there is no kernel level concept of a connection. I don't know what the receiver does if data comes in without a open file handle, but the sending side can only send it or queue it up. If you are using rts/cts hardware flow control the data could easily be sitting in the sending device until you open the tty and raise cts.
And here is a key part - it depends on the hardware. You might have a traditional UART, a Cyclades board, a Digi board... so it is always best to assume with a serial connection that you might get some stray stuff (possibly even noise).
For sockets, there are not any kernel level queues on either side until the connection is established.
Sockets aren't hardware, so they are much more consistent. You should never see stray data. -- To unsubscribe, e-mail: opensuse-programming+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-programming+owner@opensuse.org
On Fri, 2013-01-18 at 06:52 -0500, Adam Tauno Williams wrote:
On Thu, 2013-01-17 at 08:50 -0500, Greg Freemyer wrote:
Roger Oberholtzer <roger@opq.se> wrote:
I feel funny even asking the following, but I have a situation that I need to resolve. So, is there any chance we see data from before the open/flush() call?
Yes, there is a chance.
For rs-232 there is no kernel level concept of a connection. I don't know what the receiver does if data comes in without a open file handle, but the sending side can only send it or queue it up. If you are using rts/cts hardware flow control the data could easily be sitting in the sending device until you open the tty and raise cts.
And here is a key part - it depends on the hardware. You might have a traditional UART, a Cyclades board, a Digi board... so it is always best to assume with a serial connection that you might get some stray stuff (possibly even noise).
We have verified that the sender GPS sends data on the serial port even when we are not reading it. Which is good. This means that any read() we get should be rather recent. Combined with the flush and looking for the second occurrence of a record, I think we are truly getting the most recent data from this device. The network device (that had the suspicious time - at least as I see things) is most likely the source of the problem. Are you all sitting down? The supplier said that they do not always put the GPS in the unit. Sometimes it is supplied in a separate box and the customer needs to install it. The guys dealing with these systems (across the country) have verified that some systems may be this way. So, on Monday we start a new hunt. But of course, when something looks odd, please do blame the software first... Yours sincerely, Roger Oberholtzer Ramböll RST / Systems Office: Int +46 10-615 60 20 Mobile: Int +46 70-815 1696 roger.oberholtzer@ramboll.se ________________________________________ Ramböll Sverige AB Krukmakargatan 21 P.O. Box 17009 SE-104 62 Stockholm, Sweden www.rambollrst.se -- To unsubscribe, e-mail: opensuse-programming+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-programming+owner@opensuse.org
Am Freitag, 18. Januar 2013, 16:31:03 schrieb Roger Oberholtzer:
We have verified that the sender GPS sends data on the serial port even when we are not reading it. Which is good. This means that any read() we get should be rather recent. Combined with the flush and looking for the second occurrence of a record, I think we are truly getting the most recent data from this device.
The network device (that had the suspicious time - at least as I see things) is most likely the source of the problem. Are you all sitting down? The supplier said that they do not always put the GPS in the unit. Sometimes it is supplied in a separate box and the customer needs to install it. The guys dealing with these systems (across the country) have verified that some systems may be this way.
So, on Monday we start a new hunt. But of course, when something looks odd, please do blame the software first...
In order to finally nail down the supplier, you may take another (ntp) time source into account, which has rather elaborate instruments for qualification ("ntpdc -c peers"). Cheers, Pete -- To unsubscribe, e-mail: opensuse-programming+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-programming+owner@opensuse.org
On Tue, 2013-01-22 at 11:41 +0100, Hans-Peter Jansen wrote:
Am Freitag, 18. Januar 2013, 16:31:03 schrieb Roger Oberholtzer:
We have verified that the sender GPS sends data on the serial port even when we are not reading it. Which is good. This means that any read() we get should be rather recent. Combined with the flush and looking for the second occurrence of a record, I think we are truly getting the most recent data from this device.
The network device (that had the suspicious time - at least as I see things) is most likely the source of the problem. Are you all sitting down? The supplier said that they do not always put the GPS in the unit. Sometimes it is supplied in a separate box and the customer needs to install it. The guys dealing with these systems (across the country) have verified that some systems may be this way.
So, on Monday we start a new hunt. But of course, when something looks odd, please do blame the software first...
In order to finally nail down the supplier, you may take another (ntp) time source into account, which has rather elaborate instruments for qualification ("ntpdc -c peers").
Ahh the luxury of that... This is in a vehicle out on the road without access to the internet (i.e., an NTP server). We are looking at perhaps using a dedicated NTP server box (gps-based) for this. But we cannot find one that meets our needs and is reasonable in cost. Yours sincerely, Roger Oberholtzer Ramböll RST / Systems Office: Int +46 10-615 60 20 Mobile: Int +46 70-815 1696 roger.oberholtzer@ramboll.se ________________________________________ Ramböll Sverige AB Krukmakargatan 21 P.O. Box 17009 SE-104 62 Stockholm, Sweden www.rambollrst.se -- To unsubscribe, e-mail: opensuse-programming+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-programming+owner@opensuse.org
Am Dienstag, 22. Januar 2013, 13:23:28 schrieb Roger Oberholtzer:
On Tue, 2013-01-22 at 11:41 +0100, Hans-Peter Jansen wrote:
In order to finally nail down the supplier, you may take another (ntp) time source into account, which has rather elaborate instruments for qualification ("ntpdc -c peers").
Ahh the luxury of that... This is in a vehicle out on the road without access to the internet (i.e., an NTP server).
We are looking at perhaps using a dedicated NTP server box (gps-based) for this. But we cannot find one that meets our needs and is reasonable in cost.
I'm very happy with dcf77 based devices, using them since ages. The signal might even reach Schweden. Do you know of any working dcf77 devices in your environment (better known as "Funkuhren" in Germany at least)? There are simple low cost serial devices available, which are attachable with current RS232 usb adapters, since serial ports are a endangered species (at least as long as my related fix for ntp is still present in current openSUSE ntp package builds, didn't check..). I'm still using about a dozen of the serial RPC DCF77 Funkuhr from SURE GmbH: http://www.ebay.de/itm/serielle-seriell-Funkuhr-Atomzeit-RS232-NEU- TOP-/330861361482?pt=Klassische_Computer&hash=item4d08dfb94a Cheers, Pete -- To unsubscribe, e-mail: opensuse-programming+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-programming+owner@opensuse.org
On Wed, 2013-01-23 at 00:11 +0100, Hans-Peter Jansen wrote:
Am Dienstag, 22. Januar 2013, 13:23:28 schrieb Roger Oberholtzer:
On Tue, 2013-01-22 at 11:41 +0100, Hans-Peter Jansen wrote:
In order to finally nail down the supplier, you may take another (ntp) time source into account, which has rather elaborate instruments for qualification ("ntpdc -c peers").
Ahh the luxury of that... This is in a vehicle out on the road without access to the internet (i.e., an NTP server).
We are looking at perhaps using a dedicated NTP server box (gps-based) for this. But we cannot find one that meets our needs and is reasonable in cost.
I'm very happy with dcf77 based devices, using them since ages. The signal might even reach Schweden. Do you know of any working dcf77 devices in your environment (better known as "Funkuhren" in Germany at least)?
The signals do indeed get to Sweden. The thing is that we already have very high-end (Trimble and OxTS) GPS devices in our system. We just do not do a system-level sync with them at this time. ntpd is targeted to be running in a data center or other stable environment. It is not so well adapted to being started and stopped frequently, and to having the GPS time source be unavailable when it is started. Over time it sorts these things out. But when it is started/stopped a couple times a day the prognosis is not as good. There is a discussion of adding chrony to openSUSE, which is more suited to working in a hostile environment. This thread originated because of a possible issue we have with our current method, which is developed in-house. We want to stop supporting it in favor of some standard method. Yours sincerely, Roger Oberholtzer Ramböll RST / Systems Office: Int +46 10-615 60 20 Mobile: Int +46 70-815 1696 roger.oberholtzer@ramboll.se ________________________________________ Ramböll Sverige AB Krukmakargatan 21 P.O. Box 17009 SE-104 62 Stockholm, Sweden www.rambollrst.se -- To unsubscribe, e-mail: opensuse-programming+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-programming+owner@opensuse.org
participants (5)
-
Adam Tauno Williams -
Greg Freemyer -
Hans-Peter Jansen -
Per Jessen -
Roger Oberholtzer