Mailinglist Archive: opensuse (4570 mails)

< Previous Next >
Re: [SLE] Developing a Real Time Data System
  • From: "Carlos E. R." <robin1.listas@xxxxxxxxxx>
  • Date: Fri, 25 Nov 2005 12:10:24 +0000 (UTC)
  • Message-id: <Pine.LNX.4.61.0511250038230.19749@xxxxxxxxxxxxxxxx>
Hash: SHA1

The Thursday 2005-11-24 at 13:18 -0000, JUAN ERNESTO FLORES BELTRAN wrote:

> > You still haven't said what sampling speed you need, but unless it is
> > slow, you will need to save somewhere else first (as fast as posible), and
> > then move to the database as a post process.
> Sampling speed will be as slow as possible because of this is a project for
> the university...not a product for anyone so speed is subject to the designer
> programming skills and available resources: adquisition data card
> specifications, programming language, etc...

It is rather the other way round: you know what speed you require, then
define what hardware/software requirements you need to meet it.

Supposing you are to measure some sort of experimental data, like engine
speed/accel: then somewhere about ten samples per second is enough. If you
are measuring oven temperatures, then one per second or per minute. If
atmospheric weather, one per quarter/hour, say. If you want to measure an
audio amplifier response, I'd be happy with a million samples per second.
It depends on what you need!

You will find that sampling and storing data for a dozen channels at
10..100 K-s/s, non stop, quite taxing. That is withing the capabilities
of a standard daq card.

And related to that is the precision needed... 12bit is typical in
industry, 16 bits is more expensive, 8 cheaper (and for home use, really).
12 bits is roughly equivalent to what a 3 1/2 digit multimeter gives.

> just to get an idea...what limit is considered "too fast" to save somewhere
> else before saving into the database??

¡Test it!

It will not be real time, in any case. Real time means that the response
time is guaranteed to be below certain value, by the way, that's the most
important characteristic. You can not use a database like that and give
that guarantee.

You can, however, simulate a test run, send data to a database at a fixed
continuous rate for some time, and measure unexpected delays. You will
have to define how exactly are you going to save the data, what format: one
sample per register or a thousand, or the whole experiment in one

> and what technical reasons would force
> me to do this in case sampling speed increases where neccesary?? is this
> subject to any mysql or postgresql managing/storaging data limitations??

I'm no expert in databases! X-)

But they are certainly not designed for a continuous data flow storage,
non delay, non stop.

> Any link, tutorial or manual to get inmersed into real time data handling??
> would be helpfull!!

I don't have any... I had to learn it on my own.

What I did, roughly said, was to program the ISA card (that's what we
used) to measured the channels and call an IRQ, at a fixed rate. An IRQ
handler routine would save the data (a word) in memory or save to a file,
raw. This could be an independent resident program (that was in Dos).
Another routine, in the foreground, would read the data, filter it,
process, display, perhaps even react to it, whatever.

The card manufacturer supplied several code samples, from simple polling
routines to irq or dma read. Later they supplied a driver that did all the
dirty work, I only needed to call up the setup functions. I usually worked
with Advantech cards, rather common here and not expensive (except for a
hobbyist), meaning about 10 times less than National's. There are many
more manufacturers, you will have to search around, depending on your
requirements and purse. National people were usually helpful, they could
give you sound advice using their software and hardware depending on your
needs: not cheap, of course. Don't forget when looking around what kind of
software and sourcecode samples they include, it makes a very big
difference. And that that code was tested in Linux! ;-)

I'm not sure how I'd do it in Linux, but I'd like to have a device driver
that would copy data to a file or stream at a fixed rate. Probably it
exists already.

- --
Carlos Robinson
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Made with pgp4pine 1.76

< Previous Next >