[opensuse-arm] Raspberry Pi 3 interrupt handling
We have written a minimal kernel interrupt handler for the Raspberry Pi 3. It is running the current 64-bit Tumbleweed kernel. I am curious about the rate of interrupts that we might be able to capture. The ISR does little other than run when a raising edge interrupt happens. We are looking at /proc/interrupts to see how many interrupts have happened. We see that at around 23 kHz we begin to loose interrupts. The system is not doing anything else. Does that seem reasonable? I have not seen any good discussion of this. I think it is rather low. I am guessing that the issue is how the Linux kernel responds to interrupts. The housework in setting things up so that the interrupt can run must be the resource hog. Opinions? Suggestions? -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
Hi Roger, Am 30.11.2017 um 10:59 schrieb Roger Oberholtzer:
We have written a minimal kernel interrupt handler for the Raspberry Pi 3. It is running the current 64-bit Tumbleweed kernel. I am curious about the rate of interrupts that we might be able to capture.
The ISR does little other than run when a raising edge interrupt happens. We are looking at /proc/interrupts to see how many interrupts have happened.
We see that at around 23 kHz we begin to loose interrupts. The system is not doing anything else.
Does that seem reasonable? I have not seen any good discussion of this. I think it is rather low. I am guessing that the issue is how the Linux kernel responds to interrupts. The housework in setting things up so that the interrupt can run must be the resource hog.
Opinions? Suggestions?
I don't see anything openSUSE-specific in here, so I'd suggest to ask on upstream linux-rpi-kernel list about any performance expectations. Personally I haven't had any success using GPIO interrupts on my rpi3, but that may be a matter of the device/driver (SX1276) I tried it with. Can you share any more details on DT overlay or driver init you're using to set up the interrupt? What's your interrupt source? Regards, Andreas https://github.com/afaerber/lora-modules/blob/9c5fcb64d7dac953c29da527d0f929... -- SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
Am 12.12.2017 um 19:32 schrieb Andreas Färber:
Am 30.11.2017 um 10:59 schrieb Roger Oberholtzer:
We have written a minimal kernel interrupt handler for the Raspberry Pi 3. It is running the current 64-bit Tumbleweed kernel. I am curious about the rate of interrupts that we might be able to capture. [...] Personally I haven't had any success using GPIO interrupts on my rpi3, but that may be a matter of the device/driver (SX1276) I tried it with.
FTR it was indeed a custom driver issue for me, I succeeded last night. I get only a single TX Done interrupt though, so still can't comment on maximum achievable rates. Cheers, Andreas -- SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
On Tue, Dec 12, 2017 at 7:32 PM, Andreas Färber <afaerber@suse.de> wrote:
Hi Roger,
Am 30.11.2017 um 10:59 schrieb Roger Oberholtzer:
We have written a minimal kernel interrupt handler for the Raspberry Pi 3. It is running the current 64-bit Tumbleweed kernel. I am curious about the rate of interrupts that we might be able to capture.
The ISR does little other than run when a raising edge interrupt happens. We are looking at /proc/interrupts to see how many interrupts have happened.
We see that at around 23 kHz we begin to loose interrupts. The system is not doing anything else.
Does that seem reasonable? I have not seen any good discussion of this. I think it is rather low. I am guessing that the issue is how the Linux kernel responds to interrupts. The housework in setting things up so that the interrupt can run must be the resource hog.
Opinions? Suggestions?
I don't see anything openSUSE-specific in here, so I'd suggest to ask on upstream linux-rpi-kernel list about any performance expectations.
Personally I haven't had any success using GPIO interrupts on my rpi3, but that may be a matter of the device/driver (SX1276) I tried it with.
Can you share any more details on DT overlay or driver init you're using to set up the interrupt? What's your interrupt source?
GPIO pin 17. In our driver, we get interrupts on this line at up to 23 kHz. Then we start to miss some. We look at /proc/interrupts to see how many have occurred. The driver is absolute minimum while we test the interrupt speed we might obtain: #include <linux/kernel.h> #include <linux/module.h> #include <linux/interrupt.h> #include <linux/gpio.h> #define ANTS_NUM_GPIOS 1 #define ANTS_GPIO 475 static struct gpio ants_gpio = {ANTS_GPIO, GPIOF_IN, "RST ANTS"}; static struct gpio_desc *ants_gpiod; static int ants_irq = -1; static irqreturn_t ants_handle_irq(int irq, void *not_used) { return(IRQ_HANDLED); } static int __init ants_init(void) { int ret = 0; ret = gpio_request_one(ants_gpio.gpio, ants_gpio.flags, ants_gpio.label); if (ret) { printk(KERN_ERR "ANTS - GPIO request failed: %d\n", ret); goto no_gpio; } ants_gpiod = gpio_to_desc(ants_gpio.gpio); #if 0 /* Documentation/gpio/consumer.txt seems to say that this has to be done * before the descriptor can be used, but I am sceptical: */ gpio_free(ants_gpio.gpio); #endif ret = gpiod_to_irq(ants_gpiod); if (ret < 0) { printk(KERN_ERR "ANTS - No IRQ available for GPIO #%d: %d\n", ants_gpio.gpio, ret); goto no_irq; } ants_irq = ret; ret = request_irq(ants_irq, ants_handle_irq, (IRQF_TRIGGER_RISING), "ants", NULL); if (ret) { printk(KERN_ERR "ANTS - IRQ (#%d) request failed: %d\n", ants_irq, ret); goto no_irq; } printk(KERN_INFO "ANTS - Using IRQ: %d\n", ants_irq); return(0); no_irq: gpio_free(ants_gpio.gpio); no_gpio: return(ret); } static void __exit ants_exit(void) { free_irq(ants_irq, NULL); gpio_free(ants_gpio.gpio); } module_init(ants_init); module_exit(ants_exit); MODULE_AUTHOR("Ramboll RST"); MODULE_DESCRIPTION("RST distance module"); MODULE_LICENSE("GPL"); -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
On 13.12.17 13:12, Roger Oberholtzer wrote:
On Tue, Dec 12, 2017 at 7:32 PM, Andreas Färber <afaerber@suse.de> wrote:
Hi Roger,
Am 30.11.2017 um 10:59 schrieb Roger Oberholtzer:
We have written a minimal kernel interrupt handler for the Raspberry Pi 3. It is running the current 64-bit Tumbleweed kernel. I am curious about the rate of interrupts that we might be able to capture.
The ISR does little other than run when a raising edge interrupt happens. We are looking at /proc/interrupts to see how many interrupts have happened.
We see that at around 23 kHz we begin to loose interrupts. The system is not doing anything else.
Does that seem reasonable? I have not seen any good discussion of this. I think it is rather low. I am guessing that the issue is how the Linux kernel responds to interrupts. The housework in setting things up so that the interrupt can run must be the resource hog.
Opinions? Suggestions?
I don't see anything openSUSE-specific in here, so I'd suggest to ask on upstream linux-rpi-kernel list about any performance expectations.
Personally I haven't had any success using GPIO interrupts on my rpi3, but that may be a matter of the device/driver (SX1276) I tried it with.
Can you share any more details on DT overlay or driver init you're using to set up the interrupt? What's your interrupt source?
GPIO pin 17. In our driver, we get interrupts on this line at up to 23 kHz. Then we start to miss some. We look at /proc/interrupts to see how many have occurred.
The driver is absolute minimum while we test the interrupt speed we might obtain:
Would it make sense to resort to polling at those speeds instead? Maybe poll using a constant reoccuring hrtimer? I also remember some talk about FIQ interrupts being faster than normal ones, but I never used them. Alex -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
On Wed, Dec 13, 2017 at 1:35 PM, Alexander Graf <agraf@suse.de> wrote:
Would it make sense to resort to polling at those speeds instead? Maybe poll using a constant reoccuring hrtimer?
Granted polling does not have the interrupt overhead. But I wonder how it keeps up with respect to scheduling. Might one miss activity when the kernel thread is not running?
I also remember some talk about FIQ interrupts being faster than normal ones, but I never used them.
Same here. I would really like to try them! Here is some good information: https://free-electrons.com/blog/fiq-handlers-in-the-arm-linux-kernel/ https://stackoverflow.com/questions/973933/what-is-the-difference-between-fi... -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
On 13.12.17 14:00, Roger Oberholtzer wrote:
On Wed, Dec 13, 2017 at 1:35 PM, Alexander Graf <agraf@suse.de> wrote:
Would it make sense to resort to polling at those speeds instead? Maybe poll using a constant reoccuring hrtimer?
Granted polling does not have the interrupt overhead. But I wonder how it keeps up with respect to scheduling. Might one miss activity when the kernel thread is not running?
Depends on how bad your jitter is, but I would hope that you'll get better granularity than 23khz ;). Worst case you can always try and isolate one CPU to do nothing but poll. You have 4 of them after all :). But that gets quite complicated ... Alex -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
On Wed, Dec 13, 2017 at 2:28 PM, Alexander Graf <agraf@suse.de> wrote:
On 13.12.17 14:00, Roger Oberholtzer wrote:
On Wed, Dec 13, 2017 at 1:35 PM, Alexander Graf <agraf@suse.de> wrote:
Would it make sense to resort to polling at those speeds instead? Maybe poll using a constant reoccuring hrtimer?
Granted polling does not have the interrupt overhead. But I wonder how it keeps up with respect to scheduling. Might one miss activity when the kernel thread is not running?
Depends on how bad your jitter is, but I would hope that you'll get better granularity than 23khz ;).
If jitter is too big, then perhaps if more than one interrupt is pending when already in an interrupt, only the first one is seen. I do not think interrupts are stacked.
Worst case you can always try and isolate one CPU to do nothing but poll. You have 4 of them after all :). But that gets quite complicated ...
I have also been considering that. But when we are testing, there is pretty much nothing else going on. One would expect a CPU to be ready. But if the kernel thread managing the interrupt must be set up to run, maybe that is taking time. This is getting outside my area of knowledge. I'm not sure how to do this for a kernel interrupt handler. I've only seen it for user processes. Time to Google. -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
On 12/13/2017 03:42 PM, Roger Oberholtzer wrote:
On Wed, Dec 13, 2017 at 2:28 PM, Alexander Graf <agraf@suse.de> wrote:
On 13.12.17 14:00, Roger Oberholtzer wrote:
On Wed, Dec 13, 2017 at 1:35 PM, Alexander Graf <agraf@suse.de> wrote:
Would it make sense to resort to polling at those speeds instead? Maybe poll using a constant reoccuring hrtimer?
Granted polling does not have the interrupt overhead. But I wonder how it keeps up with respect to scheduling. Might one miss activity when the kernel thread is not running?
Depends on how bad your jitter is, but I would hope that you'll get better granularity than 23khz ;).
If jitter is too big, then perhaps if more than one interrupt is pending when already in an interrupt, only the first one is seen. I do not think interrupts are stacked.
Worst case you can always try and isolate one CPU to do nothing but poll. You have 4 of them after all :). But that gets quite complicated ...
I have also been considering that. But when we are testing, there is pretty much nothing else going on. One would expect a CPU to be ready. But if the kernel thread managing the interrupt must be set up to run, maybe that is taking time. This is getting outside my area of knowledge.
From my understanding in the minimal driver you posted, you don't use a threaded interrupt handler, but a handler in IRQ context. So there won't be any need for
the kernel to schedule the ISR. Or do I miss something? My guess is, that you are hitting the maximum frequency of possible interrupts. So I would prioritize the FIQ approach to see if that helps. Regards, Matthias -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
Am 13.12.2017 um 15:42 schrieb Roger Oberholtzer:
On Wed, Dec 13, 2017 at 2:28 PM, Alexander Graf <agraf@suse.de> wrote:
On 13.12.17 14:00, Roger Oberholtzer wrote:
On Wed, Dec 13, 2017 at 1:35 PM, Alexander Graf <agraf@suse.de> wrote:
Would it make sense to resort to polling at those speeds instead? Maybe poll using a constant reoccuring hrtimer?
Granted polling does not have the interrupt overhead. But I wonder how it keeps up with respect to scheduling. Might one miss activity when the kernel thread is not running?
Depends on how bad your jitter is, but I would hope that you'll get better granularity than 23khz ;).
If jitter is too big, then perhaps if more than one interrupt is pending when already in an interrupt, only the first one is seen. I do not think interrupts are stacked.
I might be wrong, but isn't that the point of "chained" interrupts? In my example code you can also see me playing with "threaded" interrupts. This is the only official documentation I could find: https://www.kernel.org/doc/html/latest/core-api/genericirq.html Cheers, Andreas -- SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
participants (4)
-
Alexander Graf -
Andreas Färber -
Matthias Brugger -
Roger Oberholtzer