В Wed, 05 Jun 2013 19:20:09 +0200 Jean Delvare пишет:

...

...

* No idea what the scsi_dh* modules are about either, no modalias and no dependencies either. Hannes?

they are related to device-mapper-multipath and not needed on normal end user systems. I wonder why they are loaded; do you have multipath enabled?

It's odd then because no, none of my systems use multipath, they are all dumb laptop/desktop hardware with basic SATA HDD or SSD in them.

They are loaded by mkinitrd ((boot|setup)-scsi_dh.sh) with reference to bnc#727428; I do not think we should build them in kernel; rather we should fix unconditional loading of them :) -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

On 6/5/13 12:41 PM, Andrey Borzenkov wrote:

В Wed, 05 Jun 2013 18:29:53 +0200 Jean Delvare пишет:

...

* I have no idea why/how sg is being loaded, it has no modalias and no other module depend on it. But /dev/sd* nodes are created so it is certainly useful. Hannes?

/usr/lib/udev/rules.d/80-drivers.rules:SUBSYSTEM=="scsi", ENV{DEVTYPE}=="scsi_device", TEST!="[module/sg]", IMPORT{builtin}="kmod load sg"

I do not think it is needed much; the practical problem is to make sure it is loaded *when* it is needed (like remote SCSI enclosure monitoring via SES or jukebox control).

This is the SCSI generic interface. You've identified the practical problem perfectly - there's no way to tell when a user will need it. It's not just for jukebox and SES. Optical media writers use it as well. -Jeff -- Jeff Mahoney SUSE Labs

Hi Andrey, Jeff, Le Wednesday 05 June 2013 à 21:33 +0400, Andrey Borzenkov a écrit :

В Wed, 05 Jun 2013 13:21:09 -0400 Jeff Mahoney пишет:

...

On 6/5/13 12:41 PM, Andrey Borzenkov wrote:

...

В Wed, 05 Jun 2013 18:29:53 +0200 Jean Delvare пишет:

...

* I have no idea why/how sg is being loaded, it has no modalias and no other module depend on it. But /dev/sd* nodes are created so it is certainly useful. Hannes?

/usr/lib/udev/rules.d/80-drivers.rules:SUBSYSTEM=="scsi", ENV{DEVTYPE}=="scsi_device", TEST!="[module/sg]", IMPORT{builtin}="kmod load sg"

Thanks for the pointer.

...

...

I do not think it is needed much; the practical problem is to make sure it is loaded *when* it is needed (like remote SCSI enclosure monitoring via SES or jukebox control).

This is the SCSI generic interface. You've identified the practical problem perfectly - there's no way to tell when a user will need it. It's not just for jukebox and SES. Optical media writers use it as well.

Are you sure? I thought sr was extended with necessary functionality ages ago. At least cdrecord dev=/dev/sr0 worked for me for years.

The sg driver has been creating a sysfs symlink from each SCSI device to its sg interface for at least 8 years. So applications do no longer need the user to point them to that sg interface. Whether optical media writers or other applications actually need the sg interface today or not, I can't say. I see that the wodim binary still has references to /dev/sg%d and the sg driver, but I don't know if it is for current or older kernels. Anyway, if there is no way to tell when sg is needed, and we end up loading that module almost unconditionally, according to our policy we should build it into the kernel, right? Jeff, Hannes, any objection? -- Jean Delvare Suse L3 -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

Am 05.06.2013 18:41, schrieb Andrey Borzenkov:

I agree for x86.

me, too, mostly (see below :-)

...

* With DRM/KMS being the norm now, I believe drm and drm_kms_helper could be built-in, even though the former is quite large. drm selects i2c-algo-bit, most (all?) DRM/KMS drivers use it. Even servers have graphics chips.

Will it prevent proprietary drivers from working? There are quite a number of users with nVidia/ATI third party drivers. As much as I would like to not use them, nouveau still has fat too much usability issues for me.

And on servers it is very useful to be able to blacklist drm, so that KMS does not kick in. VGA-over-serial emulation and other textmode consoles do not work in graphics mode. maybe there is some magic parameter to stop drm from kicking in, but "vga=0" no longer helps.

...

* I suppose fuse is only used on desktop machines by default, for gvfs, so having it as a module seems OK.

You would not believe how many servers need to mount NTFS file systems :-) ntfs-3g is implemented with FUSE. Best regards, seife -- Stefan Seyfried "If your lighter runs out of fluid or flint and stops making fire, and you can't be bothered to figure out about lighter fluid or flint, that is not Zippo's fault." -- bkw -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

Am 06.06.2013 10:13, schrieb Jean Delvare:

Hi Seife,

...

And on servers it is very useful to be able to blacklist drm, so that KMS does not kick in. VGA-over-serial emulation and other textmode consoles do not work in graphics mode.

maybe there is some magic parameter to stop drm from kicking in, but "vga=0" no longer helps.

Well, how do you "blacklist drm" exactly? My understanding is that drm is only a common helper module for all drm drivers (radeon, i915 etc.) so a "blacklist drm" statement should have no effect, should it?

"find /lib/modules -name drm.ko | xargs --no-run-if-empty rm" :-) This effectively fixes all attempts to load any kms module, because they all need drm. Oh -- and drm is *big*, so that's another reason to have it as a module. Loading a module from disk once the kernel is running is probably much faster than loading the bigger kernel from disk / network via the BIOS routines the boot loader is using.

Also KMS can be disabled with nomodeset=1 or whatever the exact boot command line syntax is these days.

If vga=0 no longer works as it used to, maybe this should be reported as a bug.

Well, it is working as before: the VGA driver does use 80x25, but in the good old times of vesafb it also did tell vesafb to not do anything, but this has changed now. If this is a bug -- I don't have the energy to discuss. All I know is I don't want kms on any server ever. Because this will force me into using Java or .NET consoles instead of using plain SSH to the management board.

...

...

...

* I suppose fuse is only used on desktop machines by default, for gvfs, so having it as a module seems OK.

You would not believe how many servers need to mount NTFS file systems :-) ntfs-3g is implemented with FUSE.

Well, if there is a consensus that almost all machines need fuse for one thing or another, I really have no problem having it built into the kernel, if there is no reason not to (KMP, blacklisting or anything else.) Miklos, can you think of any reason why having fuse built into the kernel would be a problem? -- Stefan Seyfried "If your lighter runs out of fluid or flint and stops making fire, and you can't be bothered to figure out about lighter fluid or flint, that is not Zippo's fault." -- bkw -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

...

...

...

* I suppose fuse is only used on desktop machines by default, for gvfs, so having it as a module seems OK.

You would not believe how many servers need to mount NTFS file systems :-) ntfs-3g is implemented with FUSE.

Well, if there is a consensus that almost all machines need fuse for one thing or another, I really have no problem having it built into the kernel, if there is no reason not to (KMP, blacklisting or anything else.) Miklos, can you think of any reason why having fuse built into the kernel would be a problem?

The two options to prevent possible security issues with unprivileged mounts are: 1) remove suid-root from /usr/bin/fusermount or remove /usr/bin/fusermount altogether 2) blacklist fuse module So I think the only issue with building fuse into the kernel is backward compatibility for case 2. Thanks, Miklos -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

On 2013-06-05 18:41 (GMT+0200) Takashi Iwai composed:

The only cases where we don't need KMS are with Nvidia and AMD binary drivers.

When did KMS support for mga begin? -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

On 2013-06-05 13:05 (GMT-0400) Jeff Mahoney composed:

On 6/5/13 12:58 PM, Felix Miata wrote:

...

On 2013-06-05 18:41 (GMT+0200) Takashi Iwai composed:

...

...

The only cases where we don't need KMS are with Nvidia and AMD binary drivers.

...

When did KMS support for mga begin?

3.9, I think.

NAICT, in 3.10.0-rc2 it's apparently still not ready to use. I tried a bunch of combinations of cmdline vga= &/or video= &/or nomodeset with or without various permutations of xorg.conf.d/ or xorg.conf and never could get the mga driver used in X or video= cmdline options applied on the vttys. Xorg.0.logs of attempts (plus one of same system booted to 12.1 running 1600x1200 and virtual 1920x1200): http://fm.no-ip.com/Tmp/Linux/Xorg/Mga/ Any suggestions for more to try, or just file a bug? If a bug, kernel, or Xorg? -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

On 2013-06-06 07:37 (GMT+0200) Takashi Iwai composed:

At Wed, 05 Jun 2013 12:58:18 -0400 Felix Miata wrote:

...

On 2013-06-05 18:41 (GMT+0200) Takashi Iwai composed:

...

...

The only cases where we don't need KMS are with Nvidia and AMD binary drivers.

...

When did KMS support for mga begin?

I meant the only cases where non-KMS *kernel* drivers are used.

matrox_w1 definitely needs to be excluded. My G400 12.3 install was an upgrade from 12.2. The mga driver would not load because: (EE) mga: The PCI device 0x525 at 01@00:00:0 has a kernel module claiming it. (EE) mga: This driver cannot operate until it has been unloaded. I got mga to work by setting 'blacklist matrox_w1' in /etc/modprobe.d/90-user.conf, setting NO_KMS_IN_INITRD="yes" in /etc/sysconfig/kernel, and rebuilding initrd. Fresh installation of 13.1m1 produced no hang as described in $SUBJECT, but for mga driver to work, all the same reconfiguration as in 12.3 was necessary. In both 12.3 & 13.1m1, absent configured xorg.conf or xorg.conf.d/, video mode was 1920x1440, too high for comfort with my CRT, so I used the xorg.conf file I was using in 12.2 to produce 1600x1200, and virtual 1920x1200. -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

Hi Takashi, Le Wednesday 05 June 2013 à 18:41 +0200, Takashi Iwai a écrit :

At Wed, 05 Jun 2013 18:29:53 +0200, Jean Delvare wrote:

...

* With DRM/KMS being the norm now, I believe drm and drm_kms_helper could be built-in, even though the former is quite large. drm selects i2c-algo-bit, most (all?) DRM/KMS drivers use it. Even servers have graphics chips.

I thought i2c-algo-bit is only about i915, but it's no big deal, it's small. The only cases where we don't need KMS are with Nvidia and AMD binary drivers. But built-in drm and drm_kms_helper are harmless except for the memory usage.

i2c-algo-bit is definitely used by radeon, and I can see indirect references to it in nouveau as well. Note that it is also needed for many TV / DVB adapters and a few framebuffer drivers. So I suspect almost every system out there uses it. In fact it was the first module catching my attention and getting me to ask the question: are there other modules which we'd rather build in?

...

* I suppose fuse is only used on desktop machines by default, for gvfs, so having it as a module seems OK.

NTFS will use FUSE, too.

Right, but then again not everyone mounts NTFS partitions under Linux, so I don't think this is enough to warrant building fuse in.

...

* I think microcode could be unloaded after use, maybe for security reasons it might even be preferable, or just because there is simply no point in keeping it in memory. Thomas? Either way we should keep it modular.

The microcode driver loads the firmware by itself via request_firmware() at probe. If we build in, we'll need microcode firmware in initrd, to.

Ah, good point. So we leave microcode modular, period. Still I think it would make sense to unload it after the microcode update.

...

* For sound drivers, Takashi should know better than me. I suppose that the rationale for having these modular is that servers have no sound chip?

Yes. Also I'm still providing update KMP, so it's convenient to keep them as modules.

Fair enough.

pcspkr was built in ago, but this was split as a module. Making it as a module has another merit: you can blacklist it if you are annoyed by beep sound.

Oh yes :) Thanks for your input, Takashi! -- Jean Delvare Suse L3 -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

At Thu, 06 Jun 2013 08:47:24 +0200, Jean Delvare wrote:

Takashi,

Le Wednesday 05 June 2013 à 18:41 +0200, Takashi Iwai a écrit :

...

At Wed, 05 Jun 2013 18:29:53 +0200, Jean Delvare wrote:

...

* With DRM/KMS being the norm now, I believe drm and drm_kms_helper could be built-in, even though the former is quite large. drm selects i2c-algo-bit, most (all?) DRM/KMS drivers use it. Even servers have graphics chips.

I thought i2c-algo-bit is only about i915, but it's no big deal, it's small. The only cases where we don't need KMS are with Nvidia and AMD binary drivers. But built-in drm and drm_kms_helper are harmless except for the memory usage.

Thinking again about what you said of sound drivers: did you never have to build a KMP for a DRM/KMS module which required drm, drm_kms_helper or i2c-algo-bit to be rebuilt? I suppose this would be a good enough reason to leave them modular.

Yes, it's possible. I remember someone offered update drm KMPs in the past indeed, too. I guess i2c-algo-bit has been rarely changed (you must know better :), but drm stuff is in flux. So, unless you see obvious speedup with these helpers to be built-in, I'm for keeping them as modules. Takashi -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

On 06/06/2013 10:57 AM, Jean Delvare wrote:

Hi Cristian,

Le Wednesday 05 June 2013 à 13:51 -0400, Cristian Rodríguez a écrit :

...

El 05/06/13 12:29, Jean Delvare escribió:

...

Hi all,

Having worked on built-in drivers which could be modularized, I am now looking for modular drivers which could be built-in. Our policy is that drivers which are not hardware-specific and are needed on a majority of systems should be built into the kernel directly.

You are missing autofs4 and af_packet , autofs4 is loaded unconditionally by systemd (required by .automount units) and af_packet is requested any time a dhcp client fires up.

Well, I admit that one of my machines is running SLED11 SP3, one is running openSUSE 12.2 and only two are running openSUSE 12.3. But testing only two machines would have resulted in too many false positives and I thought missing some modules would be better than listing too many. I already have more than enough to play with ;)

If I remove the SLED11 SP3 machine, two generic modules are indeed added to the list: * ata_generic * autofs4

No af_packet here, as the 3 systems use static IP addressing.

I don't think autofs4 should be loaded on all systems, but that's a systemd issue. Building it into the kernel is thus not the solution.

Not sure about ata_generic. On the one hand, it is included in the initrd and thus loaded on all my machines. But OTOH it is not bound to any device on these systems. So the right solution might be teach mkinitrd to only embed ata_generic when it is really needed, rather than building it into the kernel. Michal? OTOH, if driver auto-loading ends up kicking in anyway due to modalias, and there is no easy way to avoid that, then having it built-in would be just as good. Unless driver loading order matters so that specific drivers are used first. I see that ata_piix is always loaded first on my 3 machines, this might be on purpose. I'll do some tests.

Argl. Please don't. ata_generic is the absolute fallback if for some reason the system doesn't ship with ATA drivers for that particular HBA. Which _really_ shouldn't happen anymore. Please do _not_ have this module built-in.

...

About why the sg module is loaded, it is loaded by systemd/udev apparently a bug report was opened for that (bnc#761109 according to the logs)

Thanks for the pointer. The possibility to build sg into the kernel is mentioned in that bug as a recommendation from upstream udev developers. It's certainly worth trying, but then I don't know how the udev rule will behave (not sure if TEST=="[module/sg]" works when sg is built-in.)

The general idea here is to move over to 'bsg' for devices as far as udev is concerned. But that's a long-term goal, not sure whether we'll be able to pull that off in time for SLES12. In general I'm _quite_ unhappy with having the SCSI subsystem built in. It might be okay for openSUSE, but for SLE it will become a maintenance nightmare. Try adding a PDLP driver for a built-in module ... Is there a general consensus to move to built-in drivers for SLE 12? Cheers, Hannes -- Dr. Hannes Reinecke zSeries & Storage hare@suse.de +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

On 06/06/2013 12:40 PM, Jean Delvare wrote:

Hi Hannes,

Thanks for joining the discussion.

Le Thursday 06 June 2013 à 11:02 +0200, Hannes Reinecke a écrit :

...

On 06/06/2013 10:57 AM, Jean Delvare wrote:

...

Not sure about ata_generic. On the one hand, it is included in the initrd and thus loaded on all my machines. But OTOH it is not bound to any device on these systems. So the right solution might be teach mkinitrd to only embed ata_generic when it is really needed, rather than building it into the kernel. Michal? OTOH, if driver auto-loading ends up kicking in anyway due to modalias, and there is no easy way to avoid that, then having it built-in would be just as good. Unless driver loading order matters so that specific drivers are used first. I see that ata_piix is always loaded first on my 3 machines, this might be on purpose. I'll do some tests.

Argl. Please don't.

ata_generic is the absolute fallback if for some reason the system doesn't ship with ATA drivers for that particular HBA. Which _really_ shouldn't happen anymore.

Please do _not_ have this module built-in.

I just gave it a try, and did not hit any issue. My devices are still driven by the ata_piix driver. There must be some magic to give other drivers priority over ata_generic.

If the ata_generic module ends up being always loaded anyway, what's the problem with it being built-in instead?

...

...

Thanks for the pointer. The possibility to build sg into the kernel is mentioned in that bug as a recommendation from upstream udev developers. It's certainly worth trying, but then I don't know how the udev rule will behave (not sure if TEST=="[module/sg]" works when sg is built-in.)

The general idea here is to move over to 'bsg' for devices as far as udev is concerned. But that's a long-term goal, not sure whether we'll be able to pull that off in time for SLES12.

In general I'm _quite_ unhappy with having the SCSI subsystem built in.

Well the SCSI core and disk (sd) drivers are already built into the openSUSE kernel for quite some time. And they are modular in SLE. I see no good reason not to do the same for sg.

...

It might be okay for openSUSE, but for SLE it will become a maintenance nightmare. Try adding a PDLP driver for a built-in module ...

Sure, this is a valid concern. I have already agreed to leave sound and drm drivers modular because of this. But have you ever built a PLDP/KMP driver for sg or ata_generic?

Not for them. But I did an PLDP driver for Intels 'isci' driver, which required us to basically replace scsi_mod.ko. Which you couldn't do when it's built in. And I _still_ remember the pain we had when switching over to libata; the original ide driver were built-in making the switch-over long and painful. We really should not re-create this situation. Plus for SLES boot-times are not really a concern. If you ever waited for several _minutes_ for an Ivy Bridge server to even display the BIOS messages boot time becomes less of a concern....

...

Is there a general consensus to move to built-in drivers for SLE 12?

My focus is on openSUSE right now, even though I keep SLE in a corner of my mind, as you do.

The general consensus for openSUSE is that drivers which end up being loaded unconditionally or on almost all machines for a given architecture are candidates to be built-in, because it improves boot speed, unless we have a good reason to not do that. KMPs are one such reason. In the end this is a per-module decision.

I just want to raise a concern for SLE. Could you please file a feature request for SLE so that this issue is properly tracked? Cheers, Hannes -- Dr. Hannes Reinecke zSeries & Storage hare@suse.de +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

Le Thursday 06 June 2013 à 12:51 +0200, Hannes Reinecke a écrit :

On 06/06/2013 12:40 PM, Jean Delvare wrote:

...

Le Thursday 06 June 2013 à 11:02 +0200, Hannes Reinecke a écrit :

...

It might be okay for openSUSE, but for SLE it will become a maintenance nightmare. Try adding a PDLP driver for a built-in module ...

Sure, this is a valid concern. I have already agreed to leave sound and drm drivers modular because of this. But have you ever built a PLDP/KMP driver for sg or ata_generic?

Not for them. But I did an PLDP driver for Intels 'isci' driver, which required us to basically replace scsi_mod.ko. Which you couldn't do when it's built in.

OK, understood. But note that CONFIG_SCSI=y in the openSUSE kernels right now, and I didn't do it ;) I get the feeling that openSUSE and SLE aren't going to converge there, so we always have to adjust some settings (namely modularize everything we might ever have to overrise) when forking a new SLE kernel.

And I _still_ remember the pain we had when switching over to libata; the original ide driver were built-in making the switch-over long and painful.

I remember that too. Going for modules in an area where we anticipate an alternative implementation makes sense. The sg and bsg driver can be used together, they do not conflict, right?

We really should not re-create this situation. Plus for SLES boot-times are not really a concern.

But for SLED it is.

If you ever waited for several _minutes_ for an Ivy Bridge server to even display the BIOS messages boot time becomes less of a concern....

Yeah, my own workstation is horribly slow to boot as well. But my laptop is rather fast, and I certainly hope that in the future the BIOS part of the system initialization gets faster.

...

...

Is there a general consensus to move to built-in drivers for SLE 12?

My focus is on openSUSE right now, even though I keep SLE in a corner of my mind, as you do.

The general consensus for openSUSE is that drivers which end up being loaded unconditionally or on almost all machines for a given architecture are candidates to be built-in, because it improves boot speed, unless we have a good reason to not do that. KMPs are one such reason. In the end this is a per-module decision.

I just want to raise a concern for SLE. Could you please file a feature request for SLE so that this issue is properly tracked?

I'm afraid I don't understand what you expect from me here. -- Jean Delvare Suse L3 -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

В Thu, 06 Jun 2013 12:40:32 +0200 Jean Delvare пишет:

Hi Hannes,

Thanks for joining the discussion.

Le Thursday 06 June 2013 à 11:02 +0200, Hannes Reinecke a écrit :

...

On 06/06/2013 10:57 AM, Jean Delvare wrote:

...

Not sure about ata_generic. On the one hand, it is included in the initrd and thus loaded on all my machines. But OTOH it is not bound to any device on these systems. So the right solution might be teach mkinitrd to only embed ata_generic when it is really needed, rather than building it into the kernel. Michal? OTOH, if driver auto-loading ends up kicking in anyway due to modalias, and there is no easy way to avoid that, then having it built-in would be just as good. Unless driver loading order matters so that specific drivers are used first. I see that ata_piix is always loaded first on my 3 machines, this might be on purpose. I'll do some tests.

Argl. Please don't.

ata_generic is the absolute fallback if for some reason the system doesn't ship with ATA drivers for that particular HBA. Which _really_ shouldn't happen anymore.

Please do _not_ have this module built-in.

I just gave it a try, and did not hit any issue. My devices are still driven by the ata_piix driver. There must be some magic to give other drivers priority over ata_generic.

If the ata_generic module ends up being always loaded anyway, what's the problem with it being built-in instead?

The right solution is to not unconditionally load it, not to make it built-in. -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

On 6.6.2013 10:57, Jean Delvare wrote:

Not sure about ata_generic. On the one hand, it is included in the initrd and thus loaded on all my machines. But OTOH it is not bound to any device on these systems. So the right solution might be teach mkinitrd to only embed ata_generic when it is really needed, rather than building it into the kernel.

mkinitrd adds whatever the modalias for the storage controller resolves to. I do not have ata-generic in my initrd, nor is it loaded. Michal -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

On 6.6.2013 14:40, Jean Delvare wrote:

Hi Michal,

Le Thursday 06 June 2013 à 13:56 +0200, Michal Marek a écrit :

...

On 6.6.2013 10:57, Jean Delvare wrote:

...

Not sure about ata_generic. On the one hand, it is included in the initrd and thus loaded on all my machines. But OTOH it is not bound to any device on these systems. So the right solution might be teach mkinitrd to only embed ata_generic when it is really needed, rather than building it into the kernel.

mkinitrd adds whatever the modalias for the storage controller resolves to. I do not have ata-generic in my initrd, nor is it loaded.

I have ata_generic listed in /etc/sysconfig/kernel together with ata_piix. As I understand it, drivers are added to INITRD_MODULES there but never removed automatically, right? So it might be that it no longer matches any device on my system.

Either that, or some older version of the installer added it, even though mkinitrd no longer needed it.

Is there a magic command that would answer the question "is this driver useful for any device present on my machine"?

Remove it from the sysconfig file, run mkinitrd, reboot and see if udev loads it :).

It could also be that ata_generic is only usable when the storage controller is set to IDE mode in the BIOS, and not when set to AHCI. Maybe that's why you don't have ata_generic on your machine, it is in AHCI mode?

I can't check right now, but the controller is driven by the ahci driver at least.

As a side note, I am wondering why my systems are all in IDE mode and if I should switch them to AHCI mode.

Strange. I am by no means an expert, but I thought that AHCI had been a de-facto standard on desktop and laptop hardware for the last couple of years. Michal -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

В Thu, 06 Jun 2013 16:56:33 +0200 Jean Delvare пишет:

So I would expect udev to load it. Unless udev skips module loading if the device already has a driver bound?

Yes. -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

В Thu, 06 Jun 2013 17:41:41 +0200 Michal Marek пишет:

On 6.6.2013 16:56, Jean Delvare wrote:

...

Hi Michal,

Le Thursday 06 June 2013 à 14:55 +0200, Michal Marek a écrit :

...

On 6.6.2013 14:40, Jean Delvare wrote:

...

Is there a magic command that would answer the question "is this driver useful for any device present on my machine"?

Remove it from the sysconfig file, run mkinitrd, reboot and see if udev loads it :).

Ah ah, yes of course :) Tried that and no, ata_generic is not loaded by udev. But I do not understand why, because:

$ /sbin/modinfo ata_generic (...) alias: pci:v*d*sv*sd*bc01sc01i*

$ /sbin/lspci -nn | grep '\[0101\]' 00:1f.2 IDE interface [0101]: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Controller #1 [8086:3a20] 00:1f.5 IDE interface [0101]: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Controller #2 [8086:3a26]

So I would expect udev to load it. Unless udev skips module loading if the device already has a driver bound?

AFAIK udev only does an equivalent of /sbin/modprobe $(cat /sys/devices/pci0000:00/0000:00:1f.2/modalias)

No, it checks whether device is already claimed by driver. DRIVER!="?*", ... -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

Hi Cristian, Le Thursday 06 June 2013 à 11:41 -0400, Cristian Rodríguez a écrit :

On 06/06/2013 04:57 AM, Jean Delvare wrote:

...

I don't think autofs4 should be loaded on all systems, but that's a systemd issue. Building it into the kernel is thus not the solution.

There is no problem with systemd, systemd REQUIRES autofs in the kernel. it is part of the basic functionality.

Ah, sorry, I had misunderstood you. Yes, I see it now, systemd itself uses autofs4.

by letting autofs4 as a module you only slow down the boot process.

Slowing down just as with every other module which need to be loaded? Or is there anything special about it that makes it worse? So you think autofs4 is a good candidate to be built into the kernel, do I get this right? Miklos, is there any reason not to?

There is no "issue" here, it is the expected behaviour, it is how systemd work and will continue to whether you like it or not.

This is how things are today, yes. It might change someday. -- Jean Delvare Suse L3 -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

On 6/5/13 7:37 PM, Thomas Renninger wrote:

On Wednesday, June 05, 2013 06:29:53 PM Jean Delvare wrote:

...

Hi all,

Having worked on built-in drivers which could be modularized, I am now looking for modular drivers which could be built-in. Our policy is that drivers which are not hardware-specific and are needed on a majority of systems should be built into the kernel directly.

So I checked which modules were loaded on my 4 systems. After filtering out hardware-specific modules, I am left with the following list:

ACPI and thermal drivers: * button * processor * thermal_sys

Block device drivers * cdrom * scsi_dh * scsi_dh_alua * scsi_dh_emc * scsi_dh_hp_sw * scsi_dh_rdac * sg * sr_mod

Graphics helper drivers * drm * drm_kms_helper * i2c_algo_bit

Filesystem drivers: * fuse * lockd * sunrpc

CPU drivers: * microcode

Sound drivers: * pcspkr * snd * snd_page_alloc * snd_pcm * snd_seq * snd_seq_device * snd_timer * soundcore

If anyone has an opinion on whether these should be built into the kernel or left as modules, this is the right time to discuss it. My personal thoughts, with mostly x86 in mind:

* The ACPI button and processor drivers could be built-in. Most x86 systems use ACPI and they supposedly all have at least one processor and one power button ;) And thermal_sys is selected by processor. There still is the acpi_processor_load_module() function in: drivers/acpi/processor_perflib.c which tries to load acpi_cpufreq.ko which might not work if processor.ko is built in. But this should not be needed since acpi_cpufreq is tried to be loaded via acpi CPU capability flag and the function should just vanish.

There might be side effects with processor.ko as quite some stuff depends on it (e.g. must get active after intel_idle driver or acpi_idle is registered first and possibly other side-effects), but in general this should work or it's a bug. Hm, can someone double check whether processor is compiled in on other distros? If not, we should not be the first doing it and run into unforseen problems.

No objections with the button driver.

...

...

* I think microcode could be unloaded after use, Hm, could this be done inside the module already? microcode driver could always return a negative value in its init function whether it could load a firmware file or not. (unless microcode.keep=1 is passed if one wants to examine something)

This would avoid: modprobe microcode rmmod microcode userspace control and possible race conditions. Does this make sense?

From a purely technical perspective, yes. We'd need to change the module loader to not treat whatever error code it would return as something other than an error though. -Jeff -- Jeff Mahoney SUSE Labs

Hi Jeff, Thomas,

Le Wednesday 05 June 2013 à 19:40 -0400, Jeff Mahoney a écrit :

...

On 6/5/13 7:37 PM, Thomas Renninger wrote:

...

On Wednesday, June 05, 2013 06:29:53 PM Jean Delvare wrote:

...

* I think microcode could be unloaded after use,

Hm, could this be done inside the module already? microcode driver could always return a negative value in its init function whether it could load a firmware file or not. (unless microcode.keep=1 is passed if one wants to examine something)

This would avoid: modprobe microcode rmmod microcode userspace control and possible race conditions. Does this make sense?

Oh, I thought user-space had to do something for the CPU microcode to actually get updated. Or has it changed and just loading the module triggers the update now? Yep, that changed. Intel per CPU family-model-stepping split up files are here: /lib/firmware/intel-ucode and AMD here: /lib/firmware/amd-ucode and this is fetched via firmware loader instead of a tiny binary pushing

On Thursday, June 06, 2013 08:55:56 AM Jean Delvare wrote: things through /dev/cpu/microcode. The latter is deprecated for some time: CONFIG_MICROCODE_OLD_INTERFACE=y This even changed from SLES 11 SP2 -> SP3. Factory microcode_ctl package still needs some cleanup. Anyway, keeping this one as a module is a good idea. Thomas -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

Hi Linda, Le Thursday 06 June 2013 à 06:04 -0700, Linda Walsh a écrit :

ACPI and thermal drivers: * button # dont' need this one on any system * processor * thermal_sys # nor this one

Block device drivers * cdrom * scsi_dh #none of these are needed on my systems -- * scsi_dh_alua # I might be able to use sg & sr_mod * scsi_dh_emc * scsi_dh_hp_sw * scsi_dh_rdac * sg * sr_mod

Graphics helper drivers * drm # Don't need any of these * drm_kms_helper * i2c_algo_bit

Filesystem drivers: * fuse # might use these * lockd # I use smb not nfs -- its considerable * sunrpc # faster

CPU drivers: * microcode # it fits my HW, but have never used it

Sound drivers: * pcspkr # sure * snd # none of the rest * snd_page_alloc * snd_pcm * snd_seq * snd_seq_device * snd_timer * soundcore --------------------------- Those are my 3 computers -- 1 desktop, 1 server, 1 ancient celeron...

Thanks for the feedback, this is appreciated. I'm surprised by a few things though, if you could clarify... You claim that none of your systems has the "button" driver loaded? The processor driver depends on thermal_sys, so you can't have the former without the latter. No optical drive in any of your systems? sg is loaded automatically so you certainly have it. If you don't use drm, does it mean you have binary graphics drivers everywhere? If you use a non-trivial desktop environment, you certainly use fuse even if you don't know. At least Gnome and Xfce use it. Microcode loading is automatic, so you certainly have used it already but you didn't know.

Seriously... what need to be forced on for built-in are deviced needed to **BOOT** enough so 1 hard disk is loaded and anything else can be picked up off the first hard disk.

This isn't the current strategy for openSUSE kernels, sorry. This has been discussed several times in the past. Long story sort: loading modules takes time and uses extra memory, so building everything as a module is not optimal from a performance point of view. As a consequence, it was decided that non hardware-specific drivers which are needed on a large majority of systems should be built into the kernel - unless we have a good reason not to.

Anything more than that and you are hurting everyone by forcing everyone to include things they will never use. It keeps the kernel lean -- and memory use and overhead lower.

It's not about building everything into the kernel. I have established a list of modules loaded on all my machines. Each individual case has been discussed with the interested people and the conclusion is that about only 4 modules in the initial list could be built-in. This isn't a huge number, so I don't think there is anything to be afraid of. Also note that a few weeks ago I have modularized about 20 drivers which were built-in so far, for specific hardware most people do not have, so it was really wasted boot time and memory for everybody. But nobody (including you) had been paying attention to that before I looked into it. So please don't come and complain now.

Things needed for my main linux system to boot: megaraid_sas, maybe pcieport.

Pcieport is built-in already, megaraid_sas is hardware-specific so it goes to your initrd.

to come up all the way -- bnx, e1000e ehci-pci, ioatdma ixbge lpc_ich & uhci_hcd but those aren't needed to start the boot process.. (at least those are HW items). There are many other I can load -- some with more effect / usefulness than others. But none needed to be built in...even though many I will use -- I don't build them in because they aren't needed immediately at boot.

So you're building your own kernel? If so, you don't care at all about the changes being discussed here.

Think of MS's boot process -- they have things that boot "at boot", things that boot as the system comes up, things that boot only as needed" and things that start about 20-30 seconds after the rest (not needed for but but likely needed for full system functionality.

To be honest, I don't know a thing about how MS does it, and to be even more honest (if this is possible), I don't really care.

Doing that allows you to get booted to a point where you are responsive to the user.

Ah ah, let me laugh, really. You get booted to a point where you see your desktop and a totally unresponsive system because ten dozen services, programs, applets, whatever, are trying to load in the background in a totally uncontrolled way. That is how I'd describe my limited Windows experience (mainly on systems I do not own.)

So I would SERIOUSLY reconsider building modules in just because you think many people will eventually use them during their session -- that's the wrong "bar" -- the only ones, IMO that should at front is the minimal amount of drivers needed to get most people started.

Most modules get loaded by udev or systemd before the desktop environment is loaded. So built-in or modular make absolutely no different in that respect. Only a few kernel drivers can really be loaded on-demand, and these are obviously not candidates to being built-in.

Besides booting faster,

Experiments have shown this is exactly the other way around. Otherwise I wouldn't be doing what I'm doing...

it has the added advantage of few drivers loading will give a higher boot reliability. Too many times I have things that didn't need to come up at boot for me to have been able to get to the machine (and fix any outstanding problems), but the trend now is toward *not trying* and rolling over dead at the slightest hairs output place. VS. a fault tolerant machine that comes up in spite of errors in spme state that it can be remotely used to repair itself and come up to full function.

This is a completely different discussion. This is the point at which I stopped reading you, because really, if you want to send long long messages like that, you first should ensure they are properly formatted so that just reading them is not such a burden. Thanks nevertheless, -- Jean Delvare Suse L3 -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

Le Thursday 06 June 2013 à 12:36 -0700, Linda Walsh a écrit :

Jean Delvare wrote:

...

You claim that none of your systems has the "button" driver loaded?

---

They will load it -- but it isn't needed... it doesn't do anything. They will work w/o it as well (none of mine are laptops).

This driver is responsible for capturing and routing the power button, sleep button and lid events. Without it, you can't shut down your machine cleanly with the power button. Just try it. Well maybe you don't need this feature, but it is enabled by default and I'm sure most users expect it to work. Disable this feature and users will press the button for 4 seconds and hard shut-down the machine instead. Yes, like Windows users do.

...

The processor driver depends on thermal_sys, so you can't have the former without the latter.

---

# ls -d /sys/module/{proc*,therm*} ls: cannot access /sys/module/therm*: No such file or directory /sys/module/processor/

Certainly because you rebuilt your kernel with CONFIG_THERMAL!=m. I wouldn't want to run a kernel without it. We are discussing changes to be done (or not) to the openSUSE kernel flavor config files. Any local configuration change you have already done may lead you to different conclusions which are irrelevant for the discussion. Have you taken the time to read the whole discussion thread?

...

sg is loaded automatically so you certainly have it.

----

Yes.. I just wasn't sure what it was for.

As discussed somewhere else in this thread, sg may not be needed on all systems, but it is impossible to tell when it is needed and when not. So from the distribution perspective, the choice is between building it as a module and loading it (almost) unconditionally, and building it into the kernel. The only advantages of having it as a module are: to have the possibility to override it with a KMP, and to let advanced users blacklist it (you might actually have to edit the udev rule that loads it) or unload it. I'm not convinced that these advantages are worth the small extra boot time delay for all users.

...

If you don't use drm, does it mean you have binary graphics drivers everywhere?

----

??? Um... neither of my systems running linux full time run a desktop they both run as servers and only boot up to rc3. My

These days, with KMS, even the console is backed by DRM drivers.

other if it runs linux would try to use a binary nvidia driver... which might use drm, dunno. been a while since I trie.

No, the binary Nvidia driver doesn't use drm.

...

(...) This isn't the current strategy for openSUSE kernels, sorry. This has been discussed several times in the past. Long story sort: loading modules takes time and uses extra memory, so building everything as a module is not optimal from a performance point of view.

----

That's just plain not true. My system boots considerably faster if I don't boot things that aren't needed.

You system boots faster because you built a custom kernel for it. Nobody is arguing about that. But the discussion I am having here is how to make the default openSUSE kernels boot faster on average.

Microsoft will disagree with you as well as far as user perceptions go and getting to some sort of responsiveness about your system (which isn't the same as talking absolute numbers -- perceptions are what drives the market, not facts).

I did not measure any number when seeing Windows systems starting. My perception is exactly what I reported: the desktop shows up fast, but then it takes a long time before the system is actually usable. What Microsoft (and others) are advertising are actually numbers - desktop appears N seconds after boot, yay! So I'm afraid you god it exactly the wrong way around. What drives the market is, well, marketing, habits, patents and lobbying. Neither perception not facts, I'm afraid. You seem to be a big fan of Microsoft BTW, I'm curious why (but don't get me wrong, still very happy that) you run Linux on your machines ;-)

...

As a consequence, it was decided that non hardware-specific drivers which are needed on a large majority of systems should be built into the kernel - unless we have a good reason not to.

----

That's why I build my own kernel and have generally until the latest mess with systemd, booted in about half the time as a suse kernel.

This is both interesting and impressive. I know that others at Suse have made such experiments already, sometimes as hackweek projects. Have you been able to isolate the specific kernel components which slowed down boot the most on your systems?

...

Also note that a few weeks ago I have modularized about 20 drivers which were built-in so far, for specific hardware most people do not have, so it was really wasted boot time and memory for everybody. But nobody (including you) had been paying attention to that before I looked into it. So please don't come and complain now.

----

I didn't complain because they never were in my kernel to begin with ;-).

Fair enough, but I can't recall you thanking me for it either ;-)

I've been building my own for over 10 years. I have tried using the suse kernel but it was too big, and too slow and had behaviors that were too different from the vanilla kernel to be able to report kernel bugs against it.

I'm not sure why you bothered taking part in this discussion then. Apparently you have decided long ago that distribution kernels were not for you. This is your right and I respect that. But whatever my conclusions are at the end of this discussion, you will keep building your custom kernel anyway.

(...) When I see my desktop, it's totally responsive. The unnecessary services are booted in background at low priority. They don't affect foreground services.

They do. "In background at low priority" is a lie as long as you have a single end-user-class, spinning HDD. Even we have experienced that with beagle, tracker etc. in the past. The I/O is the bottleneck and in fact you can hear it. I used to know my Windows system was ready when I stopped hearing the hard disk drive work like crazy. Thankfully SSDs are improving the situation significantly.

That's because you are booting with a initrd.

Yes, we are booting with an initrd. And I am trying to make that happen as fast as possible. I am not discussing whether initrd is good or if it could be avoided or if it could be replaced by something else. If you have an interest in this, feel free to work on it, start your own discussion thread, etc.

Once you've added in that cost, you've hidden most other benefits.. My system would boot from start of unpacking linux to system prompt and all services running in ~25 seconds.

That number says nothing until you compare it with other approaches on the same system. My workstation takes less than 10 seconds from unpacking linux to graphical prompt, with the desktop kernel flavor. That's much faster than yours, all it means is that a SSD, a fast CPU and enough RAM helps a lot for fast booting. It doesn't mean that the openSUSE desktop kernel is intrinsically faster than your custom kernel. -- Jean Delvare Suse L3 -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

Hi Linda, Le Thursday 06 June 2013 à 12:41 -0700, Linda Walsh a écrit :

Jean Delvare wrote:

...

So you're building your own kernel? If so, you don't care at all about the changes being discussed here.

--- BTW -- This is a circular argument.

I build my own kernel because the kernel suse has included include doesn't meet my needs (memory usage, performance, etc).

We have a number of kernel flavors for each architecture, and we try to make these generic enough to satisfy as many users as possible. However it is impossible to have an optimized kernel for every system. If a user really wants that, he/she has to rebuild the kernel with a custom configuration file. But in general the gain is simply not worth the pain. Also note that the kernel is not the only package in this case. Configuration decisions are made in every package, trying to cover the needs of the majority, often at the expense of compactness or performance, by a few percents.

So to say that I don't care about what you choose to build is based on the fact that I had to build my own to get what I want. If you could build things flexibly enough for my needs then I wouldn't **have** to build my own kernel (I might still just to keep hands on latest kernel builds, but at least I could switch off between my own and a suse kernel -- something I tried to do for a long time and gave up on back in the 11.x or 10.x timeframe.

I am curious how special your systems are, that you believe they gain significantly from a custom kernel. I doubt this is only about a few drivers being built-in and you don't use them, right? There must be more than that? -- Jean Delvare Suse L3 -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org

Hi Cristian, Le Saturday 08 March 2014 à 12:31 -0300, Cristian Rodríguez a écrit :

af_packet module is also loaded pretty much everywhere, as it is required by at least the DHCP client..you could at least set it to Y in the -desktop variant

This was discussed before. Not everyone is using DHCP. Most of my machines don't, and I'm running the desktop kernel flavor on all of them. -- Jean Delvare SUSE L3 Support -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org