Ralph Ellis wrote:

opensuse.org, You don't have to wait. Just take a look at the updater problems on the mailing list and include the Smart and Smart Gui

not even necessary, updates can be done with YOU (but it's a good idea to make it soon) jdd -- http://www.dodin.net http://dodin.org/galerie_photo_web/expo/index.html http://lucien.dodin.net http://fr.susewiki.org/index.php?title=Gérer_ses_photos -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Ralph Ellis wrote:

There is not a huge performance difference between the 32 bit and 64 bit setups unless you have a lot of memory. 64 bit setups can use more than 2 gigs of memory and I believe that it is more efficient.

I don't about any performance differences, but a 32bit machine has no problems utilising more than 2Gb of memory. /Per Jessen, Zürich -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Randall R Schulz wrote:

On Saturday 10 June 2006 13:24, Per Jessen wrote:

...

I don't about any performance differences, but a 32bit machine has no problems utilising more than 2Gb of memory.

But a 32-bit machine does hit its physical RAM limit at 4GB,

AFAIK, it's the address space size that is 4Gb, not the physical RAM limit. Intel Pentium 32bit processors since around the PentiumPro have all had 36 address lines, so max 64Gb physical memory. For instance, I have a 32bit Compaq machine that is quite happy with its 5Gb physical RAM. /Per Jessen, Z�rich -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Randall R Schulz wrote:

...

AFAIK, it's the address space size that is 4Gb, not the physical RAM limit. Intel Pentium 32bit processors since around the PentiumPro have all had 36 address lines, so max 64Gb physical memory. For instance, I have a 32bit Compaq machine that is quite happy with its 5Gb physical RAM.

Ah. I wasn't aware of that. Does Linux allow full exploitation of the memory management hardware of those processors so as to use the full 36 bits / 64 GB of RAM? Presumably it does.

There is a config option that enables the use of PAE (Physical Address Extension - CONFIG_HIGHMEM64G. AFAIK, that's all you need.

And this is all x86 processors more including and since the first PentiumPro?

I'm not sure, but at least all Pentium Pro, -II, -III and -4 models. http://en.wikipedia.org/wiki/Physical_Address_Extension /Per Jessen, Zürich -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Alexey Eremenko wrote:

...

...

And this is all x86 processors more including and since the first PentiumPro?

I'm not sure, but at least all Pentium Pro, -II, -III and -4 models.

Linux supports this only if your kernel supports it. Perhaps you will need to rebuild your kernel.

I believe the SUSE kernel-bigsmp has PAE enabled.

But this is not recommended because there is huge performance penalty from using over 4GB of RAM on 32-bit systems. (enabling PAE) - better way is to use AMD64 instructions.

But is that an alternative on all Intel 32bit systems? I don't know about the performance penalties, but the big hardware manufacturers have all been building machines with room for 8-16-32Gb - quite possibly even up to the full 64Gb. Maybe that was mostly for marketing, but still. Do you have any more info on the performance penalties related to using PAE? /Per Jessen, Zürich -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Bryan, On Sunday 11 June 2006 13:57, Bryan J. Smith wrote:

...

In actuality, I do _not_ recommend more than 1GiB on x86.

That makes no sense. On systems I've used recently (since multi-megabyte RAM configurations became an economical option for desktop systems), going from one gig to two has made a big difference for many of the scenarios I encounter in everyday work. I see no reason to stop anywhere short of the 4 GB that is natively addressable by a 32-bit architecture.

....

-- Bryan J. Smith

Randall Schulz -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Bryan J. Smith wrote:

On Sun, 2006-06-11 at 19:38 -0700, Randall R Schulz wrote:

...

That makes no sense. On systems I've used recently (since multi-megabyte RAM configurations became an economical option for desktop systems), going from one gig to two has made a big difference for many of the scenarios I encounter in everyday work. I see no reason to stop anywhere short of the 4 GB that is natively addressable by a 32-bit architecture.

Let me re-phrase ...

- 1GiB on Linux/x86 - 2GiB on NT/x86

If you go more than 1GB in Linux, go x86-64 -- specifically AMD64 and _not_ EM64T (aka IA-32e). There is a _significant_ performance hit when you go into the 4G/4G or 64G models.

But when your machine has 2Gb, you've yet to go into those models. So _why_ do _you_ not recommend more than 1Gb on x86? Just as Randall, I see no reason why one should not use e.g. 2 or 4Gb with an x86 processor. The performance hit is presumably that of going to a three-level address lookup when using PAE. /Per Jessen, Zürich -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Bryan J. Smith wrote:

My point is, and continues to be, if you need more than 1GiB of memory in Linux (2GiB in NT), you should be running Linux/x86-64 on an AMD64 processor (and not EM64T).

Bryan, I know that's your point, but you're neglecting to explain _why_. I'm not asking your advice, but an explanation. In particular as to why it is a bad idea to run an x86-machine with more than 1Gb RAM. /Per Jessen, Zürich -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Per Jessen wrote:

Randall R Schulz wrote:

...

...

AFAIK, it's the address space size that is 4Gb, not the physical RAM limit. Intel Pentium 32bit processors since around the PentiumPro have all had 36 address lines, so max 64Gb physical memory. For instance, I have a 32bit Compaq machine that is quite happy with its 5Gb physical RAM. Ah. I wasn't aware of that. Does Linux allow full exploitation of the memory management hardware of those processors so as to use the full 36 bits / 64 GB of RAM? Presumably it does.

There is a config option that enables the use of PAE (Physical Address Extension - CONFIG_HIGHMEM64G. AFAIK, that's all you need.

...

And this is all x86 processors more including and since the first PentiumPro?

I'm not sure, but at least all Pentium Pro, -II, -III and -4 models.

http://en.wikipedia.org/wiki/Physical_Address_Extension

That's sort of thing has been around for years and is known as memory mapping. For example, many of the old mini-computers used it and there was even an a version of the Zilog Z80 (Z800?) that used it. However, it imposes a performance penalty and applications have to know how to use it, to gain any benefit. -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Per Jessen wrote:

James Knott wrote:

...

...

http://en.wikipedia.org/wiki/Physical_Address_Extension [snip] However, it imposes a performance penalty and applications have to know how to use it, to gain any benefit.

I suspect you're right about the performance penalty, although I'd like to know more about it, but the applications need know nothing about PAE whatsoever. The application address space does not change.

While I don't have the exact details of PAE, in general memory mapping works by mapping physical memory outside of the normal addressing range, to an address within that range. This means that only a portion of that physical address range can be visible at a given time. An application has to be able to tell the operating system what portion of the physical memory it wants and also where in address space it wants it to be located. Here's some more info. http://www.microsoft.com/whdc/system/platform/server/PAE/pae_os.mspx -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Per Jessen wrote:

James Knott wrote:

...

While I don't have the exact details of PAE, in general memory mapping works by mapping physical memory outside of the normal addressing range, to an address within that range. This means that only a portion of that physical address range can be visible at a given time. An application has to be able to tell the operating system what portion of the physical memory it wants and also where in address space it wants it to be located.

Err, no. In Linux and many other OSes with virtual memory, no application needs to "tell the operating system what portion of the physical memory it wants" nor "where in address space it wants it to be located.". It only needs to say HOW MUCH it wants, and the virtual memory manager takes care of the rest.

Here's an example of why your application does not need to know about PAE. When you install a SUSE 10.1 system on your machine with 1Gb RAM (for instance), you can pick e.g. the default kernel and no doubt your system and your applications will run fine. Then you add some more RAM - let's say you now have 8Gb RAM. To make use of this, you now install the bigsmp kernel or alternatively you just rebuild your current kernel with the CONFIG_HIGHMEM64G option enabled. Your applications aren't touched, yet they run just fine - but of course you can now run a lot more of them. They don't need to know about how much memory your system has, nor how it is accessed - again, the virtual memory manager takes care of that.

Another example - I typically build applications on systems with a lot less memory than my servers. Maybe 512M, maybe 1Gb. So definitely without PAE. But these applications run fine on my servers with a lot more memory, without knowing anything about it. After all, their address space has not changed. It's still 4Gb per process, perhaps with a 3Gb/1Gb split for user/kernel.

In another message, I sent a link to a Microsoft description of PAE, where you'll find things such as: "Addressing physical memory above 4 GB requires more than the 32 bits of address offered by the standard operating mode of Intel (32-bit) processors. To this end, Intel introduced the 36-bit physical addressing mode called PAE, starting with the Intel Pentium Pro processor. This article describes some techniques that Microsoft® Windows® operating systems and several UNIX operating systems use to provide support to applications using PAE mode addressing. Because processes running in these environments have 32-bit pointers, the operating system must manage and present PAE's 36 bits of address in such a way that the applications can practically use it. The key question is: how does the operating system solve this problem? The performance, functionality, simplicity of programming, and reliability of how these issues are handled will determine the usefulness of the large memory support." and "3. Application Windowing A PAE-enabled operating system can introduce an API to allow a properly coded application access to physical memory anywhere in the system, even though it may be above 4 GB. Ideally, the API to allocate "high" physical memory and create or move the window should be quick and simple to code. This is highly advantageous for applications that require fast access to large amounts of data in memory. Sharing high memory between processes can introduce quite a bit of complexity into the API and the implementation. Windows avoids this kind of sharing. In addition, the support of paging makes the design and implementation of the operating system much more difficult and makes deterministic performance more difficult to achieve. Windows avoids paging of high memory as well." Without using PAE, a 32 bit app, has no way to tell the OS how much memory beyond 4 GB it wants. If an app doesn't use the PAE API, it can't access beyond that 4 GB. Perhaps you'd better read that article and get back to me. -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

On Mon, 2006-06-12 at 15:03 +0200, Per Jessen wrote:

Isn't that more or less what I started this thread with?

The problem is that you're applying kernel modes to application ABI.

Which is what I am ALSO saying. The application does not go beyond the 4Gb address space, the operating system (in our case Linux) does that for the application.

Again, re-read this portion ... "Because processes running in these environments have 32-bit pointers" First, the processes are built for 32-bit/4GiB pointers. That's a _hard_ limit. Secondly, the processes are built for various Linux memory models limitations. GCC can do several -- some automagically -- based on what the process uses. Several people have rebuilt x86 applications that can use up to 3GiB total. Now read this ... "the operating system must manage and present PAE's 36 bits of address in such a way that the applications can practically use it." That does _not_ mean the applications can use 36-bit. That means the OS is using page translation to "window" 36-bit physical memory into 32-bit flat memory.

1) do we have such an API in Linux? 2) is it necessary except for those rare cases where a single process needs to address more than 4Gb? I still maintain that (at least in Linux) using PAE is something for the kernel to do, not the application, with the possible exception of applications needing more than 4Gb of memory.

Yes, when the process doesn't need more than the common memory model limits used by x86 Linux kernels. Here's a good intro, but it still does _not_ cover application/user-space: http://www-128.ibm.com/developerworks/linux/library/l-memmod/index.html About the only thing it says under Extended paging is ... "This approach has practical limitations with respect to demand paging." A program can't just "demand page" in anything. You can't have the processor looking at more than the 1-3GiB allocated to user-space. That means programs using more than 1-3GiB of user-space have to be built _specially_ for the kernel/processor to handle. And that's before we look at the issues of the _hard_ 32-bit/4GiB limit of normalized pointers (although the solution to paging solves most of it). -- Bryan J. Smith Professional, technical annoyance mailto:b.j.smith@ieee.org http://thebs413.blogspot.com ------------------------------------------------------- Illegal Immigration = "Representation Without Taxation" -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Bryan J. Smith wrote:

On Mon, 2006-06-12 at 15:03 +0200, Per Jessen wrote:

...

Isn't that more or less what I started this thread with?

The problem is that you're applying kernel modes to application ABI.

...

Which is what I am ALSO saying. The application does not go beyond the 4Gb address space, the operating system (in our case Linux) does that for the application.

Again, re-read this portion ...

"Because processes running in these environments have 32-bit pointers"

First, the processes are built for 32-bit/4GiB pointers. That's a _hard_ limit.

[ clipped a bunch, maybe the answer I want :-) ] Specific single application: Radiance http://www.radiance-online.org/ ( it was here in the 10.0 distro. ) Do I take a hit trying to use Radiance on 1.5Gb x86? What about when I get a 3D CAD package sending building structures to Radiance for lighting design? Wine not wanted here. I am an end user, not much of a programmer. -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Per Jessen wrote:

James Knott wrote:

...

While I don't have the exact details of PAE, in general memory mapping works by mapping physical memory outside of the normal addressing range, to an address within that range. This means that only a portion of that physical address range can be visible at a given time. An application has to be able to tell the operating system what portion of the physical memory it wants and also where in address space it wants it to be located.

Err, no. In Linux and many other OSes with virtual memory, no application needs to "tell the operating system what portion of the physical memory it wants" nor "where in address space it wants it to be located.". It only needs to say HOW MUCH it wants, and the virtual memory manager takes care of the rest.

Forgot to mention in the other note. We're not talking about virtual memory here, which is an entirely different thing. Virtual memory is the process for making disk space appear as actual memory. This is transparent to the app. Memory mapping is a technique for allowing an app to access more memory space than it's normally able to. For example, in an earlier note, I mentioned the Zilog Z80 CPU, which, with 16 bit addressing, could only access 64 KB of memory. The memory mapper allowed it to access considerably more, but still only a total of 64 K at a time. With PAE, the same situation occurs. On 32 bit computes, a process can normally access only a 4 GB range. PAE allows access to more than that range, provided that the total amount of available space is no more than 4 GB. In order to use that greater range, an app has to use the PAE API to instruct the OS to make that memory beyond 4 GB available. If the app doesn't use PAE, it can't go beyond the basis 4 GB. Memory mapping and PAE work by mapping physical memory into a "window" within the normal address range. -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Per Jessen wrote:

James Knott wrote:

...

We're not talking about virtual memory here, which is an entirely different thing. Virtual memory is the process for making disk space appear as actual memory.

But we are talking about an application accessing memory, which on Linux is done exclusively through the virtual memory manager. That the VMM will use the PAE for addressing up to 64Gb of physical memory does not concern the application.

How is a 32 bit app supposed to even know about the memory beyond 4 GB, if it doesn't use PAE or equivalent? Like the memory mapping systems of years past, PAE is used to allow the use of a greater address range than the CPU is normally capable of. You might want to read up on the LIM boards used in the old XT systems. http://www.smartcomputing.com/editorial/dictionary/detail.asp?guida=&searchtype=1&DicID=8800&RefType=Dictionary

...

On 32 bit computes, a process can normally access only a 4 GB range. PAE allows access to more than that range, [snip] In order to use that greater range, an app has to use the PAE API to instruct the OS to make that memory beyond 4 GB available. If the app doesn't use PAE, it can't go beyond the basis 4 GB.

OK - do you know how this is done on Linux? I'm really curious as I've always considered this kernel-only stuff. I have no problem with the concept of mapping physical memory (normally not used by the kernel) into my own address-space, I just did not think Linux provided that option.

Sorry, I don't. That's a bit out of my area of expertise.

My understanding of Linux' use of PAE is that enabling CONFIG_HIGHMEM64 essentially makes the kernel switch to a three-level instead of a two-level address lookup.

/Per Jessen, Zürich

-- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Per Jessen wrote:

James Knott wrote:

...

How is a 32 bit app supposed to even know about the memory beyond 4 GB, if it doesn't use PAE or equivalent?

The application is not supposed to know. Really. Maybe some of its virtual memory is mapped to physical memory at >4Gb, but the application should not and does not care.

I think we've reached the point where you're trying to strech a point. My entire discussion has been about software in general, trying to access memory beyond the natural 4 GB limit of a 32 bit CPU. Whether it's the OS or app is irrelevant to the discussion. While the OS may be using it, an app still cannot access anything beyond that limit.

The operating system does it all, and the application address space remains at 4Gb. Like I've said all along.

/Per Jessen, Zürich

-- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

On Mon, 2006-06-12 at 14:35 +0200, Per Jessen wrote:

But we are talking about an application accessing memory, which on Linux is done exclusively through the virtual memory manager. That the VMM will use the PAE for addressing up to 64Gb of physical memory does not concern the application.

But how does the 32-bit application use its pointers to access and differentiate more than 4GiB? And that's _before_ we even look at how the application's object code is actually built into a memory model the kernel can support. I think people keep confusing the difference between "total userspace" among _all_ applications and "total userspace" used by *1* application. Segmentation, translation paging, etc... is addressed by the kernel for _all_ applications. But then you have "hard" 32-bit limitations in the application itself for x86. That's when both the application code and object code memory model _really_ come into play!

OK - do you know how this is done on Linux? I'm really curious as I've always considered this kernel-only stuff. I have no problem with the concept of mapping physical memory (normally not used by the kernel) into my own address-space, I just did not think Linux provided that option.

Oracle uses it. One of the reasons I implemented Opteron/x86-64 as soon as it became available in 2004 was because of the 2.5GiB "base" 32-bit of Oracle 9/10, and performance hit for using more.

My understanding of Linux' use of PAE is that enabling CONFIG_HIGHMEM64 essentially makes the kernel switch to a three-level instead of a two-level address lookup.

That's another story. People keep merging different kernel, API and ABI concepts. The kernel gives you more than 1-3GiB for _all_ applications through various modes -- including virtualizing memory. The application can use more than the hard 32-bit/4GiB pointer limitations by various techniques, as well as has to be built against the memory model the kernel supports (for more than 1-3GiB). It's easy to use page translation at the x86 kernel level to provide _more_ than 1-3GiB or even 4GiB of physical RAM to _different_ applications. The application-level issues arise when the x86 kernel attempts to allow an application to use more than 1-3GiB (using compile-time/directives) or even 4GiB (which requires some application code changes) for a _single_ application. AMD solved it very nicely with "Long Mode" for up to 48-bit physical addressing (although it's currently a 40-bit platform limitation in how EV6 works, which also affects A64/Opteron too) and then put in compatibility with PAE as well. You can't just do a "malloc" on x86 Linux and get more than 4GiB. -- Bryan P.S. the "3GiB" model for programs to use is not limited to x86 Linux. There is a "3GiB" model for x86 NT as well, including applications built for it (that won't run unless you pass an option to enable it in the boot.ini). -- Bryan J. Smith Professional, technical annoyance mailto:b.j.smith@ieee.org http://thebs413.blogspot.com ------------------------------------------------------- Illegal Immigration = "Representation Without Taxation" -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

On Mon, 2006-06-12 at 15:59 +0200, Per Jessen wrote:

That's certainly possible. Right from the beginning of this thread I've said the application address does not change, but remains at 4Gb. And that someone running non-PAE-aware applications may perfectly well benefit from running on a PAE-aware operating system such as Linux (when built with CONFIG_HIGHMEM64G).

No, it's quite the opposite! If you're running 1GiB processes, the _biggest_ performance hit you can do is run a PAE enabled kernel.

In summary - the application does not need to know about PAE to use memory addressed through PAE.

As long as the process is built for a 3GiB or lower model, yes.

Do you know what "it" is? I wonder if "it" is an option for mysql and maxdb too?

Since it's closed source, I don't know. Everything I've written that uses a lot of memory has been for POSIX64. I adopted Alpha from day 1 in the early '90s, and moved to SPARC64. I _avoided_ PCs for "heavy duty" work until the Opteron came out. Now I love Opteron -- both Linux/Opteron as well as Solaris/Opteron. Solaris/Opteron still gets my vote over Linux/Opteron for server duties (long story).

Well, if we exclude applications using the unknown PAE API, isn't the above really the _whole_ story?

I've never written to the various APIs myself, and there are all sorts of kernel conditions. Once you go beyond the 1-2GiB user-space models, all bets are off (even on 3GiB).

My point exactly. But I'm still interested in how it's done.

Hit one of the GCC forums and ask about the i686 target. They'll probably tell you more than you wanted to know. -- Bryan J. Smith Professional, technical annoyance mailto:b.j.smith@ieee.org http://thebs413.blogspot.com ------------------------------------------------------- Illegal Immigration = "Representation Without Taxation" -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

On Mon, 2006-06-12 at 14:35 +0200, Per Jessen wrote:

But we are talking about an application accessing memory, which on Linux is done exclusively through the virtual memory manager. That the VMM will use the PAE for addressing up to 64Gb of physical memory does not concern the application.

Also, to come at it from the VMM avenue ... While the VMM _can_ address the issue of _all_ programs using more than 1-3GiB or even 4GiB _total_ for _all_ applications, it can_not_ address the program's issue of allocating more memory than allowed by 32-bit/4GiB pointers. It also can't address issues where an application is built for the 1-3GiB model in its object code. That's utterly different than the kernel using page translation and relocating _individual_ programs. We're not talking programs that have to "work with" the kernel at "it's level" to allocate total memory in excess of what can be done for typical process user-space -- let alone _hard_ 32-bit/4GiB pointer limitations. -- Bryan J. Smith Professional, technical annoyance mailto:b.j.smith@ieee.org http://thebs413.blogspot.com ------------------------------------------------------- Illegal Immigration = "Representation Without Taxation" -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

On Sun, 2006-06-11 at 18:59 +0200, Per Jessen wrote:

Err, no. In Linux and many other OSes with virtual memory, no application needs to "tell the operating system what portion of the physical memory it wants" nor "where in address space it wants it to be located.". It only needs to say HOW MUCH it wants, and the virtual memory manager takes care of the rest.

That's _not_ entirely true. Again, here's where there is a difference between API and ABI come into play. If a 32-bit application needs to use more than 1-2GiB of memory (depending on the model), then there _are_ at least _compile-time_ issues to resolve -- possibly compiler/linker directives, etc... if not select code modification. -- Bryan J. Smith Professional, technical annoyance mailto:b.j.smith@ieee.org http://thebs413.blogspot.com ------------------------------------------------------- Illegal Immigration = "Representation Without Taxation" -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Anders Johansson wrote:

I suspect James is talking about an application that uses more than that amount - one that actually uses PAE itself. This is not the default in linux, only the kernel is PAE aware, so applications on 32 bit systems are stuck with the 4GB limit

Would a Linux application even be able to use PAE by itself? Does the kernel provide that option/interface? /Per Jessen, Zürich -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Bryan J. Smith wrote:

On Mon, 2006-06-12 at 13:32 +0200, Per Jessen wrote:

...

Would a Linux application even be able to use PAE by itself? Does the kernel provide that option/interface?

Yes. It's really a _complex_ discussion. And my answer below is _still_ a _mega_-simplification.

On x86, the kernel has various code to do this, along with GCC compile-time/directives. Applications that want to address more than 1-3GiB (depending on the model) of user-space memory have to be built differently. It's _never_ a matter of getting access to _all_ 4GiB of memory, even if the pointers are capable of (or being rationalized to) address upto 32-bit/4GiB. This is in _addition_ to how the kernel-processor actually do PAE (36-bit) -- which is a _serious_ performance hit.

Bryan, that really does not explain anything, not even simplified. How about starting with how a Linux application gets to use PAE by itself? What are the GCC options and assuming ordinary malloc() calls will not suffice, what are the kernel interfaces I need to use ? Or is there perhaps even a library that'll let me do this? I know how my application can get to use more than e.g. the standard 3Gb of user-space, but how do I get it beyond the 4Gb address-space? Which is what we're talking about. Well, that's what I'm talking about anyway :-) As to the serious performance hit, I guess it's a trade-off (as usual) - do I want to use the extra 12Gb on my 16GB system despite the performance hit, or will 4Gb really do. (and if so, why on earth did I bother with the extra 12Gb :-) /Per Jessen, Zürich -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

I suspect James is talking about an application that uses more than that amount - one that actually uses PAE itself. This is not the default in linux, only the kernel is PAE aware, so applications on 32 bit systems are stuck with the 4GB limit This is not entirely true. An application not only has its own memory component, it also has shared

On Sunday 11 June 2006 5:12 pm, Anders Johansson wrote: libraries and a bunch of other things. If the kernel is PAE aware, the application can also avail itself of these features. I was reading a Sybase paper on the Adaptive Server Enterprise which does this. But, even if a 32-bit application itself is not PAE aware, it can utilize then extended memory indirectly. -- Jerry Feldman Boston Linux and Unix user group http://www.blu.org PGP key id:C5061EA9 PGP Key fingerprint:053C 73EC 3AC1 5C44 3E14 9245 FB00 3ED5 C506 1EA9 -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

On Mon, 2006-06-12 at 09:20 -0400, Jerry Feldman wrote:

This is not entirely true. An application not only has its own memory component, it also has shared libraries and a bunch of other things. If the kernel is PAE aware, the application can also avail itself of these features. I was reading a Sybase paper on the Adaptive Server Enterprise which does this. But, even if a 32-bit application itself is not PAE aware, it can utilize then extended memory indirectly.

Any PAE kernel can map any process beyond 4GiB or whatever is physically paged into user-space memory (which is 3GiB or lower). It's when the application is using more than those user-space limits, let alone the _hard_ 32-bit/4GiB limit, that you've at least have to modify how it is built, if not select interaction in the code itself (such as between major modules). An application may still load if not. And an application will very much croak if it attempts to allocate and work on more memory than the base memory model limits at any one time. You can have up to 64GiB of physical RAM in PAE, but you can't have anything address more than 4GiB at any time -- before we even look at user-space/memory model constraints.

From a server I/O standpoint, depends on your kernel model, more and more kernel-space memory becomes a _major_ issue when using the 3G/1G model. Especially if it's tapping in from PAE 36-bit areas, that eats up reserved kernel memory -- taking away from memory mapped I/O.

Again, more than 1-2GiB on a server -- and I don't care what OS -- you should be on AMD64, *PERIOD*. I hate to seemingly "flaunt my resume," but I tell you this from experience integrating Opteron all over the financial industry the last 2 years. User-space is the least concern -- especially when user-space takes away from kernel space that is very much needed from memory mapped I/O and other reservations. -- Bryan J. Smith Professional, technical annoyance mailto:b.j.smith@ieee.org http://thebs413.blogspot.com ------------------------------------------------------- Illegal Immigration = "Representation Without Taxation" -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

It's when the application is using more than those user-space limits, let alone the _hard_ 32-bit/4GiB limit, that you've at least have to modify how it is built, if not select interaction in the code itself (such as between major modules). Exactly. There are some 32-bit applications that do explicitly user more

On Monday 12 June 2006 10:11 am, Bryan J. Smith wrote: than the 4GB. These would include database servers and CAD applications. -- Jerry Feldman Boston Linux and Unix user group http://www.blu.org PGP key id:C5061EA9 PGP Key fingerprint:053C 73EC 3AC1 5C44 3E14 9245 FB00 3ED5 C506 1EA9 -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

On Monday 12 June 2006 12:18 pm, Per Jessen wrote:

Does anyone know _how_ this is done?  I.e. which kernel interface or library do they use?  This functionality must be coming from somewhere. Bryan mentioned Oracle (obviously closed-source), but surely a publicly available kernel interface could benefit other open-source applications. And I mentioned Sybase. One way to do this is to use large shared memory segments. -- Jerry Feldman Boston Linux and Unix user group http://www.blu.org PGP key id:C5061EA9 PGP Key fingerprint:053C 73EC 3AC1 5C44 3E14 9245 FB00 3ED5 C506 1EA9

-- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Jerry Feldman wrote:

On Sun, 11 Jun 2006 12:40:16 -0400 James Knott wrote:

...

While I don't have the exact details of PAE, in general memory mapping works by mapping physical memory outside of the normal addressing range, to an address within that range. This means that only a portion of that physical address range can be visible at a given time. An application has to be able to tell the operating system what portion of the physical memory it wants and also where in address space it wants it to be located. The way an application works in Linux is that it is layed out into a 32-bit or 64-bit virtual environment. In general, an application consists of 3 major sections: Text - these are the instructions Data - Initialized data BSS - Unititialized data. Everything is mapped into a virtual address space. Additionally, shared libraries are also mapped into virtual memory. This virtual memory is blocked into pages (the page size can be changed on some systems). When a page is resident in physical memory, its virtual addresses are translated to physical addresses.

PAE is not virtual memory. Virtual memory is a method of making disk space appear as memory. PAE is a memory mapping technique, that allows an applcation to access memory (real or virtual) beyond the normal 4 GB limit. -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Anders Johansson wrote:

On Sunday 11 June 2006 22:06, James Knott wrote:

...

PAE is not virtual memory. Virtual memory is a method of making disk space appear as memory.

In general, virtual memory is a way of mapping one set of memory segments onto another memory segment. It's not inherent in the definition that it has to do with swap space, although that is a common use for it. But you can use a linux system without swap, and you'd still be using virtual memory. In particular, each application would still see its own contiguous memory space, even though it could be using non-contiguous chunks of real, physical memory

...

PAE is a memory mapping technique, that allows an applcation to access memory (real or virtual) beyond the normal 4 GB limit.

It looks to me like a variation of the standard Intel segmentation thing

Quite so. Back in the 8088 & 8086 days, the CPU had to use segment registers, to go beyond 64K bytes. The old DOS com files were an example of apps that couldn't go beyond 64K, because they didn't use the segment registers. The exe programs could use them to access the entire 1 MB range of the CPU. However, the applications had to "know" about using the segment registers, to access various areas of that 1 MB. Using the segment registers imposed a performance penalty. And again, only 64 K could be accessible at any time, for each of the 4 segment registers. -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

On Sunday 11 June 2006 23:16, Bryan J. Smith wrote:

Case-in-point: AMD 52-bit PAE uses no less than a 7-layer logic to provide 20-bit Real86/Virtual86, 32-bit Protected386 and 36-bit Processor Address Extension (PAE) modes.

PAE stands for Physical Address Extension, not "processor" I'll let someone who knows more about x86 assembly comment on the rest, I've tried my best to stay away from that mess

Regardless of the "base API model" used, the actual "application binary interface" (ABI) of the unliked as well as linked object code is the kicker.

"ABI of the unlinked as well as linked object code"?? Clarity is the child of understanding. -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

In fact, C compilers targets often build much better code for today's superscalar and/or RISC architectures than an assembler coder can. I certainly can attest to that. In general, the classic compiler is designed in 3 passes with the last being code generation. One many RISC architectures, including the Digital/Compaq/HP Alpha chip, there is another

On Sunday 11 June 2006 8:31 pm, Bryan J. Smith wrote: pass called a scheduler. The scheduler looks at the instruction stream and uses latency tables to reorder instructions to reduce latencies. The Alpha could issue 4 instructions simultaneously, so the scheduler would try to set up the groupings to optimize this. One reason why the high level language (C, FORTRAN, etc) is better than assembler, is that eventhough some assemblers do include the schedulers, the assembler coder can chose register combinations that cause the scheduler not to be able to generate the nest code. On the down side, if you have ever tried to debug scheduled code, you will find that it jumps all over the place. Just a quick note regarding Intel's EPIC - many of the compiler people from Digital/Compaq are currently at Intel. The EPIC architecture (eg IA64) relies upon compiler technology. -- Jerry Feldman Boston Linux and Unix user group http://www.blu.org PGP key id:C5061EA9 PGP Key fingerprint:053C 73EC 3AC1 5C44 3E14 9245 FB00 3ED5 C506 1EA9 -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

On Sunday 11 June 2006 4:06 pm, James Knott wrote:

PAE is not virtual memory.  Virtual memory is a method of making disk space appear as memory.  PAE is a memory mapping technique, that allows an applcation to access memory (real or virtual) beyond the normal 4 GB limit. This is true. Bryan and I had a bit of discussion about it. However, an application program is still mapped into virtual space. Linux, however, when implementing PAE extensions in the kernel can allow a 32-bit application to use more than 4GB (actually 64GB). -- Jerry Feldman Boston Linux and Unix user group http://www.blu.org PGP key id:C5061EA9 PGP Key fingerprint:053C 73EC 3AC1 5C44 3E14 9245 FB00 3ED5 C506 1EA9

-- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

On Sun, 2006-06-11 at 12:40 -0400, James Knott wrote:

While I don't have the exact details of PAE, in general memory mapping works by mapping physical memory outside of the normal addressing range, to an address within that range. This means that only a portion of that physical address range can be visible at a given time. An application has to be able to tell the operating system what portion of the physical memory it wants and also where in address space it wants it to be located.

There is a whole can of worms you open when you start talking about 32-bit flat, PAE 36-bit (PPro+/Athlon+) and newer PAE 52-bit (x86-64) from translation and performance standpoints.

Here's some more info. http://www.microsoft.com/whdc/system/platform/server/PAE/pae_os.mspx

Know that Linux's PAE and non-PAE memory models and approaches differ _radically_ from NT's. -- Bryan J. Smith Professional, technical annoyance mailto:b.j.smith@ieee.org http://thebs413.blogspot.com ------------------------------------------------------- Illegal Immigration = "Representation Without Taxation" -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com

Bryan J. Smith wrote:

On Sun, 2006-06-11 at 18:26 +0200, Per Jessen wrote:

...

I suspect you're right about the performance penalty, although I'd like to know more about it, but the applications need know nothing about PAE whatsoever. The application address space does not change.

If you have 1GiB of RAM or less, it's recommended you rebuild the kernel with the old 1G/3G model for a good 10-20% performance boost.

I thought that was standard in the SUSE-supplied kernel - I'll have to check now. /Per Jessen, Zürich -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com