Physical memory available != virtual memory available for allocation. In an 8GB DRAM system, the BIOS gives the OS approximately 7.5GB (because of various I/O issues). You should be able to get the kernel to allocate a large amount but once you allocate larger than phys memory, you will page awfully. You don't say whether the app requesting the memory is 32-bit or 64-bit. I assume 64-bit. Which PGI compiler (F77? F90?) and which version (5.0? 5.1? 5.1-3?) are you using? How are you allocating the array? ] -Rich ... ] AMD Fellow ] richard.brunner at amd com
-----Original Message----- From: JOSE LUIS RICARDO-CHAVEZ [mailto:ricardo@irsamc.ups-tlse.fr] Sent: Tuesday, February 24, 2004 4:33 AM To: Andreas Jaeger Cc: suse-amd64@suse.com Subject: Re: [suse-amd64] Kernel support for large arrays...
Andreas Jaeger wrote:
JOSE LUIS RICARDO-CHAVEZ <ricardo@irsamc.ups-tlse.fr> writes:
recently we have installed in our lab an Opteron system (Tyan 2882 motherboard, two Opteron 244 processor, 8 GB memory) running SuSE AMD64 9.0 Using the PGI compiler suite we did some memory allocation tests. We found a modest 4 GB memory limit per allocated array. According to PGI engineers this is not a compiler fault, the limit is just set in kernel's mmap.c routine. Would it be safe to change this value and recompile the kernel or it would require changing some other routines?
There's no such limit in the kernel. You should be allocate such an array without problems.
Andreas
Hi Andreas,
You are right, I looked into the file /usr/include/asm-x86-64/processor.h and found the following:
/* * User space process size: 512GB - 1GB (default). */ #define TASK_SIZE (0x0000007fc0000000)
/* This decides where the kernel will search for a free chunk of vm * space during mmap's. */ #define IA32_PAGE_OFFSET ((current->personality & ADDR_LIMIT_3GB) ? 0xc0000000 : 0xFFFFe000) #define TASK_UNMAPPED_32 (PAGE_ALIGN(IA32_PAGE_OFFSET / 3)) #define TASK_UNMAPPED_64 PAGE_ALIGN(TASK_SIZE/3) #define TASK_UNMAPPED_BASE \ (test_thread_flag(TIF_IA32) ? TASK_UNMAPPED_32 : TASK_UNMAPPED_64)
So the problem is not related to the way the OS maps physical memory into virtual memory. Could it be possible that the motherboard is tuned to support mainly a 32-bit system and by that reason it uses the last megabytes right before the 4 GB limit to allocate space for drivers? I've heard that Tyan is considering to provide a BIOS version designed to support only 64-bit operating systems, I hate to wait but maybe that's the solution.
Greeting from Toulose,
- Jose Luis
-- José Luis RICARDO-CHAVEZ Tel. +33-5-61-55-68-33 Laboratoire de Physique Quantique Fax +33-5-61-55-60-65 Université Paul Sabatier 118 route de Narbonne F-31062 Toulouse, France
e-mail: ricardo@irsamc.ups-tlse.fr