Mailinglist Archive: opensuse-amd64 (380 mails)

[JOSE LUIS RICARDO-CHAVEZ <ricardo@xxxxxxxxxxxxxxxxxx>]

richard.brunner@xxxxxxx wrote:

> Physical memory available != virtual
> memory available for allocation. In an 8GB DRAM system, the BIOS gives
> the OS approximately 7.5GB (because of various I/O issues).
>
> You should be able to get the kernel to allocate a large amount
> but once you allocate larger than phys memory, you will page awfully.
>
> You don't say whether the app requesting the memory is 32-bit or 64-bit.
> I assume 64-bit.
>
> Which PGI compiler (F77? F90?) and which version (5.0? 5.1? 5.1-3?)
> are you using? How are you allocating the array?
>
> ] -Rich ...
> ] AMD Fellow
> ] richard.brunner at amd com 
>
>
>  
>
> > -----Original Message-----
> > From: JOSE LUIS RICARDO-CHAVEZ [mailto:ricardo@xxxxxxxxxxxxxxxxxx] 
> > Sent: Tuesday, February 24, 2004 4:33 AM
> > To: Andreas Jaeger
> > Cc: suse-amd64@xxxxxxxx
> > Subject: Re: [suse-amd64] Kernel support for large arrays...
> >
> >
> > Andreas Jaeger wrote:
> >
> >    
> >
> > > JOSE LUIS RICARDO-CHAVEZ <ricardo@xxxxxxxxxxxxxxxxxx> writes:
> > >
> > >
> > >
> > >      
> > >
> > > > recently we have installed in our lab an Opteron system (Tyan 2882 
> > > > motherboard, two Opteron 244 processor, 8 GB memory) running SuSE 
> > > > AMD64 9.0 Using the PGI compiler suite we did some memory 
> > > >        
> > allocation 
> >    
> >
> > > > tests. We found a modest 4 GB
> > > > memory limit per allocated array. According to PGI engineers this is
> > > > not a compiler
> > > > fault, the limit is just set in kernel's mmap.c routine. 
> > > >        
> > Would it be safe to
> >    
> >
> > > > change this value and recompile the kernel or it would require
> > > > changing some other
> > > > routines?
> > > >   
> > > >
> > > >        
> > > There's no such limit in the kernel.  You should be allocate such an 
> > > array without problems.
> > >
> > > Andreas
> > >
> > >
> > >      
> > Hi Andreas,
> >
> > You are right, I looked into the file 
> > /usr/include/asm-x86-64/processor.h
> > and found the following:
> >
> > /*
> > * User space process size: 512GB - 1GB (default).
> > */
> > #define TASK_SIZE    (0x0000007fc0000000)
> >
> > /* This decides where the kernel will search for a free chunk of vm
> > * space during mmap's.
> > */
> > #define IA32_PAGE_OFFSET ((current->personality & ADDR_LIMIT_3GB) ? 
> > 0xc0000000 : 0xFFFFe000)
> > #define TASK_UNMAPPED_32 (PAGE_ALIGN(IA32_PAGE_OFFSET / 3)) 
> > #define TASK_UNMAPPED_64 PAGE_ALIGN(TASK_SIZE/3)
> > #define TASK_UNMAPPED_BASE    \
> >    (test_thread_flag(TIF_IA32) ? TASK_UNMAPPED_32 : 
> > TASK_UNMAPPED_64) 
> >
> > So the problem is not related to the way the OS maps
> > physical memory into virtual memory.
> > Could it be possible that the motherboard is tuned to support 
> > mainly a 32-bit system and by that reason it uses the last 
> > megabytes right before the 4 GB limit to allocate space for 
> > drivers? I've heard that Tyan is considering to provide a 
> > BIOS version designed to support only 64-bit operating 
> > systems, I hate to wait but maybe that's the solution.
> >
> > Greeting from Toulose,
> >
> > - Jose Luis
> >
> > --
> > José Luis RICARDO-CHAVEZ               Tel. +33-5-61-55-68-33 
> > Laboratoire de Physique Quantique      Fax  +33-5-61-55-60-65
> > Université Paul Sabatier                      
> > 118 route de Narbonne
> > F-31062 Toulouse, France
> >
> > e-mail: ricardo@xxxxxxxxxxxxxxxxxx
> >
> >
> >
> >    
>
>
>  
Hi Rich,

The memory allocation test is done with a simple F90 program:

PROGRAM bigalloc
IMPLICIT NONE
INTEGER :: i
INTEGER, PARAMETER :: SIZE = 800 * 1000 * 1000
REAL (KIND=8), ALLOCATABLE :: a(:)
PRINT *,'Allocating big array...'
ALLOCATE ( a ( SIZE ) )
! Array initialization.
a = 0.0d0
! Do something.
DO i = 1, 10
  a = DBLE(i)
END DO
PRINT *,'Done.'
PRINT *,'Freeing memory...'
IF ( ALLOCATED ( a ) ) DEALLOCATE ( a )
PRINT *,'Done.'
END PROGRAM bigalloc

The system has a Tyan S2882 motherboard with two 244 processors and 8 GB 
memory,
it runs kernel version 2.4.21-193-smp (SuSE 9.0 + SiI SATA drivers). The 
compiler
version is pgf90 5.1-2 (linux86-64) and the F90FLAGS are " -i8 -O -tp 
k8-64".
The kernel message is:
bigalloc[6564]: segfault at 0000002b1318e000 rip 0000000000401b40 rsp 
0000007fbffff560 error 6

Regards,

- Jose Luis

--
José Luis RICARDO-CHAVEZ               Tel. +33-5-61-55-68-33 
Laboratoire de Physique Quantique      Fax  +33-5-61-55-60-65
Université Paul Sabatier                      
118 route de Narbonne
F-31062 Toulouse, France

e-mail: ricardo@xxxxxxxxxxxxxxxxxx