[rds-devel] Has anyone tried RDS on IA64 Linux?

Or Gerlitz ogerlitz at voltaire.com
Tue Apr 29 01:51:29 PDT 2008


Pradeep wrote:
> Another panic is seen when large buffers are used for RDMA(1MB):
>
> [root at hpatm125 ~]# rds-stress -r 10.0.0.125 -s 10.0.0.4 -p 4000 -t4 -d8 -D1048576 -T10
> Starting up..Kernel panic - not syncing: 
> arch/ia64/hp/common/sba_iommu.c: I/O MMU @ c0000000fed01000 is out of mapping resources
>
> [ didn't print a stack trace in this case ]
rds has a module param through which you can control how many FMRs (a 
mapping associated with each RDMA issued by rds) it would use at max, 
the default is 2K, as each FMR maps at most 256 pages, this would 
consume 512K entries at the HCA MMU (MTT) during setup time (when the 
fmr pool is allocated), but nothing at the I/O MMU at this point.

I/O MMU wise, assuming each RDMA is wrapped with dma_map/unmap on both 
sides, it depends how many transactions are inflight at your test. I 
think that -t4 -d8 means that you have at most 32 transactions running 
in parallel (Olaf, am I correct?) which will consume 8K (32*256) 
pages-entries at the MMU, is it too much for this system?

Or.





More information about the rds-devel mailing list