[rds-devel] Has anyone tried RDS on IA64 Linux?
Or Gerlitz
ogerlitz at voltaire.com
Tue Apr 29 01:51:29 PDT 2008
Pradeep wrote:
> Another panic is seen when large buffers are used for RDMA(1MB):
>
> [root at hpatm125 ~]# rds-stress -r 10.0.0.125 -s 10.0.0.4 -p 4000 -t4 -d8 -D1048576 -T10
> Starting up..Kernel panic - not syncing:
> arch/ia64/hp/common/sba_iommu.c: I/O MMU @ c0000000fed01000 is out of mapping resources
>
> [ didn't print a stack trace in this case ]
rds has a module param through which you can control how many FMRs (a
mapping associated with each RDMA issued by rds) it would use at max,
the default is 2K, as each FMR maps at most 256 pages, this would
consume 512K entries at the HCA MMU (MTT) during setup time (when the
fmr pool is allocated), but nothing at the I/O MMU at this point.
I/O MMU wise, assuming each RDMA is wrapped with dma_map/unmap on both
sides, it depends how many transactions are inflight at your test. I
think that -t4 -d8 means that you have at most 32 transactions running
in parallel (Olaf, am I correct?) which will consume 8K (32*256)
pages-entries at the MMU, is it too much for this system?
Or.
More information about the rds-devel
mailing list