[rds-devel] [PATCH bugfix] rds/ib: correlate max pages per fmr to the fmr page shift

[Or Gerlitz]

Correlate the maximum number of "pages" an fmr should map to page shift used with
this pool, make the value of the fmr_message size module param and the RDS_FMR_SIZE
constant value to reflect the semantics of being the maximum size of an RDMA transfer.

The current code assumes that the fmr page shift is always 12 which means that
without this patch on HCAs supporting smaller then 4K "page" (fmr chunks) sizes
this test in ib_rdma.c/ib_map_fmr() would fail

	if (page_cnt > fmr_message_size)
		return -EINVAL;

Signed-off-by: Or Gerlitz <ogerlitz at voltaire.com>

Index: ofa_1_3_kernel-20080113-0200/net/rds/ib.c
===================================================================
--- ofa_1_3_kernel-20080113-0200.orig/net/rds/ib.c
+++ ofa_1_3_kernel-20080113-0200/net/rds/ib.c
@@ -100,7 +100,7 @@ void rds_ib_add_one(struct ib_device *de
 	fmr_param.pool_size	    = fmr_pool_size;
 	fmr_param.dirty_watermark   = fmr_pool_size / 4;
 	fmr_param.cache		    = 0;
-	fmr_param.max_pages_per_fmr = fmr_message_size;
+	fmr_param.max_pages_per_fmr = fmr_message_size >> rds_ibdev->fmr_page_shift;
 	fmr_param.page_shift	    = rds_ibdev->fmr_page_shift;
 	fmr_param.access	    = (IB_ACCESS_LOCAL_WRITE |
 				       IB_ACCESS_REMOTE_READ |
Index: ofa_1_3_kernel-20080113-0200/net/rds/ib.h
===================================================================
--- ofa_1_3_kernel-20080113-0200.orig/net/rds/ib.h
+++ ofa_1_3_kernel-20080113-0200/net/rds/ib.h
@@ -8,7 +8,7 @@

 #define RDS_IB_RESOLVE_TIMEOUT_MS	5000

-#define RDS_FMR_SIZE			256
+#define RDS_FMR_SIZE			(1024 * 1024)
 #define RDS_FMR_POOL_SIZE		2048

 #define RDS_IB_MAX_SGE			29