[rds-devel] Running rds-stress with large message size (bcopy-mode).
Pradeep
pradeep at cup.hp.com
Fri May 9 08:53:23 PDT 2008
Hi Olaf,
Is there any special settings I need to do to run rds-stress with 64K
message-size
(bcopy)? I set rmem_max and wmem_max to 8MB. Processes on both sides are
stuck in
poll syscall and aren't making any forward progress.
Config: two x86-32 systems with OFED-1.3.1-20080507-0600 daily build.
Command: rds-stress -r 10.0.0.54 -s 10.0.0.53 -p 4000 -t1 -d4 -q65536
[If the depth is changed to 1, it works fine]
[root at iblp0054 ~]# rds-info
TCP Connections:
LocalAddr LPort RemoteAddr RPort HdrRemain DataRemain SentNx
t ExpectUna SeenUna
Counters:
CounterName Value
conn_reset 7
recv_drop_bad_checksum 0
recv_drop_old_seq 0
recv_drop_no_sock 0
recv_drop_dead_sock 0
recv_deliver_raced 0
recv_delivered 425966
recv_queued 425974
recv_immediate_retry 0
recv_delayed_retry 0
recv_ack_required 195692
recv_rdma_bytes 0
recv_ping 0
send_queue_empty 91143
send_queue_full 2635
send_sem_contention 0
send_sem_queue_raced 0
send_immediate_retry 0
send_delayed_retry 0
send_drop_acked 0
send_ack_required 101332
send_rdma 0
send_rdma_bytes 0
send_pong 0
page_remainder_hit 210074
page_remainder_miss 2917
cong_update_queued 2
cong_update_received 34
cong_send_error 2
cong_send_blocked 0
ib_connect_raced 1
ib_listen_closed_stale 0
ib_tx_cq_call 520946
ib_tx_cq_event 545111
ib_tx_ring_full 0
ib_tx_sg_mapping_failure 0
ib_tx_stalled 0
ib_rx_cq_call 498235
ib_rx_cq_event 1991023
ib_rx_ring_empty 0
ib_rx_refill_from_cq 498231
ib_rx_refill_from_thread 2
ib_rx_alloc_limit 0
ib_ack_sent 119146
ib_ack_send_failure 0
ib_ack_send_delayed 733
ib_ack_received 73904
ib_rdma_mr_alloc 0
ib_rdma_mr_free 0
ib_rdma_mr_used 0
ib_rdma_mr_pool_flush 2
ib_rdma_mr_pool_wait 0
tcp_data_ready_calls 0
tcp_write_space_calls 0
tcp_sndbuf_full 0
tcp_connect_raced 0
tcp_listen_closed_stale 0
RDS Sockets:
BoundAddr BPort ConnAddr CPort SndBuf RcvBuf
10.0.0.54 4001 0.0.0.0 0 524736 524736
0.0.0.0 0 0.0.0.0 0 55296 55296
RDS Connections:
LocalAddr RemoteAddr NextTX NextRX Flg
10.0.0.54 10.0.0.53 425963 425975 --C
Receive Message Queue:
LocalAddr LPort RemoteAddr RPort Seq Bytes
Send Message Queue:
LocalAddr LPort RemoteAddr RPort Seq Bytes
Retransmit Message Queue:
LocalAddr LPort RemoteAddr RPort Seq Bytes
TCP Connections:
LocalAddr LPort RemoteAddr RPort HdrRemain DataRemain
SentNxt ExpectUna SeenUna
Counters:
CounterName Value
conn_reset 7
recv_drop_bad_checksum 0
recv_drop_old_seq 0
recv_drop_no_sock 0
recv_drop_dead_sock 0
recv_deliver_raced 0
recv_delivered 425962
recv_queued 425962
recv_immediate_retry 0
recv_delayed_retry 0
recv_ack_required 101332
recv_rdma_bytes 0
recv_ping 0
send_queue_empty 98851
send_queue_full 2864
send_sem_contention 0
send_sem_queue_raced 0
send_immediate_retry 0
send_delayed_retry 0
send_drop_acked 0
send_ack_required 195692
send_rdma 0
send_rdma_bytes 0
send_pong 0
page_remainder_hit 210053
page_remainder_miss 2917
cong_update_queued 33
cong_update_received 3
cong_send_error 0
cong_send_blocked 0
ib_connect_raced 6
ib_listen_closed_stale 0
ib_tx_cq_call 474145
ib_tx_cq_event 499912
ib_tx_ring_full 0
ib_tx_sg_mapping_failure 0
ib_tx_stalled 0
ib_rx_cq_call 543559
ib_rx_cq_event 2035935
ib_rx_ring_empty 0
ib_rx_refill_from_cq 543559
ib_rx_refill_from_thread 1
ib_rx_alloc_limit 0
ib_ack_sent 73904
ib_ack_send_failure 0
ib_ack_send_delayed 601
ib_ack_received 119146
ib_rdma_mr_alloc 0
ib_rdma_mr_free 0
ib_rdma_mr_used 0
ib_rdma_mr_pool_flush 2
ib_rdma_mr_pool_wait 0
tcp_data_ready_calls 0
tcp_write_space_calls 0
tcp_sndbuf_full 0
tcp_connect_raced 0
tcp_listen_closed_stale 0
RDS Sockets:
BoundAddr BPort ConnAddr CPort SndBuf RcvBuf
10.0.0.53 4001 0.0.0.0 0 131071 131071
0.0.0.0 0 0.0.0.0 0 55296 55296
RDS Connections:
LocalAddr RemoteAddr NextTX NextRX Flg
10.0.0.53 10.0.0.54 425975 425963 --C
Receive Message Queue:
LocalAddr LPort RemoteAddr RPort Seq Bytes
Send Message Queue:
LocalAddr LPort RemoteAddr RPort Seq Bytes
Retransmit Message Queue:
LocalAddr LPort RemoteAddr RPort Seq Bytes
More information about the rds-devel
mailing list