[rds-devel] Running rds-stress with large message size (bcopy-mode).
Olaf Kirch
olaf.kirch at oracle.com
Tue May 13 08:24:57 PDT 2008
On Friday 09 May 2008 17:53:23 Pradeep wrote:
>
> Hi Olaf,
>
> Is there any special settings I need to do to run rds-stress with 64K
> message-size
> (bcopy)? I set rmem_max and wmem_max to 8MB. Processes on both sides are
> stuck in
> poll syscall and aren't making any forward progress.
Could be you're hitting the congestion monitoring bug I fixed on May 7.
Try a later OFED snapshot. If that doesn't help, try running with
--use-cong-monitor=0
> Command: rds-stress -r 10.0.0.54 -s 10.0.0.53 -p 4000 -t1 -d4 -q65536
This works like a charm here...
Olaf
>
> [If the depth is changed to 1, it works fine]
>
> [root at iblp0054 ~]# rds-info
>
> TCP Connections:
> LocalAddr LPort RemoteAddr RPort HdrRemain DataRemain SentNx
> t ExpectUna SeenUna
>
> Counters:
> CounterName Value
> conn_reset 7
> recv_drop_bad_checksum 0
> recv_drop_old_seq 0
> recv_drop_no_sock 0
> recv_drop_dead_sock 0
> recv_deliver_raced 0
> recv_delivered 425966
> recv_queued 425974
> recv_immediate_retry 0
> recv_delayed_retry 0
> recv_ack_required 195692
> recv_rdma_bytes 0
> recv_ping 0
> send_queue_empty 91143
> send_queue_full 2635
> send_sem_contention 0
> send_sem_queue_raced 0
> send_immediate_retry 0
> send_delayed_retry 0
> send_drop_acked 0
> send_ack_required 101332
> send_rdma 0
> send_rdma_bytes 0
> send_pong 0
> page_remainder_hit 210074
> page_remainder_miss 2917
> cong_update_queued 2
> cong_update_received 34
> cong_send_error 2
> cong_send_blocked 0
> ib_connect_raced 1
> ib_listen_closed_stale 0
> ib_tx_cq_call 520946
> ib_tx_cq_event 545111
> ib_tx_ring_full 0
> ib_tx_sg_mapping_failure 0
> ib_tx_stalled 0
> ib_rx_cq_call 498235
> ib_rx_cq_event 1991023
> ib_rx_ring_empty 0
> ib_rx_refill_from_cq 498231
> ib_rx_refill_from_thread 2
> ib_rx_alloc_limit 0
> ib_ack_sent 119146
> ib_ack_send_failure 0
> ib_ack_send_delayed 733
> ib_ack_received 73904
> ib_rdma_mr_alloc 0
> ib_rdma_mr_free 0
> ib_rdma_mr_used 0
> ib_rdma_mr_pool_flush 2
> ib_rdma_mr_pool_wait 0
> tcp_data_ready_calls 0
> tcp_write_space_calls 0
> tcp_sndbuf_full 0
> tcp_connect_raced 0
> tcp_listen_closed_stale 0
>
> RDS Sockets:
> BoundAddr BPort ConnAddr CPort SndBuf RcvBuf
> 10.0.0.54 4001 0.0.0.0 0 524736 524736
> 0.0.0.0 0 0.0.0.0 0 55296 55296
>
> RDS Connections:
> LocalAddr RemoteAddr NextTX NextRX Flg
> 10.0.0.54 10.0.0.53 425963 425975 --C
>
> Receive Message Queue:
> LocalAddr LPort RemoteAddr RPort Seq Bytes
>
> Send Message Queue:
> LocalAddr LPort RemoteAddr RPort Seq Bytes
>
> Retransmit Message Queue:
> LocalAddr LPort RemoteAddr RPort Seq Bytes
>
> TCP Connections:
> LocalAddr LPort RemoteAddr RPort HdrRemain DataRemain
> SentNxt ExpectUna SeenUna
>
> Counters:
> CounterName Value
> conn_reset 7
> recv_drop_bad_checksum 0
> recv_drop_old_seq 0
> recv_drop_no_sock 0
> recv_drop_dead_sock 0
> recv_deliver_raced 0
> recv_delivered 425962
> recv_queued 425962
> recv_immediate_retry 0
> recv_delayed_retry 0
> recv_ack_required 101332
> recv_rdma_bytes 0
> recv_ping 0
> send_queue_empty 98851
> send_queue_full 2864
> send_sem_contention 0
> send_sem_queue_raced 0
> send_immediate_retry 0
> send_delayed_retry 0
> send_drop_acked 0
> send_ack_required 195692
> send_rdma 0
> send_rdma_bytes 0
> send_pong 0
> page_remainder_hit 210053
> page_remainder_miss 2917
> cong_update_queued 33
> cong_update_received 3
> cong_send_error 0
> cong_send_blocked 0
> ib_connect_raced 6
> ib_listen_closed_stale 0
> ib_tx_cq_call 474145
> ib_tx_cq_event 499912
> ib_tx_ring_full 0
> ib_tx_sg_mapping_failure 0
> ib_tx_stalled 0
> ib_rx_cq_call 543559
> ib_rx_cq_event 2035935
> ib_rx_ring_empty 0
> ib_rx_refill_from_cq 543559
> ib_rx_refill_from_thread 1
> ib_rx_alloc_limit 0
> ib_ack_sent 73904
> ib_ack_send_failure 0
> ib_ack_send_delayed 601
> ib_ack_received 119146
> ib_rdma_mr_alloc 0
> ib_rdma_mr_free 0
> ib_rdma_mr_used 0
> ib_rdma_mr_pool_flush 2
> ib_rdma_mr_pool_wait 0
> tcp_data_ready_calls 0
> tcp_write_space_calls 0
> tcp_sndbuf_full 0
> tcp_connect_raced 0
> tcp_listen_closed_stale 0
>
> RDS Sockets:
> BoundAddr BPort ConnAddr CPort SndBuf RcvBuf
> 10.0.0.53 4001 0.0.0.0 0 131071 131071
> 0.0.0.0 0 0.0.0.0 0 55296 55296
>
> RDS Connections:
> LocalAddr RemoteAddr NextTX NextRX Flg
> 10.0.0.53 10.0.0.54 425975 425963 --C
>
> Receive Message Queue:
> LocalAddr LPort RemoteAddr RPort Seq Bytes
>
> Send Message Queue:
> LocalAddr LPort RemoteAddr RPort Seq Bytes
>
> Retransmit Message Queue:
> LocalAddr LPort RemoteAddr RPort Seq Bytes
>
>
> _______________________________________________
> rds-devel mailing list
> rds-devel at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/rds-devel
>
--
Olaf Kirch | --- o --- Nous sommes du soleil we love when we play
okir at lst.de | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax
More information about the rds-devel
mailing list