[rds-devel] [PATCH] RDS: waking poll waiters

[Vladimir Sokolovsky]

Hi Zach,
Here is the waking poll waiters patch, updated following your comments:

RDS: rds_poll() always returns POLLOUT.  If you try to send and get
EWOULDBLOCK because the remote receiver is congested then you don't get
to wait for POLLOUT to be raised before sending again -- it's always
raised.  This is worked around by implementing some kind of exponential
back-off while retrying the send.  This can be done by increasing the
timeout given to poll().  At each poll() timeout expiry the send is
tried again.

This idea of waking poll() waiters on congestion notification hopes to
cut down on the latency between when the send could succeed and when the
poll() timeout hits and the send is retried.  If we receive a congestion
bitmap update from the remote node we wake poll() waiters, giving them a
chance to retry their send.

Signed-off-by: Vladimir Sokolovsky <vlad at mellanox.co.il>

diff --git a/net/rds/af_rds.c b/net/rds/af_rds.c
index 6e7d889..e8cdd2c 100644
--- a/net/rds/af_rds.c
+++ b/net/rds/af_rds.c
@@ -43,6 +43,7 @@
  static spinlock_t rds_sock_lock = SPIN_LOCK_UNLOCKED;
  unsigned long rds_sock_count;
  static LIST_HEAD(rds_sock_list);
+DECLARE_WAIT_QUEUE_HEAD(rds_poll_waitq);

  /*
   * This is called as the final descriptor referencing this socket is 
closed.
@@ -141,6 +142,8 @@ static unsigned int rds_poll(struct file *file, 
struct socket *sock,

         poll_wait(file, sk->sk_sleep, wait);

+       poll_wait(file, &rds_poll_waitq, wait);
+
         read_lock_irqsave(&sk->sk_callback_lock, flags);
         if (!list_empty(&rs->rs_recv_queue))
                 mask |= POLLIN;
diff --git a/net/rds/cong.c b/net/rds/cong.c
index 8e0109e..949d3a4 100644
--- a/net/rds/cong.c
+++ b/net/rds/cong.c
@@ -36,6 +36,8 @@

  #include "rds.h"

+extern wait_queue_head_t rds_poll_waitq;
+
  /*
   * This file implements the receive side of the unconventional congestion
   * management in RDS.
@@ -212,6 +214,8 @@ void rds_cong_map_updated(struct rds_cong_map *map)
         rds_stats_inc(s_cong_update_received);
         if (waitqueue_active(&map->m_waitq))
                 wake_up(&map->m_waitq);
+       if (waitqueue_active(&rds_poll_waitq))
+               wake_up_all(&rds_poll_waitq);
  }

  /*


Regards,
Vladimir