| Message ID | 1675755374-107598-1-git-send-email-alibuda@linux.alibaba.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Deliver confirm/delete rkey message in parallel | expand |
On 07.02.23 08:36, D. Wythe wrote: > From: "D. Wythe" <alibuda@linux.alibaba.com> > > According to the SMC protocol specification, we know that all flows except > confirm_rkey adn delete_rkey are exclusive, confirm/delete rkey flows > can run concurrently (local and remote). > > However, although the protocol allows, all flows are actually mutually > exclusive in implementation, deus to we are waiting for LLC message > in serial. > > On the one hand, this implementation does not conform to the protocol > specification, on the other hand, this implementation aggravates the > time for establishing or destroying a SMC-R connection, connection > have to be queued in smc_llc_wait. > > This patch will improve the performance of the short link scenario > by about 5%. In fact, we all know that the performance bottleneck > of the short link scenario is not here. > > This patch try use rtokens or rkey to correlate a confirm/delete > rkey message with its response. > > This patch contains two parts. > > At first, we have added the process > of asynchronously waiting for the response of confirm/delete rkey > messages, using rtokens or rkey to be correlate with. > > And then, we try to send confirm/delete rkey message in parallel, > allowing parallel execution of start (remote) or initialization (local) > SMC_LLC_FLOW_RKEY flows. > > D. Wythe (2): > net/smc: allow confirm/delete rkey response deliver multiplex > net/smc: make SMC_LLC_FLOW_RKEY run concurrently > > net/smc/smc_core.h | 1 + > net/smc/smc_llc.c | 263 +++++++++++++++++++++++++++++++++++++++++------------ > net/smc/smc_llc.h | 6 ++ > net/smc/smc_wr.c | 10 -- > net/smc/smc_wr.h | 10 ++ > 5 files changed, 220 insertions(+), 70 deletions(-) > As we already discussed, on this changes we need to test them carefully so that we have to be sure that the communicating with z/OS should not be broken. We'll let you know as soon as the testing is finished.
On 2/8/23 7:29 AM, Wenjia Zhang wrote: > > > On 07.02.23 08:36, D. Wythe wrote: >> From: "D. Wythe" <alibuda@linux.alibaba.com> >> >> According to the SMC protocol specification, we know that all flows except >> confirm_rkey adn delete_rkey are exclusive, confirm/delete rkey flows >> can run concurrently (local and remote). >> >> However, although the protocol allows, all flows are actually mutually >> exclusive in implementation, deus to we are waiting for LLC message >> in serial. >> >> On the one hand, this implementation does not conform to the protocol >> specification, on the other hand, this implementation aggravates the >> time for establishing or destroying a SMC-R connection, connection >> have to be queued in smc_llc_wait. >> >> This patch will improve the performance of the short link scenario >> by about 5%. In fact, we all know that the performance bottleneck >> of the short link scenario is not here. >> >> This patch try use rtokens or rkey to correlate a confirm/delete >> rkey message with its response. >> >> This patch contains two parts. >> >> At first, we have added the process >> of asynchronously waiting for the response of confirm/delete rkey >> messages, using rtokens or rkey to be correlate with. >> >> And then, we try to send confirm/delete rkey message in parallel, >> allowing parallel execution of start (remote) or initialization (local) >> SMC_LLC_FLOW_RKEY flows. >> >> D. Wythe (2): >> net/smc: allow confirm/delete rkey response deliver multiplex >> net/smc: make SMC_LLC_FLOW_RKEY run concurrently >> >> net/smc/smc_core.h | 1 + >> net/smc/smc_llc.c | 263 +++++++++++++++++++++++++++++++++++++++++------------ >> net/smc/smc_llc.h | 6 ++ >> net/smc/smc_wr.c | 10 -- >> net/smc/smc_wr.h | 10 ++ >> 5 files changed, 220 insertions(+), 70 deletions(-) >> > > As we already discussed, on this changes we need to test them carefully so that we have to be sure that the communicating with z/OS should not be broken. We'll let you know as soon as the testing is finished. Hi, Wenjia Thanks again for your test. Considering that we have reached an agreement on protocol extension, we can temporarily postpone this modification until we introduce the protocol extension into the Linux community version. Then we can avoid the compatibility with z/OS. Best wishes. D. Wythe
On 2/8/23 11:04 AM, D. Wythe wrote: > > > On 2/8/23 7:29 AM, Wenjia Zhang wrote: >> >> >> On 07.02.23 08:36, D. Wythe wrote: >>> From: "D. Wythe" <alibuda@linux.alibaba.com> >>> >>> According to the SMC protocol specification, we know that all flows except >>> confirm_rkey adn delete_rkey are exclusive, confirm/delete rkey flows >>> can run concurrently (local and remote). >>> >>> However, although the protocol allows, all flows are actually mutually >>> exclusive in implementation, deus to we are waiting for LLC message >>> in serial. >>> >>> On the one hand, this implementation does not conform to the protocol >>> specification, on the other hand, this implementation aggravates the >>> time for establishing or destroying a SMC-R connection, connection >>> have to be queued in smc_llc_wait. >>> >>> This patch will improve the performance of the short link scenario >>> by about 5%. In fact, we all know that the performance bottleneck >>> of the short link scenario is not here. >>> >>> This patch try use rtokens or rkey to correlate a confirm/delete >>> rkey message with its response. >>> >>> This patch contains two parts. >>> >>> At first, we have added the process >>> of asynchronously waiting for the response of confirm/delete rkey >>> messages, using rtokens or rkey to be correlate with. >>> >>> And then, we try to send confirm/delete rkey message in parallel, >>> allowing parallel execution of start (remote) or initialization (local) >>> SMC_LLC_FLOW_RKEY flows. >>> >>> D. Wythe (2): >>> net/smc: allow confirm/delete rkey response deliver multiplex >>> net/smc: make SMC_LLC_FLOW_RKEY run concurrently >>> >>> net/smc/smc_core.h | 1 + >>> net/smc/smc_llc.c | 263 +++++++++++++++++++++++++++++++++++++++++------------ >>> net/smc/smc_llc.h | 6 ++ >>> net/smc/smc_wr.c | 10 -- >>> net/smc/smc_wr.h | 10 ++ >>> 5 files changed, 220 insertions(+), 70 deletions(-) >>> >> >> As we already discussed, on this changes we need to test them carefully so that we have to be sure that the communicating with z/OS should not be broken. We'll let you know as soon as the testing is finished. > > > Hi, Wenjia > > Thanks again for your test. > > Considering that we have reached an agreement on protocol extension, > we can temporarily postpone this modification until we introduce the protocol extension > into the Linux community version. Then we can avoid the compatibility with z/OS. > > > Best wishes. > D. Wythe > We can temporarily postpone this modification until we introduce the protocol extension into the Linux community version IF we can't pass the z/OS compatible test. :-) Sorry for the problem in my description. Thanks. D. Wythe
On 08.02.23 04:09, D. Wythe wrote: > > > On 2/8/23 11:04 AM, D. Wythe wrote: >> >> >> On 2/8/23 7:29 AM, Wenjia Zhang wrote: >>> >>> >>> On 07.02.23 08:36, D. Wythe wrote: >>>> From: "D. Wythe" <alibuda@linux.alibaba.com> >>>> >>>> According to the SMC protocol specification, we know that all flows >>>> except >>>> confirm_rkey adn delete_rkey are exclusive, confirm/delete rkey flows >>>> can run concurrently (local and remote). >>>> >>>> However, although the protocol allows, all flows are actually mutually >>>> exclusive in implementation, deus to we are waiting for LLC message >>>> in serial. >>>> >>>> On the one hand, this implementation does not conform to the protocol >>>> specification, on the other hand, this implementation aggravates the >>>> time for establishing or destroying a SMC-R connection, connection >>>> have to be queued in smc_llc_wait. >>>> >>>> This patch will improve the performance of the short link scenario >>>> by about 5%. In fact, we all know that the performance bottleneck >>>> of the short link scenario is not here. >>>> >>>> This patch try use rtokens or rkey to correlate a confirm/delete >>>> rkey message with its response. >>>> >>>> This patch contains two parts. >>>> >>>> At first, we have added the process >>>> of asynchronously waiting for the response of confirm/delete rkey >>>> messages, using rtokens or rkey to be correlate with. >>>> >>>> And then, we try to send confirm/delete rkey message in parallel, >>>> allowing parallel execution of start (remote) or initialization (local) >>>> SMC_LLC_FLOW_RKEY flows. >>>> >>>> D. Wythe (2): >>>> net/smc: allow confirm/delete rkey response deliver multiplex >>>> net/smc: make SMC_LLC_FLOW_RKEY run concurrently >>>> >>>> net/smc/smc_core.h | 1 + >>>> net/smc/smc_llc.c | 263 >>>> +++++++++++++++++++++++++++++++++++++++++------------ >>>> net/smc/smc_llc.h | 6 ++ >>>> net/smc/smc_wr.c | 10 -- >>>> net/smc/smc_wr.h | 10 ++ >>>> 5 files changed, 220 insertions(+), 70 deletions(-) >>>> >>> >>> As we already discussed, on this changes we need to test them >>> carefully so that we have to be sure that the communicating with z/OS >>> should not be broken. We'll let you know as soon as the testing is >>> finished. >> >> >> Hi, Wenjia >> >> Thanks again for your test. >> >> Considering that we have reached an agreement on protocol extension, >> we can temporarily postpone this modification until we introduce the >> protocol extension >> into the Linux community version. Then we can avoid the compatibility >> with z/OS. >> >> >> Best wishes. >> D. Wythe >> > > We can temporarily postpone this modification until we introduce the > protocol extension > into the Linux community version IF we can't pass the z/OS compatible > test. :-) > > Sorry for the problem in my description. > > Thanks. > D. Wythe > Sorry that it took a bit lang to test. But it looks good to me. Please let me know if you still want to postpone it. Best Wenjia
From: "D. Wythe" <alibuda@linux.alibaba.com> According to the SMC protocol specification, we know that all flows except confirm_rkey adn delete_rkey are exclusive, confirm/delete rkey flows can run concurrently (local and remote). However, although the protocol allows, all flows are actually mutually exclusive in implementation, deus to we are waiting for LLC message in serial. On the one hand, this implementation does not conform to the protocol specification, on the other hand, this implementation aggravates the time for establishing or destroying a SMC-R connection, connection have to be queued in smc_llc_wait. This patch will improve the performance of the short link scenario by about 5%. In fact, we all know that the performance bottleneck of the short link scenario is not here. This patch try use rtokens or rkey to correlate a confirm/delete rkey message with its response. This patch contains two parts. At first, we have added the process of asynchronously waiting for the response of confirm/delete rkey messages, using rtokens or rkey to be correlate with. And then, we try to send confirm/delete rkey message in parallel, allowing parallel execution of start (remote) or initialization (local) SMC_LLC_FLOW_RKEY flows. D. Wythe (2): net/smc: allow confirm/delete rkey response deliver multiplex net/smc: make SMC_LLC_FLOW_RKEY run concurrently net/smc/smc_core.h | 1 + net/smc/smc_llc.c | 263 +++++++++++++++++++++++++++++++++++++++++------------ net/smc/smc_llc.h | 6 ++ net/smc/smc_wr.c | 10 -- net/smc/smc_wr.h | 10 ++ 5 files changed, 220 insertions(+), 70 deletions(-)