| Message ID | CAGE_+C4JWA0DrcV4rT7J6hXsbYPL2Po31wFPvLESe_sKq_16FQ@mail.gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [v2,1/5] remote-curl: avoid hang if curl asks for more data after eof | expand |
Jiří Hruška <jirka@fud.cz> writes: > It has been observed that under some circumstances, libcurl can call > our `CURLOPT_READFUNCTION` callback `rpc_out()` again even after > already getting a return value of zero (EOF) back once before. > > Because `rpc->flush_read_but_not_sent` is reset to false immediately > the first time an EOF is returned, the repeated call goes again to > `rpc_read_from_out()`, which tries to read more from the child process > pipe and the whole operation gets stuck - the child process is already > trying to read a response back and will not write anything to the > output pipe anymore, while the parent/remote process is now blocked > waiting to read more too and never even finishes sending the request. > > The bug is fixed by moving the reset of the `flush_read_but_not_sent` > flag to `post_rpc()`, only before `rpc_out()` would be potentially used > the next time. This makes the callback behave like fread() and return > a zero any number of times at the end of a finished upload. > > Signed-off-by: Jiri Hruska <jirka@fud.cz> Thanks for splitting this out - this makes it much easier to review. I can see that the patch indeed works, but some things need to be clearer (described below). In the commit message, I think we should add a historical note, something like: `flush_read_but_not_sent` was introduced in a97d00799a (remote-curl: use post_rpc() for protocol v2 also, 2019-02-21), which needed a way to indicate that Git should not read from a certain child process anymore once "flush" has been received from the child process. This field was scoped to what was believed the minimum necessary - the interval between "flush" being received and EOF being reported to Curl. However, as described at the beginning of the commit message, we need the scope of this to be greater than that - in case Curl continues to ask us for more data, we still need to remember that "flush" has been received. Therefore, change this field from `flush_read_but_not_sent` to `flush_read`, which both is simpler and is a fix for this bug. Feel free to use this verbatim, or adapt it as you wish (or write your own). As described in the historical note, I also think we need to change the name of this field, since we are changing its semantics. I am trying to be better at indicating when I think a prefix subset of a patch set is worth merging on its own (so that, if some patches in a patch set are good and some still need to be discussed, we have the option of merging a prefix of a patch set). So, assuming my comments are addressed, I think this patch is good to go in on its own, even if we don't want some of the later patches. I'll review the rest of this patch set later.
diff --git a/remote-curl.c b/remote-curl.c index ef05752ca5..199c4615a5 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -705,9 +705,10 @@ static size_t rpc_out(void *ptr, size_t eltsize, * The line length either does not need to be sent at * all or has already been completely sent. Now we can * return 0, indicating EOF, meaning that the flush has - * been fully sent. + * been fully sent. It is important to keep returning 0 + * as long as needed in that case, as libcurl invokes + * the callback multiple times at EOF sometimes. */ - rpc->flush_read_but_not_sent = 0; return 0; } /* @@ -963,6 +964,7 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece */ headers = curl_slist_append(headers, "Transfer-Encoding: chunked"); rpc->initial_buffer = 1; + rpc->flush_read_but_not_sent = 0; curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, rpc_out); curl_easy_setopt(slot->curl, CURLOPT_INFILE, rpc);
It has been observed that under some circumstances, libcurl can call our `CURLOPT_READFUNCTION` callback `rpc_out()` again even after already getting a return value of zero (EOF) back once before. Because `rpc->flush_read_but_not_sent` is reset to false immediately the first time an EOF is returned, the repeated call goes again to `rpc_read_from_out()`, which tries to read more from the child process pipe and the whole operation gets stuck - the child process is already trying to read a response back and will not write anything to the output pipe anymore, while the parent/remote process is now blocked waiting to read more too and never even finishes sending the request. The bug is fixed by moving the reset of the `flush_read_but_not_sent` flag to `post_rpc()`, only before `rpc_out()` would be potentially used the next time. This makes the callback behave like fread() and return a zero any number of times at the end of a finished upload. Signed-off-by: Jiri Hruska <jirka@fud.cz> --- remote-curl.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) curl_easy_setopt(slot->curl, CURLOPT_SEEKFUNCTION, rpc_seek);