| Message ID | patch-v2-6.6-686febb8cdc-20230202T093706Z-avarab@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | imap-send: replace auto-probe libcurl with hard dependency | expand |
Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes: > Before [1] we'd force the "imap.host" to be set, even if the > "imap.tunnel" was set, and then proceed to not use the "host" for > establishing a connection, as we'd use the tunneling command. > > However, we'd still use the "imap.host" if it was set as the "host" > field given to the credential helper, and in messages that were shared > with the non-tunnel mode, until a preceding commit made these OpenSSL > codepaths tunnel-only. > > Let's always give "host=tunnel" to the credential helper when in the > "imap.tunnel" mode, and rephrase the relevant messages to indicate > that we're tunneling. This changes the existing behavior, but that > behavior was emergent and didn't make much sense. If we were using > "imap.tunnel" the value in "imap.host" might be entirely unrelated to > the host we're tunneling to. Let's not pretend to know more than we do > in that case. > > 1. 34b5cd1fe9f (Don't force imap.host to be set when imap.tunnel is > set, 2008-04-22) > > Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> > --- I agree with the above flow of thought in principle, but I wonder if "tunnel" is distinct enough to allow credential helpers to tell that they are dealing with a "tunnel", and not a host whose name happens to be "tunnel". Would it help to use a token that can never be a valid hostname instead, I wonder? > @@ -1004,7 +1004,7 @@ static struct imap_store *imap_open_store(struct imap_server_conf *srvc, const c > if (!CAP(AUTH_CRAM_MD5)) { > fprintf(stderr, "You specified " > "CRAM-MD5 as authentication method, " > - "but %s doesn't support it.\n", srvc->host); > + "but tunnel doesn't support it.\n"); Do we need some article before "tunnel"? > if (CAP(NOLOGIN)) { > - fprintf(stderr, "Skipping account %s@%s, server forbids LOGIN\n", > - srvc->user, srvc->host); > + fprintf(stderr, "Skipping account %s, server forbids LOGIN\n", > + srvc->user); > goto bail; OK. We are talking to whatever "tunnel" is that was spawned to talk somewhere we do not have a way to know, so trying to say <this user> at <that host> is futile. Makes sense. > - if (!server.host) { > - if (!server.tunnel) { > - fprintf(stderr, "no imap host specified\n"); > - return 1; > - } > - server.host = "tunnel"; > + if (!server.host && !server.tunnel) { > + fprintf(stderr, "no imap host specified\n"); > + return 1; > } OK, this is a natural consequence that we no longer abuse server.host in the tunneling case. Makes sense. Thanks. Will queue.
On Thu, Feb 02, 2023 at 10:44:17AM +0100, Ævar Arnfjörð Bjarmason wrote: > Before [1] we'd force the "imap.host" to be set, even if the > "imap.tunnel" was set, and then proceed to not use the "host" for > establishing a connection, as we'd use the tunneling command. > > However, we'd still use the "imap.host" if it was set as the "host" > field given to the credential helper, and in messages that were shared > with the non-tunnel mode, until a preceding commit made these OpenSSL > codepaths tunnel-only. > > Let's always give "host=tunnel" to the credential helper when in the > "imap.tunnel" mode, and rephrase the relevant messages to indicate > that we're tunneling. This changes the existing behavior, but that > behavior was emergent and didn't make much sense. If we were using > "imap.tunnel" the value in "imap.host" might be entirely unrelated to > the host we're tunneling to. Let's not pretend to know more than we do > in that case. If you tunnel to two different hosts, how is the credential system supposed to know which is which? If you really want to distinguish connecting to $host versus tunneling to $host, I think you'd have to invent some new URL scheme (imap-tunnel:// or something). But IMHO it is not really worth it. Your statement of "the value in imap.host might be entirely unrelated" does not match my experience. I don't use imap-send, but I've been doing imap-tunneling with various programs for two decades, and it's pretty normal to configure both, and to consider the tunnel command as an implementation detail for getting to the host. For example, my mutt config is like[1]: set folder = imap://example.com/ set tunnel = "ssh example.com /etc/rimapd" and I expect to be able to refer to folders as imap://example.com/foo, etc (well, in mutt you'd use the shorthand "=foo", but the idea is the same). So if we see: [imap] host = example.com tunnel = ssh example.com /etc/rimapd we should likewise think of it as example.com, but with an implementation detail of how to contact the server. Of course if you don't set imap.host, then we don't have anything useful to say. But as you saw, in that case imap-send will default the host field to the word "tunnel". -Peff [1] In my experience the main reason to tunnel is to avoid auth altogether, so for those cases the credential code wouldn't matter either way. But I imagine there may be some people who use it to pierce a firewall or some other network obstacle, and really do want it to be otherwise just like a connection to $host.
On Fri, Feb 03 2023, Jeff King wrote: > On Thu, Feb 02, 2023 at 10:44:17AM +0100, Ævar Arnfjörð Bjarmason wrote: > >> Before [1] we'd force the "imap.host" to be set, even if the >> "imap.tunnel" was set, and then proceed to not use the "host" for >> establishing a connection, as we'd use the tunneling command. >> >> However, we'd still use the "imap.host" if it was set as the "host" >> field given to the credential helper, and in messages that were shared >> with the non-tunnel mode, until a preceding commit made these OpenSSL >> codepaths tunnel-only. >> >> Let's always give "host=tunnel" to the credential helper when in the >> "imap.tunnel" mode, and rephrase the relevant messages to indicate >> that we're tunneling. This changes the existing behavior, but that >> behavior was emergent and didn't make much sense. If we were using >> "imap.tunnel" the value in "imap.host" might be entirely unrelated to >> the host we're tunneling to. Let's not pretend to know more than we do >> in that case. > > If you tunnel to two different hosts, how is the credential system > supposed to know which is which? > > If you really want to distinguish connecting to $host versus tunneling > to $host, I think you'd have to invent some new URL scheme > (imap-tunnel:// or something). > > But IMHO it is not really worth it. Your statement of "the value in > imap.host might be entirely unrelated" does not match my experience. I > don't use imap-send, but I've been doing imap-tunneling with various > programs for two decades, and it's pretty normal to configure both, and > to consider the tunnel command as an implementation detail for getting > to the host. For example, my mutt config is like[1]: > > set folder = imap://example.com/ > set tunnel = "ssh example.com /etc/rimapd" > > and I expect to be able to refer to folders as imap://example.com/foo, > etc (well, in mutt you'd use the shorthand "=foo", but the idea is the > same). So if we see: > > [imap] > host = example.com > tunnel = ssh example.com /etc/rimapd > > we should likewise think of it as example.com, but with an > implementation detail of how to contact the server. Except that mutt config is different than the imap-send case in that it would presumably break if you changed: set folder = imap://example.com/ set tunnel = "ssh example.com /etc/rimapd" So that one of the two was example.org instead of example.com (or whatever). I agree that "give this to the auth helper" might be useful in general, but our current documentation says: To use the tool, `imap.folder` and either `imap.tunnel` or `imap.host` must be set to appropriate values. And the docs for "imap.tunnel" say "Required when imap.host is not set", and "imap.host" says "Ignored when imap.tunnel is set, but required otherwise". Perhaps we should bless this as an accidental feature instead of my proposed patch, but that's why I made this change. It seemed like an unintentional bug that nobody intended. Especially as you're focusing on the case where it contrary to the docs would do what you mean, but consider (same as the doc examples, but the domains are changed): [imap] folder = "INBOX.Drafts" host = imap://imap.bar.com user = bob pass = p4ssw0rd [imap] folder = "INBOX.Drafts" tunnel = "ssh -q -C user@foo.com /usr/bin/imapd ./Maildir 2> /dev/null" I.e. I have a config for "bar.com" I tried earlier, but now I'm trying to connect to "foo.com", because I read the docs and notice it prefers "tunnel" to "host" I think it's going to ignore that "imap.host", but it's going to provide the password for bar.com to foo.com if challenged. So I think if we want to keep this it would be better to have a imap.tunnel.credentialHost or something, to avoid conflating the two. But I think it's okey to just remove this until someone has this explicit use-case. I doubt that we have any users relying on this, as it's not only undocumented, but the documentation explicitly states that it doesn't work like this. > Of course if you don't set imap.host, then we don't have anything useful > to say. But as you saw, in that case imap-send will default the host > field to the word "tunnel". Isn't that more of a suggestion that nobody cares about this? Presumably if we had users trying to get this to work someone would have complained that they wanted a custom string rather than "tunnel", as the auth helper isn't very helpful in that case...
On Fri, Feb 03, 2023 at 10:12:27PM +0100, Ævar Arnfjörð Bjarmason wrote: > Except that mutt config is different than the imap-send case in that it > would presumably break if you changed: > > set folder = imap://example.com/ > set tunnel = "ssh example.com /etc/rimapd" > > So that one of the two was example.org instead of example.com (or > whatever). No, it would be perfectly happy (and in fact they do not exactly match in my config). When you have a tunnel, that is the authoritative way to get to the host, and the hostname mostly becomes a label. But importantly, it is still used for certain things like local cache keys, which would be shared with a non-tunnel connection to the same name. Recent versions of mutt do also use the hostname for TLS verification, if your tunnel is not itself secure (though this is not the default; you have to set special options to tell it so). > I agree that "give this to the auth helper" might be useful in general, > but our current documentation says: > > To use the tool, `imap.folder` and either `imap.tunnel` or `imap.host` must be set > to appropriate values. > > And the docs for "imap.tunnel" say "Required when imap.host is not set", > and "imap.host" says "Ignored when imap.tunnel is set, but required > otherwise". > > Perhaps we should bless this as an accidental feature instead of my > proposed patch, but that's why I made this change. It seemed like an > unintentional bug that nobody intended. Yes, I agree that the scenario I'm giving is contrary to what the docs say. But IMHO it is worth preferring what the code does now versus what the docs say. The current behavior misbehaves if you configure things badly (accidentally mix and match imap.host and imap.tunnel). Your new behavior misbehaves if you have two correctly-configure imap stanzas (both with a host/tunnel combo). Those are both fairly unlikely scenarios, and the outcomes are similar (we mix up credentials), but: 1. In general, all things being equal, I'd rather trust the code as the status quo. People will complain if you break their working setup. They won't if you fix the documentation. 2. In the current behavior, if it's doing the wrong thing, your next step is to fix your configuration (don't mix and match imap.host and imap.tunnel). In your proposed behavior, there is no fix. You are simply not allowed to use two different imap tunnels with credential helpers, because the helpers don't receive enough context to distinguish them. And that is not even "two imap tunnels in the same config". It is really per user. If I have two repositories, each with "imap.tunnel" in their local config, they will still invoke the same credential helpers, and both will just see host=tunnel. The namespace for "host" really is global and should be unique (ideally across the Internet, but at the very least among the hosts that the user contacts). One fix would be to pass the tunnel command as the hostname. But even that has potential problems. Certainly you'd have to tweak it (it's a shell command, and hostnames have syntactic restrictions, including forbidding newlines). And you'd probably want to swap out protocol=imap for something else. But I'm not sure if helpers may complain (e.g., I seem to recall that the osxkeychain helper translates protocol strings into integer constants that the keychain API understands). > Especially as you're focusing on the case where it contrary to the docs > would do what you mean, but consider (same as the doc examples, but the > domains are changed): > > [imap] > folder = "INBOX.Drafts" > host = imap://imap.bar.com > user = bob > pass = p4ssw0rd > > [imap] > folder = "INBOX.Drafts" > tunnel = "ssh -q -C user@foo.com /usr/bin/imapd ./Maildir 2> /dev/null" > > I.e. I have a config for "bar.com" I tried earlier, but now I'm trying > to connect to "foo.com", because I read the docs and notice it prefers > "tunnel" to "host" I think it's going to ignore that "imap.host", but > it's going to provide the password for bar.com to foo.com if challenged. Yes, that's a rather unfortunate effect of the way we do config parsing (it looks to the user like stanzas, but we don't parse it that way; the second stanza could even be in a different file!). Though as I said above, I still think this case does not justify making the code change, I do think it's the most compelling argument, and would make sense to include in the commit message if we did want to do this. > So I think if we want to keep this it would be better to have a > imap.tunnel.credentialHost or something, to avoid conflating the two. Yes, there are many config schemes that would avoid this problem. If you are going to tie the two together, I think it would make sense to use real subsections based on the host-name, like: # hostname is the subsection key; it also becomes a label when # necessary [imap "example.com"] # does not even need to mention a hostname. We'll assume example.com # here. tunnel = "any-command" # assumes example.com as hostname; not needed if you are using a # tunnel, of course protocol = imaps But I would not bother going to that work myself. IMHO imap-send is somewhat of an abomination, and I'd actually be just as happy if it went away. But what you are doing seems to go totally in the wrong direction to me (keeping it, but breaking a rare but working use case to the benefit of a rare but broken misconfiguration). > > Of course if you don't set imap.host, then we don't have anything useful > > to say. But as you saw, in that case imap-send will default the host > > field to the word "tunnel". > > Isn't that more of a suggestion that nobody cares about this? Presumably > if we had users trying to get this to work someone would have complained > that they wanted a custom string rather than "tunnel", as the auth > helper isn't very helpful in that case... Not if they did: [imap] host = example.com tunnel = some-command -Peff
On Sat, Feb 04 2023, Jeff King wrote: > On Fri, Feb 03, 2023 at 10:12:27PM +0100, Ævar Arnfjörð Bjarmason wrote: > [...] >> I agree that "give this to the auth helper" might be useful in general, >> but our current documentation says: >> >> To use the tool, `imap.folder` and either `imap.tunnel` or `imap.host` must be set >> to appropriate values. >> >> And the docs for "imap.tunnel" say "Required when imap.host is not set", >> and "imap.host" says "Ignored when imap.tunnel is set, but required >> otherwise". >> >> Perhaps we should bless this as an accidental feature instead of my >> proposed patch, but that's why I made this change. It seemed like an >> unintentional bug that nobody intended. > > Yes, I agree that the scenario I'm giving is contrary to what the docs > say. But IMHO it is worth preferring what the code does now versus what > the docs say. The current behavior misbehaves if you configure things > badly (accidentally mix and match imap.host and imap.tunnel). Your new > behavior misbehaves if you have two correctly-configure imap stanzas > (both with a host/tunnel combo). Those are both fairly unlikely > scenarios, and the outcomes are similar (we mix up credentials), but: > > 1. In general, all things being equal, I'd rather trust the code as > the status quo. People will complain if you break their working > setup. They won't if you fix the documentation. > > 2. In the current behavior, if it's doing the wrong thing, your next > step is to fix your configuration (don't mix and match imap.host > and imap.tunnel). In your proposed behavior, there is no fix. You > are simply not allowed to use two different imap tunnels with > credential helpers, because the helpers don't receive enough > context to distinguish them. > > And that is not even "two imap tunnels in the same config". It is > really per user. If I have two repositories, each with > "imap.tunnel" in their local config, they will still invoke the > same credential helpers, and both will just see host=tunnel. The > namespace for "host" really is global and should be unique (ideally > across the Internet, but at the very least among the hosts that the > user contacts). > > One fix would be to pass the tunnel command as the hostname. But even > that has potential problems. Certainly you'd have to tweak it (it's a > shell command, and hostnames have syntactic restrictions, including > forbidding newlines). And you'd probably want to swap out protocol=imap > for something else. But I'm not sure if helpers may complain (e.g., I > seem to recall that the osxkeychain helper translates protocol strings > into integer constants that the keychain API understands). > >> Especially as you're focusing on the case where it contrary to the docs >> would do what you mean, but consider (same as the doc examples, but the >> domains are changed): >> >> [imap] >> folder = "INBOX.Drafts" >> host = imap://imap.bar.com >> user = bob >> pass = p4ssw0rd >> >> [imap] >> folder = "INBOX.Drafts" >> tunnel = "ssh -q -C user@foo.com /usr/bin/imapd ./Maildir 2> /dev/null" >> >> I.e. I have a config for "bar.com" I tried earlier, but now I'm trying >> to connect to "foo.com", because I read the docs and notice it prefers >> "tunnel" to "host" I think it's going to ignore that "imap.host", but >> it's going to provide the password for bar.com to foo.com if challenged. > > Yes, that's a rather unfortunate effect of the way we do config parsing > (it looks to the user like stanzas, but we don't parse it that way; the > second stanza could even be in a different file!). > > Though as I said above, I still think this case does not justify making > the code change, I do think it's the most compelling argument, and would > make sense to include in the commit message if we did want to do this. > >> So I think if we want to keep this it would be better to have a >> imap.tunnel.credentialHost or something, to avoid conflating the two. > > Yes, there are many config schemes that would avoid this problem. If you > are going to tie the two together, I think it would make sense to use > real subsections based on the host-name, like: > > # hostname is the subsection key; it also becomes a label when > # necessary > [imap "example.com"] > > # does not even need to mention a hostname. We'll assume example.com > # here. > tunnel = "any-command" > > # assumes example.com as hostname; not needed if you are using a > # tunnel, of course > protocol = imaps > > But I would not bother going to that work myself. IMHO imap-send is > somewhat of an abomination, and I'd actually be just as happy if it went > away. But what you are doing seems to go totally in the wrong direction > to me (keeping it, but breaking a rare but working use case to the > benefit of a rare but broken misconfiguration). > >> > Of course if you don't set imap.host, then we don't have anything useful >> > to say. But as you saw, in that case imap-send will default the host >> > field to the word "tunnel". >> >> Isn't that more of a suggestion that nobody cares about this? Presumably >> if we had users trying to get this to work someone would have complained >> that they wanted a custom string rather than "tunnel", as the auth >> helper isn't very helpful in that case... > > Not if they did: > > [imap] > host = example.com > tunnel = some-command Yes, but how would they have ended up doing that? By discarding the documentation and throwing things at the wall & hoping they'd stick? I take all your general points above, and generally agree with them. Re #1 we should generally prefer current behavior over the docs, and re #2: Yes, I agree this might be useful in princple, and hardcoding "host=tunnel" doesn't leave a way to pass a custom "host to the auth helper. I also don't care enough to argue about it, so I'll leave the first hunk here out of any re-roll. We'll continue to pass "host" down in that case, i.e. I'll only adjust the error messages. I just don't get how anyone could have come to rely on this so that we'd care about supporting it. Because mutt has a feature that looks similar, users might have configured git-imap-send thinking it might do the same thing, and gotten lucky? I guess in principle that could be true, but I think it's more likely that nobody's ever had reason to use it that way. I.e. if you use the "tunnel" the way the docs suggest you won't hit the credential helper, as you're authenticating with "ssh", and using "imapd" to directly operate on a Maildir path.
Jeff King <peff@peff.net> writes: > Yes, I agree that the scenario I'm giving is contrary to what the docs > say. But IMHO it is worth preferring what the code does now versus what > the docs say. The current behavior misbehaves if you configure things > badly (accidentally mix and match imap.host and imap.tunnel). Your new > behavior misbehaves if you have two correctly-configure imap stanzas > (both with a host/tunnel combo). Those are both fairly unlikely > scenarios, and the outcomes are similar (we mix up credentials), but: > > 1. In general, all things being equal, I'd rather trust the code as > the status quo. People will complain if you break their working > setup. They won't if you fix the documentation. > > 2. In the current behavior, if it's doing the wrong thing, your next > step is to fix your configuration (don't mix and match imap.host > and imap.tunnel). In your proposed behavior, there is no fix. You > are simply not allowed to use two different imap tunnels with > credential helpers, because the helpers don't receive enough > context to distinguish them. > > And that is not even "two imap tunnels in the same config". It is > really per user. If I have two repositories, each with > "imap.tunnel" in their local config, they will still invoke the > same credential helpers, and both will just see host=tunnel. The > namespace for "host" really is global and should be unique (ideally > across the Internet, but at the very least among the hosts that the > user contacts). All good points. > Yes, there are many config schemes that would avoid this problem. If you > are going to tie the two together, I think it would make sense to use > real subsections based on the host-name, like: > > # hostname is the subsection key; it also becomes a label when > # necessary > [imap "example.com"] > > # does not even need to mention a hostname. We'll assume example.com > # here. > tunnel = "any-command" > > # assumes example.com as hostname; not needed if you are using a > # tunnel, of course > protocol = imaps > > But I would not bother going to that work myself. IMHO imap-send is > somewhat of an abomination, and I'd actually be just as happy if it went > away. But what you are doing seems to go totally in the wrong direction > to me (keeping it, but breaking a rare but working use case to the > benefit of a rare but broken misconfiguration). Yup.
On Sun, Feb 05, 2023 at 10:51:04PM +0100, Ævar Arnfjörð Bjarmason wrote: > > Not if they did: > > > > [imap] > > host = example.com > > tunnel = some-command > > Yes, but how would they have ended up doing that? By discarding the > documentation and throwing things at the wall & hoping they'd stick? That's what I would have tried without reading the documentation at all, based on using other programs that tunnel imap. I'm just one data point, of course. > I just don't get how anyone could have come to rely on this so that we'd > care about supporting it. > > Because mutt has a feature that looks similar, users might have > configured git-imap-send thinking it might do the same thing, and gotten > lucky? It's less "mutt happens to do it this way" and more "associating a host is strictly more useful, because it lets you interact with all the other host-like features". It's only imap-send's funky config scheme that makes it easy to mis-configure. > I guess in principle that could be true, but I think it's more likely > that nobody's ever had reason to use it that way. I.e. if you use the > "tunnel" the way the docs suggest you won't hit the credential helper, > as you're authenticating with "ssh", and using "imapd" to directly > operate on a Maildir path. As I said, my main use of tunneling is to trigger the imap server's preauth mode. But there are other reasons one might want to do so, like piercing a firewall. E.g.: [imap] host = internal.example.com tunnel = "ssh bastion-server nc internal.example.com 143" -Peff
On Tue, Feb 07 2023, Jeff King wrote: > On Sun, Feb 05, 2023 at 10:51:04PM +0100, Ævar Arnfjörð Bjarmason wrote: > >> > Not if they did: >> > >> > [imap] >> > host = example.com >> > tunnel = some-command >> >> Yes, but how would they have ended up doing that? By discarding the >> documentation and throwing things at the wall & hoping they'd stick? > > That's what I would have tried without reading the documentation at all, > based on using other programs that tunnel imap. I'm just one data point, > of course. > >> I just don't get how anyone could have come to rely on this so that we'd >> care about supporting it. >> >> Because mutt has a feature that looks similar, users might have >> configured git-imap-send thinking it might do the same thing, and gotten >> lucky? > > It's less "mutt happens to do it this way" and more "associating a host > is strictly more useful, because it lets you interact with all the other > host-like features". It's only imap-send's funky config scheme that > makes it easy to mis-configure. > >> I guess in principle that could be true, but I think it's more likely >> that nobody's ever had reason to use it that way. I.e. if you use the >> "tunnel" the way the docs suggest you won't hit the credential helper, >> as you're authenticating with "ssh", and using "imapd" to directly >> operate on a Maildir path. *nod* I'll just note that you elided the part where I noted that I don't really care, and will submit some re-roll that's compatible with the current imap.{host,tunnel} interaction. I think you might be right that people might rely on this after having discovered this undocumented interaction by accident. But I also think that the lack of questions about how to get imap-send's tunnel mode to work with auth helpers (at least I couldn't find any on-list), which is what you'd run into if you went by the documentation & were trying to get htat ot work, is a pretty good sign that this may be either entirely unused by anyone, or at best very obscure. > As I said, my main use of tunneling is to trigger the imap server's > preauth mode. But there are other reasons one might want to do so, like > piercing a firewall. E.g.: > > [imap] > host = internal.example.com > tunnel = "ssh bastion-server nc internal.example.com 143" I'll definitely leave this out of a re-roll of this topic, but I did come up with an opinionated replacement on top. That commitdwhich rips out non-PREAUTH (i.e. any authentication) support, as well as SSL support that isn't using curl from git-imap-send.c. Here: https://github.com/avar/git/commit/8498089f8e5a3d050b44008a7947ef3cefe2a2dd I.e. if we just say that we're not going to support this use-case anymore we can get rid of all of the OpenSSL reliance in-tree, except for the optional (and hardly ever used) OPENSSL_SHA1, and uses-only-one-API-function "HAVE_OPENSSL_CSPRNG" use. I.e. we'd support tunneling like this still (from the manpage): [imap] folder = "INBOX.Drafts" tunnel = "ssh -q -C user@example.com /usr/bin/imapd ./Maildir 2> /dev/null" But if your use of imap.tunnel is to essentially use git-imap-send.c for what you could use another shell (or systemd or whatever) to invoke a "ssh" or "stunnel" command for you, we'd say too bad, just do that instead. So your example of: [imap] host = internal.example.com tunnel = "ssh bastion-server nc internal.example.com 143" Would instead be: 1. Arrange for the equivalent of that to run outside of git-imap-send, e.g.: ssh -N -R 1430:internal.example.com:143 bastion-server 2. Use "imap.host" to connect to that "remote" box with libcurl, but just use "localhost:1430" Given the obscurity of git-imap-send overall, and how trivial the workaround is I don't think that's unreasonable, even with an aggressive transition period. As that commit shows we have a surprising amount of code required to support just this one use-case (and I'm not even sure I got all of it). Or at least: 7 files changed, 89 insertions(+), 509 deletions(-) With most being OpenSSL library use, so if we can find a way to not keeping supporting that...
Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes: > I think you might be right that people might rely on this after having > discovered this undocumented interaction by accident. > > But I also think that the lack of questions about how to get imap-send's > tunnel mode to work with auth helpers (at least I couldn't find any > on-list), which is what you'd run into if you went by the documentation > & were trying to get htat ot work, is a pretty good sign that this may > be either entirely unused by anyone, or at best very obscure. I actually think the misconfiguration (from documentation's point of view) Peff is taking advantage of is a behaviour you would naturally expect, if you do not read the documentation but are merely aware of the presence of .host and .tunnel and guess what these do. And those who felt it was a natural design would probably not have asked any question about it. Documenting the current behaviour better would not hurt. Updating the behaviour and documenting the new behaviour would not help anybody.
On Tue, Feb 07 2023, Junio C Hamano wrote: > Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes: > >> I think you might be right that people might rely on this after having >> discovered this undocumented interaction by accident. >> >> But I also think that the lack of questions about how to get imap-send's >> tunnel mode to work with auth helpers (at least I couldn't find any >> on-list), which is what you'd run into if you went by the documentation >> & were trying to get htat ot work, is a pretty good sign that this may >> be either entirely unused by anyone, or at best very obscure. > > I actually think the misconfiguration (from documentation's point of > view) Peff is taking advantage of is a behaviour you would naturally > expect, if you do not read the documentation but are merely aware of > the presence of .host and .tunnel and guess what these do. And > those who felt it was a natural design would probably not have asked > any question about it. Documenting the current behaviour better > would not hurt. Updating the behaviour and documenting the new > behaviour would not help anybody. Sure, we don't have to belabor the point. It's moot for a re-roll of this topic in any case (I won't be changing this behavior). But do I take it from the non-reply to what came afterwards that you're not interested in a (not a part of this topic) proposal for us to say "if you want that, arrange for ssh to do it for you", which would allow for finally dropping libssl as a non-trivial direct dependency? Or just that you didn't get to considering that?
On Tue, Feb 07, 2023 at 09:39:48PM +0100, Ævar Arnfjörð Bjarmason wrote: > *nod* I'll just note that you elided the part where I noted that I don't > really care, and will submit some re-roll that's compatible with the > current imap.{host,tunnel} interaction. Yeah, sorry, I should have said "yes, thank you" there. :) I wasn't meaning to continue arguing, but just trying to answer your "how would they even find this?" confusion. > I.e. if we just say that we're not going to support this use-case > anymore we can get rid of all of the OpenSSL reliance in-tree, except > for the optional (and hardly ever used) OPENSSL_SHA1, and > uses-only-one-API-function "HAVE_OPENSSL_CSPRNG" use. Yeah, getting rid of that openssl code is a reasonable goal. And this may seem counter-intuitive, but I'm actually _more_ in favor of that than the change you proposed here, even though it potentially breaks more users. That's because I feel like we're buying something useful with it, whereas with the patch we've been discussing, the tradeoff was less clear to me. That said, it seems like there should be a path forward for supporting tunnels via curl, and then we could be getting rid of the openssl dependency _and_ all of the custom and rarely-run imap code. But that's an even bigger task, and I not only wouldn't want to work on it, I'm not even sure I'd want to review it. I'm slightly regretting getting involved here at all, because it seems like none of us actually care at all about imap-send, and this has turned into a big discussion. I mostly chimed in because it seemed like I had a perspective you didn't on how people might use tunnels, and it felt like I should speak up for folks whose use cases might be getting broken. Side note: If somebody were proposing to add imap-send at all today, I'd probably say "no, that should be a separate project, and you should probably write it in some language that has a decent imap library". It really has nothing at all to do with Git in terms of implementation, and I suspect it's not super well maintained in general. But perhaps it is too late for that. > So your example of: > > [imap] > host = internal.example.com > tunnel = "ssh bastion-server nc internal.example.com 143" > > Would instead be: > > 1. Arrange for the equivalent of that to run outside of > git-imap-send, e.g.: > > ssh -N -R 1430:internal.example.com:143 bastion-server > > 2. Use "imap.host" to connect to that "remote" box with libcurl, > but just use "localhost:1430" Having done something like that before, the "arrange" step is more finicky than you might think (because sometimes it goes away, and you really want to trigger it on demand). -Peff
On Tue, Feb 07, 2023 at 01:26:10PM -0800, Junio C Hamano wrote: > Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes: > > > I think you might be right that people might rely on this after having > > discovered this undocumented interaction by accident. > > > > But I also think that the lack of questions about how to get imap-send's > > tunnel mode to work with auth helpers (at least I couldn't find any > > on-list), which is what you'd run into if you went by the documentation > > & were trying to get htat ot work, is a pretty good sign that this may > > be either entirely unused by anyone, or at best very obscure. > > I actually think the misconfiguration (from documentation's point of > view) Peff is taking advantage of is a behaviour you would naturally > expect, if you do not read the documentation but are merely aware of > the presence of .host and .tunnel and guess what these do. Just to be clear, I am not taking advantage of anything. I do not use imap-send myself, because a much better solution is to have a decent mail client that can access both imap and local maildirs. ;) I was only trying to offer some perspective as a general imap-tunnel user. -Peff
Jeff King <peff@peff.net> writes: > Side note: If somebody were proposing to add imap-send at all today, > I'd probably say "no, that should be a separate project, and you > should probably write it in some language that has a decent imap > library". It really has nothing at all to do with Git in terms of > implementation, and I suspect it's not super well maintained in > general. But perhaps it is too late for that. amen..
On Tue, Feb 07 2023, Jeff King wrote: > On Tue, Feb 07, 2023 at 09:39:48PM +0100, Ævar Arnfjörð Bjarmason wrote: > >> *nod* I'll just note that you elided the part where I noted that I don't >> really care, and will submit some re-roll that's compatible with the >> current imap.{host,tunnel} interaction. > > Yeah, sorry, I should have said "yes, thank you" there. :) I wasn't > meaning to continue arguing, but just trying to answer your "how would > they even find this?" confusion. *nod* >> I.e. if we just say that we're not going to support this use-case >> anymore we can get rid of all of the OpenSSL reliance in-tree, except >> for the optional (and hardly ever used) OPENSSL_SHA1, and >> uses-only-one-API-function "HAVE_OPENSSL_CSPRNG" use. > > Yeah, getting rid of that openssl code is a reasonable goal. And this > may seem counter-intuitive, but I'm actually _more_ in favor of that > than the change you proposed here, even though it potentially breaks > more users. That's because I feel like we're buying something useful > with it, whereas with the patch we've been discussing, the tradeoff was > less clear to me. Makes sense. > That said, it seems like there should be a path forward for supporting > tunnels via curl, and then we could be getting rid of the openssl > dependency _and_ all of the custom and rarely-run imap code. I really think it's obscure enough that just offerng users a documented way out should be neough. > [...] > Side note: If somebody were proposing to add imap-send at all today, > I'd probably say "no, that should be a separate project, and you > should probably write it in some language that has a decent imap > library". It really has nothing at all to do with Git in terms of > implementation, and I suspect it's not super well maintained in > general. But perhaps it is too late for that. I think it's a reasonable feature, but in hindsight our mistake was to think that we should be perma-forking isync, which has since moved on. I've used isync's "mbsync" extensively for IMAP in other contexts, and it works well for that. So if we were going back to the drawing board a "git-imap-sync" really should just be something in our mail tooling that can produce a Maildir, and if we wanted an IMAP helper it could invoke mbsync, offlineimap or various other "maildir to IMAP" bidirectional syncing utilities to "send" via IMAP. So, just some hook support for format-patch with some documented examples should do it, but I won't be working on that task...
On Wed, Feb 08, 2023 at 02:06:55AM +0100, Ævar Arnfjörð Bjarmason wrote: > > Side note: If somebody were proposing to add imap-send at all today, > > I'd probably say "no, that should be a separate project, and you > > should probably write it in some language that has a decent imap > > library". It really has nothing at all to do with Git in terms of > > implementation, and I suspect it's not super well maintained in > > general. But perhaps it is too late for that. > > I think it's a reasonable feature, but in hindsight our mistake was to > think that we should be perma-forking isync, which has since moved > on. I've used isync's "mbsync" extensively for IMAP in other contexts, > and it works well for that. > > So if we were going back to the drawing board a "git-imap-sync" really > should just be something in our mail tooling that can produce a Maildir, > and if we wanted an IMAP helper it could invoke mbsync, offlineimap or > various other "maildir to IMAP" bidirectional syncing utilities to > "send" via IMAP. > > So, just some hook support for format-patch with some documented > examples should do it, but I won't be working on that task... Yes, I think format-patch plus a sync program would be good. I did briefly look at the state of imap sync programs and was a bit disappointed. Many older recommendations are for software that is no longer packaged, or hard to find. And none of the ones I looked at do something as simple as "copy these messages to this imap server". They're all very interested in bidirectional sync, incremental updates, and so on. But I do think one could make mbsync or offlineimap work, if you used a dedicated folder on the server as the destination. But yeah, I don't think you or I needs to come up with a solution there. I was more proposing along the lines of: let's drop imap-send, and interested people can then make a solution based on other tools, or even spin off imap-send into its own repository. But I get that even that is some work, and it may mean complaining users. -Peff
diff --git a/imap-send.c b/imap-send.c index 9712a8d4f93..24b30c143a7 100644 --- a/imap-send.c +++ b/imap-send.c @@ -917,7 +917,7 @@ static void server_fill_credential(struct imap_server_conf *srvc, struct credent return; cred->protocol = xstrdup(srvc->use_ssl ? "imaps" : "imap"); - cred->host = xstrdup(srvc->host); + cred->host = xstrdup(srvc->tunnel ? "tunnel" : srvc->host); cred->username = xstrdup_or_null(srvc->user); cred->password = xstrdup_or_null(srvc->pass); @@ -1004,7 +1004,7 @@ static struct imap_store *imap_open_store(struct imap_server_conf *srvc, const c if (!CAP(AUTH_CRAM_MD5)) { fprintf(stderr, "You specified " "CRAM-MD5 as authentication method, " - "but %s doesn't support it.\n", srvc->host); + "but tunnel doesn't support it.\n"); goto bail; } /* CRAM-MD5 */ @@ -1021,8 +1021,8 @@ static struct imap_store *imap_open_store(struct imap_server_conf *srvc, const c } } else { if (CAP(NOLOGIN)) { - fprintf(stderr, "Skipping account %s@%s, server forbids LOGIN\n", - srvc->user, srvc->host); + fprintf(stderr, "Skipping account %s, server forbids LOGIN\n", + srvc->user); goto bail; } if (!imap->buf.sock.ssl) @@ -1434,12 +1434,9 @@ int cmd_main(int argc, const char **argv) fprintf(stderr, "no imap store specified\n"); return 1; } - if (!server.host) { - if (!server.tunnel) { - fprintf(stderr, "no imap host specified\n"); - return 1; - } - server.host = "tunnel"; + if (!server.host && !server.tunnel) { + fprintf(stderr, "no imap host specified\n"); + return 1; } /* read the messages */
Before [1] we'd force the "imap.host" to be set, even if the "imap.tunnel" was set, and then proceed to not use the "host" for establishing a connection, as we'd use the tunneling command. However, we'd still use the "imap.host" if it was set as the "host" field given to the credential helper, and in messages that were shared with the non-tunnel mode, until a preceding commit made these OpenSSL codepaths tunnel-only. Let's always give "host=tunnel" to the credential helper when in the "imap.tunnel" mode, and rephrase the relevant messages to indicate that we're tunneling. This changes the existing behavior, but that behavior was emergent and didn't make much sense. If we were using "imap.tunnel" the value in "imap.host" might be entirely unrelated to the host we're tunneling to. Let's not pretend to know more than we do in that case. 1. 34b5cd1fe9f (Don't force imap.host to be set when imap.tunnel is set, 2008-04-22) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> --- imap-send.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-)