c++ - Internal connection management in libcurl and DNS ttl

Question

libcurl caches connections and reuses them for consecutive requests. Suppose I have 4 connections in progress to example.com and I need to make 5th connection. Libcurl will try to establish new 5th connection to example.com. What will happen if at the time the 5th connection is created TTL for example.com DNS result expired? I suppose libcurl doesn't cache DNS results and simply queries OS resolver (I don't use ares), and if example.com expired in the cache a new DNS request will be made. Now, let's suppose original 4 requests to example.com were resolved as requests to 1.1.1.1 and 5th request was resolved as 5.5.5.5.

• What will happen to internal connection cache in libcurl?
• Does libcurl "throw away" and close connections to 1.1.1.1 when these 4 requests complete?
• What if I also had requests to something.com that also resolved to 1.1.1.1 ip?

If it matters I use curl mutli interface.

Answer 1

curl has its own internal DNS cache, where it will cache resolved addresses for 60 seconds by default (you can change that). Subsequent resolves of the same name will thus use the cached results within that time frame.

curl's connection cache is entirely based on the host name used in the URL so if you have an existing available connection to "example.com" sitting in the cache, that will be used for a subsequent request to that same host name. curl doesn't know nor care what the IP address for that name is or if it changed since the connection was started. It skips the entire name resolving phase when reuses a connection.

When a transfer is done and the connection is still alive, the connection is put back into the connection cache (or closed if the cache is deemed "full" due to having reached a limit).

Since the connection reuse is done based on names, having another name that resolves to the same IP of an existing connection won't make curl reuse that connection. It will resolve the name and create a new connection for that.

A connection can be kept in the connection cache for an unlimited period of time unless it gets killed to make room or gets reused. If it "dies" (due to it getting closed from the other end) it will eventually be removed from the cache when that gets noticed.

HTTP/2

PING frames etc that can be sent over HTTP/2 will not (atm) be handled for connections in the connection cache, which will lead to them getting killed by the server within soon. (libcurl 7.62.0 adds a new API to allow applications to keep such connections alive as well, see curl_easy_upkeep)

DoH

With the introduction of DoH (DNS-over-HTTPS) support coming in curl 7.62.0, the DNS cache will cache the names for TTL number of seconds and not just the default 60 when that is used.

Caveats

There are some limitations, conditions and edge cases that break what's explained here, but this is the basics.