DNS validation currently broken?
- Steven Haigh
- Steven_Haigh
- 11 mths ago
- 16replies
- Buypass Customer Supportofficial rep10 mths ago
- Answered
With re-validating certs, I managed to renew the certs using http-01 without issue, but anything that uses dns-01 as a validation seems to fail.
The error certbot is getting is:
(domains trimmed) "error":{"type":"compound","detail":"Errors during validation","subproblems":[{"type":"urn:ietf:params:acme:error:incorrectResponse","detail":"Response received didn't match the challenge's requirements","code":0},{"type":"urn:ietf:params:acme:error:incorrectResponse","detail":"Response received didn't match the challenge's requirements","code":0}],"code":0}}],"wildcard":false}
I managed to do a successful dig against multiple nameservers, and the DNS TXT response seemed to be correct.
- Oldest first
- Newest first
- Active threads
- Popular
-
- Lise Ringvold Kristoffersenofficial rep
- Product Owner
- Lise_Ringvold_Kristoffer.1
- 11 mths ago
- Reported - view
Please provide us the domain name you use.
Like-
- Steven Haigh
- Steven_Haigh
- 11 mths ago
- Reported - view
Lise Ringvold Kristoffersen one of the internal only domains I use that I was attempting DNS validation on is capwap.crc.id.au
Like 1 -
- Steven Haigh
- Steven_Haigh
- 11 mths ago
- Reported - view
As further to this, if I run the same commandset against the LetsEncrypt directory endpoint, things work straight away and a cert is issued.
However, as updating a cert requires this device to reload and dump all access points, I'd rather set it up to use a longer-lived cert so the wifi network doesn't go down as often.
Like -
- mkonofficial rep
- QA
- mkon
- 11 mths ago
- Reported - view
Steven Haigh Hi. Could you please try again now?
Like -
- Steven Haigh
- Steven_Haigh
- 11 mths ago
- Reported - view
mkon Hi mate,
Still no luck... The output from certbot:
Saving debug log to /var/log/letsencrypt/letsencrypt.log - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Processing /etc/letsencrypt/renewal/capwap.crc.id.au.conf - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Renewing an existing certificate for capwap.crc.id.au Hook '--manual-auth-hook' for capwap.crc.id.au ran with output: Domain: capwap.crc.id.au DNS update: update add _acme-challenge.capwap.crc.id.au 90 TXT "ywxlRlFxvzqdM3k72ggGU5JLtXTEM3eDXSQ6POBiflE" Sending to DNS server... Waiting 5 seconds... Record should be set, returning to Certbot Hook '--manual-cleanup-hook' for capwap.crc.id.au ran with output: Domain: capwap.crc.id.au DNS update: update delete ${HOST}.${CERTBOT_DOMAIN} TXT Sending to DNS server... Failed to renew certificate capwap.crc.id.au with error: All authorizations were not finalized by the CA. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - All renewals failed. The following certificates could not be renewed: /etc/letsencrypt/live/capwap.crc.id.au/fullchain.pem (failure) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1 renew failure(s), 0 parse failure(s) Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
The response in the debug log seems to be along the lines of:
2024-01-03 23:16:58,933:DEBUG:urllib3.connectionpool:https://api.buypass.com:443 "POST /acme-v02/authz/4tE1Frv3IhiNJwewYu-WOlxdIQMl1SBxdCXB2ucr-F8 HTTP/1.1" 200 457 2024-01-03 23:16:58,935:DEBUG:acme.client:Received response: HTTP 200 Content-Encoding: gzip Content-Type: application/json Date: Wed, 03 Jan 2024 12:16:58 GMT Link: <https://api.buypass.com/acme/directory>; rel="index" Mdc-Correlationid: 519950e5-2059-4e3e-a016-b26811a15685 Replay-Nonce: MWMxNTU1OGMtNjE4NS00MzE2LWI1NGItM2Q5ODQwZDNlZGZk Vary: Accept-Encoding Content-Length: 457 Strict-Transport-Security: max-age=63072000 {"identifier":{"type":"dns","value":"capwap.crc.id.au"},"status":"pending","challenges":[{"type":"http-01","url":"https://api.buypass.com/acme-v02/authz/4tE1Frv3IhiNJwewYu-WOlxdIQMl1SBxdCXB2ucr-F8/1","status":"pending","token":"F9F89119 A401228CC29BD020547EB5DBF5CF8951"},{"type":"dns-01","url":"https://api.buypass.com/acme-v02/authz/4tE1Frv3IhiNJwewYu-WOlxdIQMl1SBxdCXB2ucr-F8/2","status":"processing","validated":"2024-01-03T12:15:18Z","token":"715BEF9EF20817246E81150E9 D1E80625F84E0F8","error":{"type":"compound","detail":"Errors during validation","subproblems":[{"type":"urn:ietf:params:acme:error:incorrectResponse","detail":"Response received didn't match the challenge's requirements","code":0},{"typ e":"urn:ietf:params:acme:error:incorrectResponse","detail":"Response received didn't match the challenge's requirements","code":0},{"type":"urn:ietf:params:acme:error:incorrectResponse","detail":"Response received didn't match the chall enge's requirements","code":0},{"type":"urn:ietf:params:acme:error:incorrectResponse","detail":"Response received didn't match the challenge's requirements","code":0},{"type":"urn:ietf:params:acme:error:incorrectResponse","detail":"Resp onse received didn't match the challenge's requirements","code":0},{"type":"urn:ietf:params:acme:error:incorrectResponse","detail":"Response received didn't match the challenge's requirements","code":0},{"type":"urn:ietf:params:acme:err or:incorrectResponse","detail":"Response received didn't match the challenge's requirements","code":0},{"type":"urn:ietf:params:acme:error:incorrectResponse","detail":"Response received didn't match the challenge's requirements","code": 0},{"type":"urn:ietf:params:acme:error:incorrectResponse","detail":"Response received didn't match the challenge's requirements","code":0}],"code":0}}],"wildcard":false} 2024-01-03 23:16:58,935:DEBUG:acme.client:Storing nonce: MWMxNTU1OGMtNjE4NS00MzE2LWI1NGItM2Q5ODQwZDNlZGZk 2024-01-03 23:16:58,939:DEBUG:certbot._internal.error_handler:Encountered exception: Traceback (most recent call last): File "/usr/lib/python3.12/site-packages/certbot/_internal/auth_handler.py", line 108, in handle_authorizations self._poll_authorizations(authzrs, max_retries, max_time_mins, best_effort) File "/usr/lib/python3.12/site-packages/certbot/_internal/auth_handler.py", line 216, in _poll_authorizations raise errors.AuthorizationError('All authorizations were not finalized by the CA.') certbot.errors.AuthorizationError: All authorizations were not finalized by the CA.
Like
-
- mkonofficial rep
- QA
- mkon
- 11 mths ago
- Reported - view
From our logs, it seems we cant find the txt record
Like-
- Steven Haigh
- Steven_Haigh
- 11 mths ago
- Reported - view
mkon That's strange... I altered the script to not return to certbot until it had a positive response from both 8.8.8.8 and 1.1.1.1 - and get the following:
Saving debug log to /var/log/letsencrypt/letsencrypt.log Requesting a certificate for capwap.crc.id.au Hook '--manual-auth-hook' for capwap.crc.id.au ran with output: Thu Jan 4 01:06:48 AM AEDT 2024 Domain: capwap.crc.id.au DNS update: update add _acme-challenge.capwap.crc.id.au 30 TXT "ywxlRlFxvzqdM3k72ggGU5JLtXTEM3eDXSQ6POBiflE" Sending to DNS server... Validating TXT vs 8.8.8.8 and 1.1.1.1 Output from 8.8.8.8: _acme-challenge.capwap.crc.id.au. 30 IN TXT "ywxlRlFxvzqdM3k72ggGU5JLtXTEM3eDXSQ6POBiflE" _acme-challenge.capwap.crc.id.au. 30 IN RRSIG TXT 13 5 30 20240117060122 20240103130648 16016 crc.id.au. UhRc1YOd36l/Xd4phv+paIEC/uSgZGrjTyoOJe3jjRGhzbDwy/fg+ywd e2ub7uINZ9zkN6LVEVAZvlkRGvKXGw== Output from 1.1.1.1: _acme-challenge.capwap.crc.id.au. 30 IN TXT "ywxlRlFxvzqdM3k72ggGU5JLtXTEM3eDXSQ6POBiflE" _acme-challenge.capwap.crc.id.au. 30 IN RRSIG TXT 13 5 30 20240117060122 20240103130648 16016 crc.id.au. UhRc1YOd36l/Xd4phv+paIEC/uSgZGrjTyoOJe3jjRGhzbDwy/fg+ywd e2ub7uINZ9zkN6LVEVAZvlkRGvKXGw==
Yet the validation still fails.
Is it possible that there's a DNSSEC error somewhere in the chain?
Like -
- Steven Haigh
- Steven_Haigh
- 10 mths ago
- Reported - view
Given this thread with a similar problem that does NOT have DNSSEC enabled, there goes that theory.
https://community.buypass.com/t/35y3jpb/problem-validating-certain-subdomainsmkon Do you have any other suggestions for troubleshooting? As if all I change is the directory server in certbot to LetsEncrypt, the certificates are validated instantly.
Like -
- Steven Haigh
- Steven_Haigh
- 10 mths ago
- Reported - view
Well, I just set up to do a packet capture on all 3 DNS servers - ran the cert generation stuff once, saw nothing on any of the nameservers.
Checked my command line on the packet dump, ran it again and the certs were issued successfully and I saw 2 x NS servers get the verification query.
Not exactly sure what was going on, but I now have a valid cert...
Like -
- mkonofficial rep
- QA
- mkon
- 10 mths ago
- Reported - view
Steven Haigh Great that it works. Tried checking responses from your domain yesterday, but it all seemed ok. Today i actually see that one nameserver does not reply. Weird case
Like -
- Steven Haigh
- Steven_Haigh
- 10 mths ago
- Reported - view
mkon Annoyingly, while it worked for that domain, I'm trying another - and hitting the same problem - even though its using the same nameservers, certbot instance etc etc :(
I'm wondering if I'm hitting a rate limit - being as I've renewed quite a few certs on my domain in the past few days?
I'm doing a packet dump on all 3 primary DNS servers, and none are getting the request for the _acme-challenge TXT record.
Like -
- mkonofficial rep
- QA
- mkon
- 10 mths ago
- Reported - view
Steven Haigh Can`t see any issues on rate limits, if you use the same account. But it seems to be some timeout issues somewhere. If you hit a rate limit issue, the response back to your client should state that.
Like -
- Steven Haigh
- Steven_Haigh
- 10 mths ago
- Reported - view
mkon Thanks for checking... The one successful validation I got via DNS, the probe came from 172.253.x.x - and it hit ns1.crc.id.au.
When these domains fail, I see nothing hitting ns1, ns2 or ns3.
I don't know if you have access or not, but is it possible to attempt to resolve stuff from these resolvers?
Interestingly, I just tried repeating the same command as before, and now the second one I hit this issue with is now working...
I'm not exactly sure what is going on, but it seems to be at the actual DNS TXT record resolution phase.
Like
-
- Buypass Customer Supportofficial rep
- Customer Support
- Buypass_Customer_Support
- 10 mths ago
- Reported - view
Hi Steven Haigh, are you still experiencing problems with the renewal of your certificates? We have made some minor updates to our service the last couple of days.
Could you please retry?Like-
- Steven Haigh
- Steven_Haigh
- 10 mths ago
- Reported - view
Buypass Customer Support I did manage to get the certs eventually.
A couple of the services are not mission critical - so I can try a force renew of those certs if that would be helpful?
Like -
- Buypass Customer Supportofficial rep
- Customer Support
- Buypass_Customer_Support
- 10 mths ago
- Reported - view
Hi Steven Haigh , there is no need for you to force renew the certificates. Thanks for your contribution in Buypass Community.
Like