LetsEncrypt 3.1.0 - Challenges Failed for all domains

  • Hello guys, I searched the forums for similar problems but I couldn't find the right solution for my situation
    I've successfully generated certificates for a couple of domains with one of the previous versions, but the last one gives me some problems while trying to generate a new certificate for some other domain.


    This is what I get in the plugin's log:

    Code
    1. [Tue May 16 17:14:05 2017] [debug] Modules::Plugin::_call: Calling run() method on Plugin::LetsEncrypt[Tue May 16 17:14:05 2017] [debug] iMSCP::Service::__ANON__: Systemd init system has been detected[Tue May 16 17:14:05 2017] [debug] iMSCP::Execute::execute: /bin/systemctl --system is-active apache2.service[Tue May 16 17:14:05 2017] [debug] iMSCP::Provider::Service::Sysvinit::_exec: active[Tue May 16 17:14:05 2017] [debug] Plugin::LetsEncrypt::run: Executing `toadd' tasks for the `erbolandia.biz' SSL certificate[Tue May 16 17:14:05 2017] [debug] Plugin::LetsEncrypt::_issueCertificate: Required action: issue[Tue May 16 17:14:05 2017] [debug] Plugin::LetsEncrypt::_deleteLineages: Deleting any SSL certificate lineage matching the erbolandia.biz domain name[Tue May 16 17:14:05 2017] [debug] iMSCP::Execute::execute: /usr/local/sbin/certbot-auto certonly --quiet --agree-tos --email [email protected] --webroot --webroot-path /var/www/imscp/gui/plugins/LetsEncrypt/acme --preferred-challenges http --allow-subset-of-names --cert-name erbolandia.biz --domains erbolandia.biz,www.erbolandia.biz[Tue May 16 17:14:11 2017] [debug] iMSCP::Execute::getExitCode: Command exited with value: 1[Tue May 16 17:14:11 2017] [error] Plugin::LetsEncrypt::run: Challenge failed for domain erbolandia.bizChallenge failed for domain www.erbolandia.bizChallenges failed for all domains


    and this is the letsencrypt.log


    i-MSCP : 1.3.16
    System : Debian Jessie x64
    System : php-fpm, apache2, proftpd, SSL for services and panel
    Plugins : ClamAV, LetsEncrypt, Mailgraph, Monitorix, OpenDKIM, PhpSwitcher, Postscreen, RoundcubePlugins, ServerDefaultPage, SpamAssassin
    Plugins Versions : all are the latest versions


    Could you please help me in finding out if I'm doing something wrong ?


    Thank you very much, bye Kess.

  • Hi,


    please check this topic - perhaps this helps. Looks like the .well-known directory is not accessible


    Regards

    (Ubuntu 16.04, i-MSCP 1.5.1, php-Fpm, Plugins: ClamAV, CronJobs, InstantSSH, LetsEncrypt, Mailgraph, Monitorix, OpenDKIM, PhpSwitcher, PolicydSPF, Postscreen, RecaptchaPMA, RoundcubePlugins, ServerDefaultPage, SpamAssassin, YubiKeyAuth)

  • Thank you for your reply.
    I already tested the access to the .well-known directory and it works without problems.


    At least, if I click here: http://www.erbolandia.biz/.wel…n/acme-challenge/.gitkeep it works correctly.
    During the certificate request procedure I can see the 2 files that are created it the directory, they rest there for something like 2 seconds and then these files disappear... I bet because of a cleanup at the end of the procedure.


    The problem should be elsewhere, but I can't understand...

  • @kess


    During the certificate request procedure I can see the 2 files that are created it the directory,


    In what directory (full path please) the files were created exactly? According the logs you posted, the files were created in the /var/www/imscp/gui/plugins/LetsEncrypt/acme/.well-known/acme-challenge directory and that is expected. The plugin make use of the same directory for all ACME challenges.


    Can you please disable the ServerDefaultPage plugin and give a new try?

    badge.php?id=1239063037&bid=2518&key=1747635596&format=png&z=547451206

  • Hello @Nuxwin,
    i confirm that the path is

    Code
    1. /var/www/imscp/gui/plugins/LetsEncrypt/acme/.well-known/acme-challenge


    and during the request:

    I've disabled the ServerDefaultPage Plugin and restarted apache, but no changes...


    Do you need any other informations in order to investigate ?

  • Do you need any other informations in order to investigate ?

    No. Best would be to give me access to the panel and ssh ;)

    badge.php?id=1239063037&bid=2518&key=1747635596&format=png&z=547451206

  • Just for you to check further:
    apache log during request:

    Code
    1. 66.133.109.36 - - [17/May/2017:11:24:51 +0200] "GET /.well-known/acme-challenge/Rmfg8eE-nMh9eGcKA2HXNBssLUnDKaFeVfNrBICBUco HTTP/1.1" 403 - "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"66.133.109.36 - - [17/May/2017:11:24:51 +0200] "GET /.well-known/acme-challenge/WLbSRgaX5QDyecJ7C47NO5kf-d1OvhcpTV36j5JmJR4 HTTP/1.1" 403 - "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"

    files in the directory during request:

    Code
    1. root@w01 /var/www/imscp/gui/plugins/LetsEncrypt/acme/.well-known/acme-challenge # ls -la
    2. total 16K
    3. drwxr-xr-x 2 vu2000 vu2000 4.0K May 17 11:24 .
    4. drwxr-xr-x 3 vu2000 vu2000 4.0K May 16 21:37 ..
    5. -rw-r--r-- 1 vu2000 vu2000 0 May 16 21:40 .gitkeep
    6. -rw-r--r-- 1 root root 87 May 17 11:24 Rmfg8eE-nMh9eGcKA2HXNBssLUnDKaFeVfNrBICBUco
    7. -rw-r--r-- 1 root root 87 May 17 11:24 WLbSRgaX5QDyecJ7C47NO5kf-d1OvhcpTV36j5JmJR4

    Can't understand why apache returns a 403 error on these files.

  • @kess


    Can you provide me with a true SSH access and admin access to the control panel? I would investigate further because I cannot reproduce the problem on my installation. Teamviewer is a mess... Too slow and so on...


    Thanks.

    badge.php?id=1239063037&bid=2518&key=1747635596&format=png&z=547451206

  • Problem found... My bad as always...... Sorry...


    There are the contents of the file /etc/apache2/imscp/erbolandia.biz.conf

    Code
    1. # Custom Apache configuration for erbolandia.biz## Any changes made to this file will be preserved on update.# i-MSCP doesn't check the contents of this file.## This file should NOT be deleted.# Bad botSetEnvIfNoCase User-Agent ^$ bad_botsSetEnvIfNoCase User-Agent "AhrefsBot" bad_botSetEnvIfNoCase User-Agent "archive.org_bot" bad_botSetEnvIfNoCase User-Agent "MJ12bot" bad_botSetEnvIfNoCase User-Agent "facebookexternalhit" bad_botSetEnvIfNoCase User-Agent "Twitterbot" bad_botSetEnvIfNoCase User-Agent "Baiduspider" bad_botSetEnvIfNoCase User-Agent "spbot" bad_botSetEnvIfNoCase User-Agent "HaosouSpider" bad_botSetEnvIfNoCase User-Agent "MetaURI" bad_botSetEnvIfNoCase User-Agent "mediawords" bad_botSetEnvIfNoCase User-Agent "FlipboardProxy" bad_botSetEnvIfNoCase User-Agent "abot" bad_botSetEnvIfNoCase User-Agent "aipbot" bad_botSetEnvIfNoCase User-Agent "asterias" bad_botSetEnvIfNoCase User-Agent "EI" bad_botSetEnvIfNoCase User-Agent "libwww-perl" bad_botSetEnvIfNoCase User-Agent "LWP" bad_botSetEnvIfNoCase User-Agent "lwp" bad_botSetEnvIfNoCase User-Agent "MSIECrawler" bad_botSetEnvIfNoCase User-Agent "nameprotect" bad_botSetEnvIfNoCase User-Agent "PlantyNet_WebRobot" bad_botSetEnvIfNoCase User-Agent "UCmore" bad_botSetEnvIfNoCase User-Agent "Alligator" bad_botSetEnvIfNoCase User-Agent "AllSubmitter" bad_botSetEnvIfNoCase User-Agent "Anonymous" bad_botSetEnvIfNoCase User-Agent "Asterias" bad_botSetEnvIfNoCase User-Agent "autoemailspider" bad_botSetEnvIfNoCase User-Agent "Badass" bad_botSetEnvIfNoCase User-Agent "Baiduspider" bad_botSetEnvIfNoCase User-Agent "BecomeBot" bad_botSetEnvIfNoCase User-Agent "Bitacle" bad_botSetEnvIfNoCase User-Agent "bladder\ fusion" bad_botSetEnvIfNoCase User-Agent "Blogshares\ Spiders" bad_botSetEnvIfNoCase User-Agent "Board\ Bot" bad_botSetEnvIfNoCase User-Agent "Board\ Bot" bad_botSetEnvIfNoCase User-Agent "Convera" bad_botSetEnvIfNoCase User-Agent "ConveraMultiMediaCrawler" bad_botSetEnvIfNoCase User-Agent "c-spider" bad_botSetEnvIfNoCase User-Agent "DA" bad_botSetEnvIfNoCase User-Agent "DnloadMage" bad_botSetEnvIfNoCase User-Agent "Download\ Demon" bad_botSetEnvIfNoCase User-Agent "Download\ Express" bad_botSetEnvIfNoCase User-Agent "Download\ Wonder" bad_botSetEnvIfNoCase User-Agent "dragonfly" bad_botSetEnvIfNoCase User-Agent "DreamPassport" bad_botSetEnvIfNoCase User-Agent "DSurf" bad_botSetEnvIfNoCase User-Agent "DTS Agent" bad_botSetEnvIfNoCase User-Agent "EBrowse" bad_botSetEnvIfNoCase User-Agent "eCatch" bad_botSetEnvIfNoCase User-Agent "edgeio" bad_botSetEnvIfNoCase User-Agent "Email\ Extractor" bad_botSetEnvIfNoCase User-Agent "EmailSiphon" bad_botSetEnvIfNoCase User-Agent "EmailWolf" bad_botSetEnvIfNoCase User-Agent "EmeraldShield" bad_botSetEnvIfNoCase User-Agent "ESurf" bad_botSetEnvIfNoCase User-Agent "Exabot" bad_botSetEnvIfNoCase User-Agent "ExtractorPro" bad_botSetEnvIfNoCase User-Agent "FileHeap!\ file downloader" bad_botSetEnvIfNoCase User-Agent "FileHound" bad_botSetEnvIfNoCase User-Agent "Forex" bad_botSetEnvIfNoCase User-Agent "Franklin\ Locator" bad_botSetEnvIfNoCase User-Agent "FreshDownload" bad_botSetEnvIfNoCase User-Agent "FrontPage" bad_botSetEnvIfNoCase User-Agent "FSurf" bad_botSetEnvIfNoCase User-Agent "Gaisbot" bad_botSetEnvIfNoCase User-Agent "Gamespy_Arcade" bad_botSetEnvIfNoCase User-Agent "genieBot" bad_botSetEnvIfNoCase User-Agent "GetBot" bad_botSetEnvIfNoCase User-Agent "GetRight" bad_botSetEnvIfNoCase User-Agent "Gigabot" bad_botSetEnvIfNoCase User-Agent "Go!Zilla" bad_botSetEnvIfNoCase User-Agent "Go-Ahead-Got-It" bad_botSetEnvIfNoCase User-Agent "GOFORITBOT" bad_botSetEnvIfNoCase User-Agent "heritrix" bad_botSetEnvIfNoCase User-Agent "HLoader" bad_botSetEnvIfNoCase User-Agent "HooWWWer" bad_botSetEnvIfNoCase User-Agent "HTTrack" bad_botSetEnvIfNoCase User-Agent "iCCrawler" bad_botSetEnvIfNoCase User-Agent "ichiro" bad_botSetEnvIfNoCase User-Agent "iGetter" bad_botSetEnvIfNoCase User-Agent "imds_monitor" bad_botSetEnvIfNoCase User-Agent "Industry\ Program" bad_botSetEnvIfNoCase User-Agent "Indy\ Library" bad_botSetEnvIfNoCase User-Agent "InetURL" bad_botSetEnvIfNoCase User-Agent "InstallShield\ DigitalWizard" bad_botSetEnvIfNoCase User-Agent "IRLbot" bad_botSetEnvIfNoCase User-Agent "IUPUI\ Research\ Bot" bad_botSetEnvIfNoCase User-Agent "Java" bad_botSetEnvIfNoCase User-Agent "jeteye" bad_botSetEnvIfNoCase User-Agent "jeteyebot" bad_botSetEnvIfNoCase User-Agent "JoBo" bad_botSetEnvIfNoCase User-Agent "JOC\ Web\ Spider" bad_botSetEnvIfNoCase User-Agent "Kapere" bad_botSetEnvIfNoCase User-Agent "Larbin" bad_botSetEnvIfNoCase User-Agent "LeechGet" bad_botSetEnvIfNoCase User-Agent "LightningDownload" bad_botSetEnvIfNoCase User-Agent "Linkie" bad_botSetEnvIfNoCase User-Agent "Mac\ Finder" bad_botSetEnvIfNoCase User-Agent "Mail\ Sweeper" bad_botSetEnvIfNoCase User-Agent "Mass\ Downloader" bad_botSetEnvIfNoCase User-Agent "MegaIndex" bad_botSetEnvIfNoCase User-Agent "MetaProducts\ Download\ Express" bad_botSetEnvIfNoCase User-Agent "Microsoft\ Data\ Access" bad_botSetEnvIfNoCase User-Agent "Microsoft\ URL\ Control" bad_botSetEnvIfNoCase User-Agent "Missauga\ Locate" bad_botSetEnvIfNoCase User-Agent "Missauga\ Locator" bad_botSetEnvIfNoCase User-Agent "Missigua Locator" bad_botSetEnvIfNoCase User-Agent "Missouri\ College\ Browse" bad_botSetEnvIfNoCase User-Agent "Mister\ PiX" bad_botSetEnvIfNoCase User-Agent "MovableType" bad_botSetEnvIfNoCase User-Agent "Mozi!" bad_botSetEnvIfNoCase User-Agent "Mozilla/3.0 (compatible)" bad_botSetEnvIfNoCase User-Agent "Mozilla/5.0 (compatible; MSIE 5.0)" bad_botSetEnvIfNoCase User-Agent "MSIE_6.0" bad_botSetEnvIfNoCase User-Agent "MSIECrawler" badbotSetEnvIfNoCase User-Agent "MVAClient" bad_botSetEnvIfNoCase User-Agent "MyFamilyBot" bad_botSetEnvIfNoCase User-Agent "MyGetRight" bad_botSetEnvIfNoCase User-Agent "NASA\ Search" bad_botSetEnvIfNoCase User-Agent "Naver" bad_botSetEnvIfNoCase User-Agent "NaverBot" bad_botSetEnvIfNoCase User-Agent "NetAnts" bad_botSetEnvIfNoCase User-Agent "NetResearchServer" bad_botSetEnvIfNoCase User-Agent "NEWT\ ActiveX" bad_botSetEnvIfNoCase User-Agent "Nextopia" bad_botSetEnvIfNoCase User-Agent "NICErsPRO" bad_botSetEnvIfNoCase User-Agent "NimbleCrawler" bad_botSetEnvIfNoCase User-Agent "Nitro\ Downloader" bad_botSetEnvIfNoCase User-Agent "Nutch" bad_botSetEnvIfNoCase User-Agent "Offline\ Explorer" bad_botSetEnvIfNoCase User-Agent "OmniExplorer" bad_botSetEnvIfNoCase User-Agent "OutfoxBot" bad_botSetEnvIfNoCase User-Agent "P3P" bad_botSetEnvIfNoCase User-Agent "PagmIEDownload" bad_botSetEnvIfNoCase User-Agent "pavuk" bad_botSetEnvIfNoCase User-Agent "PHP\ version" bad_botSetEnvIfNoCase User-Agent "playstarmusic" bad_botSetEnvIfNoCase User-Agent "Program\ Shareware" bad_botSetEnvIfNoCase User-Agent "Progressive Download" bad_botSetEnvIfNoCase User-Agent "psycheclone" bad_botSetEnvIfNoCase User-Agent "puf" bad_botSetEnvIfNoCase User-Agent "PussyCat" bad_botSetEnvIfNoCase User-Agent "PuxaRapido" bad_botSetEnvIfNoCase User-Agent "Python-urllib" bad_botSetEnvIfNoCase User-Agent "Qwantify" bad_botSetEnvIfNoCase User-Agent "RealDownload" bad_botSetEnvIfNoCase User-Agent "RedKernel" bad_botSetEnvIfNoCase User-Agent "relevantnoise" bad_botSetEnvIfNoCase User-Agent "RepoMonkey\ Bait\ &\ Tackle" bad_botSetEnvIfNoCase User-Agent "RTG30" bad_botSetEnvIfNoCase User-Agent "SBIder" bad_botSetEnvIfNoCase User-Agent "script" bad_botSetEnvIfNoCase User-Agent "Seekbot" bad_botSetEnvIfNoCase User-Agent "seoscanners" bad_botSetEnvIfNoCase User-Agent "SemrushBot" bad_botSetEnvIfNoCase User-Agent "SiteExplorer" bad_botSetEnvIfNoCase User-Agent "SiteSnagger" bad_botSetEnvIfNoCase User-Agent "SmartDownload" bad_botSetEnvIfNoCase User-Agent "sna-" bad_botSetEnvIfNoCase User-Agent "Snap\ bot" bad_botSetEnvIfNoCase User-Agent "SpeedDownload" bad_botSetEnvIfNoCase User-Agent "Sphere" bad_botSetEnvIfNoCase User-Agent "sproose" bad_botSetEnvIfNoCase User-Agent "SQ\ Webscanner" bad_botSetEnvIfNoCase User-Agent "Stamina" bad_botSetEnvIfNoCase User-Agent "Star\ Downloader" bad_botSetEnvIfNoCase User-Agent "Teleport" bad_botSetEnvIfNoCase User-Agent "TurnitinBot" bad_botSetEnvIfNoCase User-Agent "TwengaBot" bad_botSetEnvIfNoCase User-Agent "UdmSearch" bad_botSetEnvIfNoCase User-Agent "URLGetFile" bad_botSetEnvIfNoCase User-Agent "User-Agent" bad_botSetEnvIfNoCase User-Agent "UtilMind\ HTTPGet" bad_botSetEnvIfNoCase User-Agent "WebAuto" bad_botSetEnvIfNoCase User-Agent "WebCapture" bad_botSetEnvIfNoCase User-Agent "webcollage" bad_botSetEnvIfNoCase User-Agent "WebCopier" bad_botSetEnvIfNoCase User-Agent "WebFilter" bad_botSetEnvIfNoCase User-Agent "WebReaper" bad_botSetEnvIfNoCase User-Agent "Website\ eXtractor" bad_botSetEnvIfNoCase User-Agent "WebStripper" bad_botSetEnvIfNoCase User-Agent "WebZIP" bad_botSetEnvIfNoCase User-Agent "Wells\ Search" bad_botSetEnvIfNoCase User-Agent "WEP\ Search\ 00" bad_botSetEnvIfNoCase User-Agent "Wget" bad_botSetEnvIfNoCase User-Agent "Wildsoft\ Surfer" bad_botSetEnvIfNoCase User-Agent "WinHttpRequest" bad_botSetEnvIfNoCase User-Agent "WWWOFFLE" bad_botSetEnvIfNoCase User-Agent "Xaldon\ WebSpider" bad_botSetEnvIfNoCase User-Agent "Y!TunnelPro" bad_botSetEnvIfNoCase User-Agent "YahooYSMcm" bad_botSetEnvIfNoCase User-Agent "WWWOFFLE" bad_botSetEnvIfNoCase User-Agent "Xaldon\ WebSpider" bad_botSetEnvIfNoCase User-Agent "Y!TunnelPro" bad_botSetEnvIfNoCase User-Agent "YahooYSMcm" bad_botSetEnvIfNoCase User-Agent "YandexBot" bad_botSetEnvIfNoCase User-Agent "Zade" bad_botSetEnvIfNoCase User-Agent "ZBot" bad_botSetEnvIfNoCase User-Agent "zerxbot" bad_bot<Location />Order Allow,DenyDeny from env=bad_botAllow from all</Location>

    This has been setup in order to prevent unwanted scans from unwanted spiders...
    After clearing the contents and restarting apache service, the certificate request and setup has worked perfectly.


    But now the question: if the user agent is

    Code
    1. Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)

    why is it blocked with a 403 error ?


    Thx again a lot for your time @Nuxwin, my bad...