14.6. I cannot crawl sites that use SSL (https)¶
This is directly related to cURL. You need to supply trusted root certificates for cURL installed on your server. You can ask about this to your server support.
If you are not able to resolve the issue, you can disable SSL verification for the site whose SSL certificate cannot be verified by your server. To learn how to do this, please see Disabling SSL verification for a site guide.