[Thu Jan 20 14:15:16 2011] [error] Oops, no RSA or DSA server certificate found for 'www.somedomain.com:0'?!

Whilst somedomain.com isn’t the official domain name reported (I can’t reveal the client), this error was being printed for every SSL host except the default SSL host. Ironically the default SSL host was simply a redirect to one of the others. A quick check and indeed the problem lied with the SSL hosts – we removed every ssl host and the webserver would start fine – obviously without any ssl.

My colleague and I were  perplexed. He’d tried a quite few things to fix this all without luck. A so call Server Admin told him it was due to not using ip based virtual hosts for ssl, he claimed you can’t use Name Base Virtual hosts with SSL. No doubt this was obtained from a quick google search for the error. The problems is you can run NamedBasedVirtual hosts with SSL on port 443 provided you have a wildcard SSL certificate. A wild is required for NamedBaseVirtual hosts as the SSL connection is established first before the headers are sent. A wildcard will allow any subdomain to use the SSL connection then apache will see the host header and respond with the appropriate vhost. If on only have a single certificate this does not work and you’ll need a separate IP per certificate.

Anyway, we began trying to debug the issue. First we checked the certificate files were at the specified locations – they were. Next we checked the certificates were actually valid. You can use the openssl command below to do this:

openssl x509 -noout -text -in YOURCERTIFICATE.crt

The certificate, key, and certificate authority (CA) were all valid and in date.
Next we tried putting each Vhost in to the config one by one to see if one host had errors over another. Turns out it didn’t matter what order each host was in the config file or which ssl hosts were included, they all had issues – except for the default ssl vhost.

At this point we were a little lost. So we decided to go back to basics and work out what the error really meant. We search to see what apache module the error came from. A simple grep later we’d narrowed down the error to mod_ssl. A search of the mod_ssl source code found the following instance of the message:

# grep Oops *
ssl_engine_init.c:                         "Init: Oops, you want to request client "
ssl_engine_init.c:                "Oops, no RSA or DSA server certificate found "
ssl_engine_init.c:                "Oops, no RSA or DSA server private key found?!");
ssl_engine_io.c:                    (argp != NULL ? "(BIO dump follows)" : "(Oops, no memory buffer?)"));

Looking in ssl_engine_init.c we found the error came from the following function

static void ssl_init_server_certs(server_rec *s,
                                  apr_pool_t *p,
                                  apr_pool_t *ptemp,
                                  modssl_ctx_t *mctx)
{
    const char *rsa_id, *dsa_id;
    const char *vhost_id = mctx->sc->vhost_id;
    int i;
    int have_rsa, have_dsa;
 
    rsa_id = ssl_asn1_table_keyfmt(ptemp, vhost_id, SSL_AIDX_RSA);
    dsa_id = ssl_asn1_table_keyfmt(ptemp, vhost_id, SSL_AIDX_DSA);
 
    <em>have_rsa</em> = ssl_server_import_cert(s, mctx, rsa_id, SSL_AIDX_RSA);
    <em>have_dsa</em> = ssl_server_import_cert(s, mctx, dsa_id, SSL_AIDX_DSA);
 
    if (!(have_rsa || have_dsa)) {
        ap_log_error(APLOG_MARK, APLOG_ERR, 0, s,
                <strong>"Oops, no RSA or DSA server certificate found "
                "for</strong> '%s:%d'?!", s-&gt;server_hostname, s-&gt;port);
        ssl_die();
    }
 
    for (i = 0; i &lt; SSL_AIDX_MAX; i++) {         ssl_check_public_cert(s, ptemp, mctx-&gt;pks-&gt;certs[i], i);
    }
 
    have_rsa = ssl_server_import_key(s, mctx, rsa_id, SSL_AIDX_RSA);
    have_dsa = ssl_server_import_key(s, mctx, dsa_id, SSL_AIDX_DSA);
 
    if (!(have_rsa || have_dsa)) {
        ap_log_error(APLOG_MARK, APLOG_ERR, 0, s,
                "Oops, no RSA or DSA server private key found?!");
        ssl_die();
    }
}

Hence  the error was caused by the certificates not being able to be imported. Once again we checked paths to make sure the certificates/keys were correct. Alas they were. So we began to wonder why the certificates couldn’t be found. We’d specified the correct files, confirmed they were correct. It occurred to me that perhaps the openSSL context had not been setup correctly. But why not? I took a look at the default SSL vhost which did work and noticed a single line that were not in any of the other ssl vhosts.

SSLEngine on

The comment above this line read:

#   SSL Engine Switch:
#   Enable/Disable SSL for this virtual host.

I added “SSLEngine On” to the other ssl vhosts and it worked! So it turns out you can have an vhost setup on port 443 without SSL hence for each vhost you want SSL working in you must add the above line. My colleague was extremely thankful – why it happened in the  first place, we still don’t know. We suspect previously the option may have been enabled globally. However the fix allows apache to run again and works after a clean shutdown and startup.

Here’s the url redirect list in case someone else wants to use cgit over gitweb (highly recommended)

<VirtualHost *:80>
        ServerName gitweb2.freedesktop.org
        ServerAdmin sitewranglers@freedesktop.org

        CustomLog /var/log/apache2/gitweb.freedesktop.org-access.log combined
        ErrorLog /var/log/apache2/gitweb.freedesktop.org-error.log

        #
        # Redirects mapping gitweb -> cgit
        #
        # Gitweb uses get targets seperated by ;
        #
        # /?...;...;...
        #
        # p = Project
        # a = action ( blob, blob_plain, blobdiff, commitdiff, commit, shortlog, summary, tree, log, tag, history, rss)
        # h = SHA Hash
        # o = sort order
        # hb= SHA Hash Tree Base
        # hp= ?
        # pg= page
        # f= file/dir
        #
        # Cgit uses the following:
        #
        # /project/action/?...
        #
        # action ( commit, log, diff, tree, tag, patch  )
        #
        # id = SHA Hash
        # id2 = SHA Hash
        # h = head
        #
        # Translation rules
        #
        # Project is a straight redirect
        # ---
        # /?p=([^.]+).git;       http://cgit.freedesktop.org/$1/
        #
        # Action requires a mapping
        # ---
        # a=(blob|tree) /tree/
        # a=(blobdiff|commitdiff)       /diff/
        # a=commit      /commit/
        # a=(summary)   /
        # a=(shortlog|log|history)      /log/
        # a=tag         /tag/
        # a=blob_plain  /blob/
        # a=rss    ?No CGIT Equivilant?
        #
        # Targets require mapping
        # ---
        # h=([^;]+)     id=$1
        # f=([^;]+)     /$1
        # hb=([^;]+)    id2=$1
        # hp([^;]+      id=$1
        #
        # Now putting it all together
        #
        RewriteEngine On
        #RewriteLog /tmp/rewrite.log
        #RewriteLogLevel 5
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=blob;h=([^;]+);hb=([^;]+);f=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/tree/%5?id=%3;id2=%4 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=blob;hb=([^;]+);f=([^;]+)
        RewriteRule ^/$  http://cgit.freedesktop.org/%1/tree/%4?id=%3 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=tree;h=([^;]+);hb=([^;]+);f=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/tree/%5?id=%4 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=tree;h=([^;]+);hb=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/tree/?id=%4 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=tree;hb=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/tree/?id=%3 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=tree
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/tree/? [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=blobdiff;h=([^;]+);hp=([^;]+);hb=([^;]+);f=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/diff/%6?id2=%4;id=%3;id3=%5 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=commitdiff;h=([^;]+);hp=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/diff/?id=%4;id2=%3 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=commitdiff;h=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/diff/?id=%3 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=commit;h=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/commit/?id=%3 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=shortlog;h=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/log/?id=%3 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=shortlog
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/log/? [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=log;h=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/log/?id=%3 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=log
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/log? [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=history;h=([^;]+);hb=([^;]+);f=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/log/%5?id=%4 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=history;h=([^;]+);hb=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/log/?id=%4 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=history;hb=([^;]+);f=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/log/%4?id=%3 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=history;hb=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/log/?id=%3 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=tag;h=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/tag/?id=%4 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=blob_plain;h=([^;]+);f=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/blob/%4?id=%3 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=blob_plain;f=([^;]+)
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/blob/%4 [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=rss
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/atom? [R,L,NE]
        RewriteCond %{query_string} p=([^.]+)(\.git)*;a=summary
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/? [R,L,NE]
        #
        # Fail safes incase nothing above matches, try at least to put the person in the project, els root of cgit
        RewriteCond %{query_string} p=([^.]+)(\.git)*.*
        RewriteRule ^/$ http://cgit.freedesktop.org/%1/? [R,L,NE]
        RewriteRule ^.* http://cgit.freedesktop.org/ [R,L,NE]
</VirtualHost>

    RewriteEngine on
    RewriteLog /SOMELOGFILE
    RewriteLogLevel 2
    RewriteRule ^/(.*)$ http://localhost:8180/$1 [P]
    ProxyPassReverse / http://localhost:8180/

The above will proxy the entire domain to the server on the same machine running on port 8180. The [P] tells apache to use the proxy module to do the work. This is not a redirect but a proxy so it hides the internal URL with the external facing one. The ProxyPassReverse directive tells apache to do the rewrite of the URL to remove the localhost:8180 bit back to the originating url

Have you ever found yourself needing to remove authentication from part of a website? This actually happens fairly regularly. The way you do it is as follows:

.htaccess

AuthType none
Satisfy Any

The AuthType none directive indicates apache should not prompt for a password, whilst the Satisfy Any directive tells apache that is can now use other methods to authenticate the user. You may also have to add:

Order Deny, Allow
Allow from All

Which tells apache everyone is allowed to access the page.