Trans­ition to nginx: part 4 – CGI scripts

I still have some CGI scripts on this web­site. They still work, and they are good enough for my needs. When I switched this web­site to nginx (the Word­Press setup was a little bit more com­plex than what I wrote in part 1, part 2 and part 3… the con­fig will be one of my next blog posts) I was a little bit puzzled how to do that with nginx. It took me some minutes to get an idea how to do it and to find the right FreeBSD port for this.

  • In­stall www/​fcgiwrap
  • Add the fol­low­ing to rc.conf:

fcgiwrap_enable=“YES”
fcgiwrap_user=“www”

  • Run “ser­vice fc­gi­wrap start”
  • Add the fol­low­ing to your nginx con­fig:
loc­a­tion ^~ /​cgi-​bin/​ {
    gzip off; #gzip makes scripts feel slower since they have to com­plete be­fore get­ting gzipped
    fastcgi_​pass  unix:/var/run/fcgiwrap/fcgiwrap.sock;
    fastcgi_​index index.cgi;
    fastcgi_​param SCRIPT_​FILENAME /path/to/location$fastcgi_script_name;
    fastcgi_​param GATEWAY_​INTERFACE  CGI/1.1;
}

StumbleUponXINGBalatarinBox.netDiggGoogle GmailNetvouzPlurkSiteJotTypePad PostYahoo BookmarksVKSlashdotPocketHacker NewsDiigoBuddyMarksRedditLinkedInBibSonomyBufferEmailHatenaLiveJournalNewsVinePrintViadeoYahoo MailAIMBitty BrowserCare2 NewsEvernoteMail.RuPrintFriendlyWaneloYahoo MessengerYoolinkWebnewsStumpediaProtopage BookmarksOdnoklassnikiMendeleyInstapaperFarkCiteULikeBlinklistAOL MailTwitterGoogle+PinterestTumblrAmazon Wish ListBlogMarksDZoneDeliciousFlipboardFolkdJamespotMeneameMixiOknotiziePushaSvejoSymbaloo FeedsWhatsAppYouMobdiHITTWordPressRediff MyPageOutlook.comMySpaceDesign FloatBlogger PostApp.netDiary.RuKindle ItNUjijSegnaloTuentiWykopTwiddlaSina WeiboPinboardNetlogLineGoogle BookmarksDiasporaBookmarks.frBaiduFacebookGoogle ClassroomKakaoQzoneSMSTelegramRenrenKnownYummlyShare/​Save

Tran­si­tion to nginx: part 3 — short and easy con­fig snip­pets

After some medium-​difficoulty trans­itions in part 1 and part 2, here some easy ones:

phpMy­Ad­min: take the ba­sics from one of the two other blog posts (see above) without loc­a­tion dir­ect­ives. For “loc­a­tion /​” set the doc­u­ment root and copy the “loc­a­tion ~ \.php” from the con­fig of one of the parts above. Done.

TT-​RSS: take the con­fig like for phpMy­Ad­min and add (as­sum­ing it is in the root of the server, else you have to add the path in the front of the loc­a­tion)

loc­a­tion ^~ /(utils|templates|schema|cache|lock|locale|classes) {
     deny all;
}

Al­low client-​side cach­ing for static con­tent:

loc­a­tion ~* \.(?:jpe?g|gif|png|ico|cur|gz|bz2|xz|tbz|tgz|txz|svg|svgz|mp4|ogg|ogv|webm|htc|css|js|
pdf|zip|rar|tar|txt|conf)$ {
    try_​files $uri =404;

    ex­pires 1w;     # If you are not a big site,

                    # and don’t change static con­tent of­ten,

                    # 1 week is not bad.
    access_​log off; # If you don’t need the logs
    add_​header Cache-​Control “pub­lic”;
}

Se­cur­ity: Des­pite the fact that the docs I’ve read tell that no-​SSLv3 is the de­fault, the first set­ting makes a dif­fer­ence (tested via SSLlabs“ SSLtest).

ssl_​protocols TLSv1 TLSv1.1 TLSv1.2; # No SSLv 23
ssl_​dhparam /path/to/dhparams.pem;   # gen­er­ate via “openssl dh­param –out /path/to/dhparams.pem 2048”

 

Trans­ition to nginx: part 2 – con­vert­ing a gal­lery v2 in­stall­a­tion

In my first trans­ition to nginx I wrote that I was happy about the speed in­crease I got for my Horde web­mail setup. Af­ter­wards I con­ver­ted a Gal­lery v2 in­stall­a­tion (yes, old, not un­der act­ive de­vel­op­ment any­more, but in­ternal and still work­ing). There I have not seen any ob­vi­ous speed dif­fer­ence.

I did not con­vert all .htac­cess re­write rules, the one for the “easy and beau­ti­ful” URL names was too com­plex for the con­verter for re­write I found. As it is just for in­ternal use, I just switched back to the not so nice “tech­nical” URL names.

The im­port­ant part of the apache 2.2 in­stall­a­tion:

Ex­piresAct­ive On
Ex­piresDe­fault “now plus 1 hour”
Ex­pires­By­Type image/​* “now plus 1 month”
Ex­pires­By­Type text/​javascript “now plus 1 month”
Ex­pires­By­Type application/​x-​javascript “now plus 1 month”
Ex­pires­By­Type text/​css “now plus 1 month”

<Loc­a­tion /​>
# In­sert fil­ter
SetOut­put­Fil­ter DEFLATE

# Nets­cape 4.x has some prob­lems…
Browser­Match ^Mozilla/​4 gzip-​only-​text/​html

# Nets­cape 4.06−4.08 have some more prob­lems
Browser­Match ^Mozilla/4\.0[678] no-​gzip

# MSIE mas­quer­ades as Nets­cape, but it is fine
Browser­Match \bM­SIE !no-​gzip !gzip-​only-​text/​html
# Don’t com­press im­ages
SetEn­vI­fNoCase Request_​URI \
\.(?:gif|jpe?g|png|gz|bz2|zip|pdf)$ no-​gzip dont-​vary

# Make sure prox­ies don’t de­liver the wrong con­tent
Header ap­pend Vary User-​Agent env=!dont-vary
</​Location>

The nginx con­fig:

worker_​processes  1;

error_​log  <fi­le­name>;

events {
        worker_​connections      1024;
        use                     kqueue;
}


http {
    in­clude       mime.types;
    default_​type  application/​octet-​stream;

    access_​log  <fi­le­name>;

    send­file on;

        keepalive_​timeout       15;
        client_​body_​timeout     300;
        client_​header_​timeout   12;
        send_​timeout            300;
        client_​body_​in_​file_​only clean;
        client_​body_​buffer_​size 128k;
        client_​max_​body_​size 40M;

        gzip on;
        gzip_​min_​length 1000;
        gzip_​types       text/​plain text/​xml text/​css application/​xml application/xhtml+xml application/rss+xml application/​javascript application/​x-​javascript;
        gzip_​disable     “msie6”;

        in­clude blacklist.conf;

    server {
        listen       80;
        server_​name  <host­name>;

        add_​header   x-​frame-​options            “same­ori­gin”;
        add_​header   x-​xss-​protection           “1; mode=block”;
        add_​header   x-​content-​type-​options     “nos­niff”;

        char­set utf-​8;

        #access_​log  logs/host.access.log  main;
        if ($bad_​client) { re­turn 403; }

        loc­a­tion /​ {
            root   /​usr/​local/​www/​gallery2;
            in­dex  index.php;
                loc­a­tion ~ \.php {
                        # Zero-​day ex­ploit de­fense.
                        # http://​forum​.nginx​.org/​r​e​a​d​.​p​h​p​?​2​,​8​8​8​4​5​,​p​a​g​e=3
                        # Won’t work prop­erly (404 er­ror) if the file is not stored on this server, which is en­tirely pos­sible with php-​fpm/​php-​fcgi.
                        # Com­ment the „try_​files“ line out if you set up php-​fpm/​php-​fcgi on an­other ma­chine.  And then cross your fin­gers that you won’t get hacked.
                        try_​files $uri =404;

                        fastcgi_​split_​path_​info ^(.+\.php)(/.+)$;
                        fastcgi_​keep_​conn on;
                        fastcgi_​index      index.php;
                        in­clude          fastcgi_​params;
                        fastcgi_​param      SCRIPT_​FILENAME $document_root$fastcgi_script_name;
                        fastcgi_​pass        unix:/var/run/php.fcgi;
                }
        }

        # re­dir­ect server er­ror pages to the static page /50x.html
        #
        error_​page   500 502 503 504  /50x.html;
        loc­a­tion = /50x.html {
            root   /​usr/​local/​www/​nginx-​dist;
        }

        # deny ac­cess to .htac­cess files, if Apache’s doc­u­ment root
        # con­curs with nginx’s one
        #
        loc­a­tion ~ /\.ht {
            deny all;
        }
        loc­a­tion ~ \.(inc|class)$ {
                deny all;
        }
        loc­a­tion ^~ /​lib/​tools/​po/​ {
                deny all;
        }
    }
}

Trans­ition to nginx: part 1, con­vert­ing Horde web­mail

I am a long­time apache user. It may be that I touched apache 1.2 for the first time. Re­cently I de­cided to check out nginx. So I de­cided to do it on my own web­mail sys­tem. The end res­ult is, I re­placed apache+mod_php with nginx+php-fpm there, and so far it does not look like I want to go back (it feels faster on dir­ect com­pare, same server, either apache or nginx star­ted to com­pare the speed, pure sub­ject­ive “meas­ure­ment”, no num­bers).

The long story now.

The web­mail sys­tem uses horde. There I had apache 2.4 (pre­fork mpm) and php 5.6 via mod_​php. With nginx I used php-​fpm. I used the same php flags and val­ues in php_​fpm like I used with mod_​php. I con­figured less php-​fpm max-​processes than I had al­lowed apache+mod_php to use. As nginx is not spawn­ing pro­cesses for each con­nec­tion, I have less pro­cesses and also less memory al­loc­ated as a res­ult. Con­vert­ing the re­write rules and mod_​rewrite based black­list­ing took a while (and I have not con­ver­ted all black­lists I had be­fore). So yes, this is not really com­par­ing apples with apples (I could have tried a dif­fer­ent mpm for apache, and I could have used an fcgi based php ap­proach in­stead of mod_​php, I could have moved the re­write rules out from .htac­cess files to the main con­fig).

The res­ult was on first lo­gin that the pages ap­peared no­tice­able faster. I dir­ectly switched back to apache to con­firm. This was over a WLAN con­nec­tion. A little bit later I had the same im­pres­sion when I tested this over a slow DSL link (2 MBit/​s).

Here the im­port­ant parts of the apache con­fig:

Ex­piresAct­ive On
Ex­piresDe­fault “now plus 3 hours”
Ex­pires­By­Type image/​* “now plus 2 months”
Ex­pires­By­Type text/​javascript “now plus 2 months”
Ex­pires­By­Type application/​x-​javascript “now plus 2 months”
Ex­pires­By­Type text/​css “now plus 2 months”

SetEn­vI­fNoCase Request_​URI “\.gif$” cache_me=1
SetEn­vI­fNoCase Request_​URI “\.png$” cache_me=1
SetEn­vI­fNoCase Request_​URI “\.jpg$” cache_me=1
SetEn­vI­fNoCase Request_​URI “\.jpeg$” cache_me=1
SetEn­vI­fNoCase Request_​URI “\.ico$” cache_me=1
SetEn­vI­fNoCase Request_​URI “\.css$” cache_me=1
# Al­low cach­ing on me­dia files
<If­Mod­ule mod_headers.c>
Header merge Cache-​Control “pub­lic” env=cache_me
  <If­Mod­ule ssl_module.c>
    Header add Strict-​Transport–Se­cur­ity “max-age=15768000”
  </​IfModule>
</​IfModule>
<Loc­a­tion /​>
# In­sert fil­ter
SetOut­put­Fil­ter DEFLATE

# Nets­cape 4.x has some prob­lems…
Browser­Match ^Mozilla/​4 gzip-​only-​text/​html

# Nets­cape 4.06−4.08 have some more prob­lems
Browser­Match ^Mozilla/4\.0[678] no-​gzip

# MSIE mas­quer­ades as Nets­cape, but it is fine
Browser­Match \bM­SIE !no-​gzip !gzip-​only-​text/​html
# Don’t com­press im­ages
SetEn­vI­fNoCase Request_​URI \
\.(?:gif|jpe?g|png|gz|bz2|zip)$ no-​gzip dont-​vary

# Make sure prox­ies don’t de­liver the wrong con­tent
Header ap­pend Vary User-​Agent env=!dont-vary
</​Location>
SetEn­vI­fNoCase Request_​URI “\.js$” cache_me=1
Alias /​Microsoft-​Server-​ActiveSync /​usr/​local/​www/​horde/rpc.php
Re­dir­ect­Per­man­ent /.well-known/carddav /horde/rpc.php
Ac­cept­Path­Info on

And here most parts of my nginx.conf suit­able for Horde, in­clud­ing the re­write rules from .htac­cess files:

worker_​processes  1;

error_​log  <file>;

events {
        worker_​connections      1024;
        use                     kqueue;
}


http {
        in­clude    mime.types;
        default_​type  application/​octet-​stream;

        access_​log  <file>;

        send­file on;
        keepalive_​timeout       15;
        client_​body_​timeout     300;
        client_​header_​timeout   12;
        send_​timeout        300;
        client_​body_​in_​file_​only clean;
        client_​body_​buffer_​size 128k;
        client_​max_​body_​size 10M;

        gzip on;
        gzip_​min_​length 1000;
        gzip_​types       text/​plain text/​xml text/​css application/​xml application/xhtml+xml application/rss+xml application/​javascript application/​x-​javascript;
        gzip_​disable     “msie6”;

        in­clude blacklist.conf;

        server {
                listen     443 ssl spdy;
                server_​name  <host­name>;

                ssl_​certificate         <file>;
                ssl_​certificate_​key     <file>;

                ssl_​session_​cache       shared:SSL:10m;
                ssl_​session_​timeout     15m;
                ssl_​ciphers     <ciper_​list>;
                ssl_​prefer_​server_​ciphers  on;

                # op­tional: see https://​www​.owasp​.org/​i​n​d​e​x​.​p​h​p​/​L​i​s​t​_​o​f​_​u​s​e​f​u​l​_​H​T​T​P​_​h​e​a​d​ers
                add_​header                        strict-​transport-​security “max-age=31536000”;
                add_​header                        x-​frame-​options          “same­ori­gin”;
                add_​header                        x-​xss-​protection        “1; mode=block”;
                add_​header                        x-​content-​type-​options        “nos­niff”;

                root                            /​usr/​local/​www/​horde;
                in­dex                              index.php;

                char­set utf8;

                access_​log  <log­file>;
                if ($bad_​client) { re­turn 403; }

                loc­a­tion /​ {
                        loc­a­tion /​Microsoft-​Server-​ActiveSync {
                                alias                   /usr/local/www/horde/rpc.php;
                                fastcgi_​split_​path_​info ^(.+\.php)(/.+)$;
                                fastcgi_​keep_​conn on;
                                in­clude                 fastcgi_​params;
                                fastcgi_​param           SCRIPT_​FILENAME /usr/local/www/horde/rpc.php;
                                fastcgi_​pass            unix:/var/run/php.fcgi;
                        }

                        loc­a­tion /autodiscover/autodiscover.xml {
                                alias                   /usr/local/www/horde/rpc.php;
                                fastcgi_​split_​path_​info ^(.+\.php)(/.+)$;
                                fastcgi_​keep_​conn on;
                                in­clude                 fastcgi_​params;
                                fastcgi_​param           SCRIPT_​FILENAME /usr/local/www/horde/rpc.php;
                                fastcgi_​pass            unix:/var/run/php.fcgi;
                        }

                        loc­a­tion /Autodiscover/Autodiscover.xml {
                                alias                   /usr/local/www/horde/rpc.php;
                                fastcgi_​split_​path_​info ^(.+\.php)(/.+)$;
                                fastcgi_​keep_​conn on;
                                in­clude                 fastcgi_​params;
                                fastcgi_​param           SCRIPT_​FILENAME /usr/local/www/horde/rpc.php;
                                fastcgi_​pass            unix:/var/run/php.fcgi;
                        }

                        loc­a­tion ^~ /(static|themes)/ {
                                ex­pires                 1w;
                                add_​header              Cache-​Control pub­lic;
                        }
                        loc­a­tion ^~ /services/ajax.php {
                                fastcgi_​split_​path_​info ^(.+\.php)(/.+)$;
                                fastcgi_​keep_​conn on;
                                in­clude                 fastcgi_​params;
                                fastcgi_​param           SCRIPT_​FILENAME $document_root$fastcgi_script_name;
                                fastcgi_​pass            unix:/var/run/php.fcgi;
                        }

                        loc­a­tion ~ \.php {
                                # Zero-​day ex­ploit de­fense.
                                # http://​forum​.nginx​.org/​r​e​a​d​.​p​h​p​?​2​,​8​8​8​4​5​,​p​a​g​e=3
                                # Won’t work prop­erly (404 er­ror) if the file is not stored on this server, which is en­tirely pos­sible with php-​fpm/​php-​fcgi.
                                # Com­ment the „try_​files“ line out if you set up php-​fpm/​php-​fcgi on an­other ma­chine.  And then cross your fin­gers that you won’t get hacked.
                                try_​files $uri =404;

                                fastcgi_​split_​path_​info ^(.+\.php)(/.+)$;
                                fastcgi_​keep_​conn on;
                                fastcgi_​index           index.php;
                                in­clude                 fastcgi_​params;
                                fastcgi_​param           SCRIPT_​FILENAME $document_root$fastcgi_script_name;
                                fastcgi_​pass            unix:/var/run/php.fcgi;
                        }

                        try_​files                          $uri $uri/​ /rampage.php?$args;
                }
        # re­dir­ect server er­ror pages to the static page /50x.html
        #
        error_​page   500 502 503 504  /50x.html;
        loc­a­tion = /50x.html {
                root   /​usr/​local/​www/​nginx-​dist;
        }

        # deny ac­cess to .htac­cess files, if Apache’s doc­u­ment root
        # con­curs with nginx’s one
        #
                loc­a­tion ~ /\.ht {
                        deny all;
                }
                loc­a­tion ~ /(config|lib|locale|po|scripts|templates)/ {
                        deny all;
                }
                loc­a­tion ^~ /​rpc/​ {
                        if (!-e $request_​filename){
                                re­write ^(.*)$ /rpc/index.php/$1 break;
                        }
                }
                loc­a­tion ^~ /​kronolith/​feed/​ {
                        if (!-e $request_​filename){
                                re­write ^(.*)$ /kronolith/feed/index.php?c=$1 break;
                        }
                }
                loc­a­tion ^~ /​content/​ {
                        if (!-e $request_​filename){
                                re­write ^(.*)$ /content/index.php break;
                        }
                }
                loc­a­tion ^~ /whups/(queue|query)/ {
                        if (!-e $request_​filename){
                                re­write ^/([0 – 9]+)/?$ /whups/queue/index.php?id=$1;
                        }
                        re­write ^/([0 – 9]+)/rss/?$ /whups/queue/rss.php?id=$1;
                        re­write ^/([a-zA-Z0-9_]+)/?$ /whups/queue/index.php?slug=$1;
                        re­write ^/([a-zA-Z0-9_]+)/rss/?$ /whups/queue/rss.php?slug=$1;
                }
                loc­a­tion ^~ /​whups/​ticket/​ {
                        if (!-e $request_​filename){
                                re­write ^/([0 – 9]+)/?$ /whups/ticket/index.php?id=$1;
                        }
                        re­write ^/([0 – 9]+)/rss/?$ /whups/ticket/rss.php?id=$1;
                        re­write ^/([0 – 9]+)/([a-z]+)(\.php)?$ /whups/ticket/$2.php?id=$1 break;
                }
                loc­a­tion ^~ /.well-known/carddav {
                        re­turn 301 https://​web​mail​.Leidinger​.net/​r​p​c​.​php;
                }
                loc­a­tion ^~ /​admin/​ {
                        al­low <local>;
                        deny all;
                }
                loc­a­tion ~ /test.php$ {
                        al­low <local>;
                        deny all;
                }

                # Me­dia: im­ages, icons, video, au­dio, HTC, archives
                loc­a­tion ~* \.(?:jpe?g|gif|png|ico|cur|gz|bz2|tbz|tgz|svg|svgz|mp4|ogg|ogv|webm|htc|css|js|pdf|zip|rar|tar|txt|pl|conf)$ {
                        try_​files $uri =404;

                        ex­pires 1w;
                        access_​log off;
                        add_​header Cache-​Control “pub­lic”;
                }
        }
}

Es­sen Hack­thon 2015 – last day status

I com­mit­ted the 64bit sup­port for the linux base ports (dis­abled by de­fault, check the com­mit mes­sage), but this broke the INDEX build. Port­mgr was faster than me to re­vert it. All er­rors are mine. I think most of the work is done, I just need to find out what the cor­rect way is to handle this make/​fmake dif­fer­ence (mal­formed con­di­tional).