Monday, September 07, 2009

How to Proxy a Website to HTTPS with nginx, Start to Finish

Setting up a secure website is a trivial task often overlooked by site owners, possibly because they believe it is costly or technically challenging. Anyone who has set up their own HTTP webserver can easily proxy that site to HTTPS using free software. SSL certificates cost around £10 a year, a similar cost to registering a domain name.

I'm going to describe the steps I took to proxy QatarLiving to a secure site at F1a.me. QatarLiving is one of the most popular websites in Qatar, a small Middle Eastern state.

Step 0. Prerequisites

You'll need administrator (root) privileges to your own server to run an HTTPS site on port 443. I'm happy with my virtualized Linux box, hosted by Media Temple. It costs $50 a month, which is not the cheapest deal out there nor is the server particularly powerful, but I'm happy with the company's service and the box is sufficient to host all my websites and twitter feeds. If you sign up with Media Temple, you should install compilers and development tools.

Step 1. Buy your SSL certificate

Purchase an SSL certificate for your secure site. I bought mine from 123-reg. It is possible to run a secure site without buying a certificate, which is known as "self signing"; however, users of your site will be presented with scary warnings by their browser. It's best just to splash out on the SSL certificate: they're cheap.

Step 2. Downloads

If you've never come across nginx (pronounced Engine-X), don't feel too down-heartened. This tiny webserver fits into a niche of specialized webservers (including boa, lighttpd and thttpd) that compete on speed and small memory footprint. Many large sites have chosen nginx as their front-line server, including one of the very largest blogging sites, WordPress.com. I chose to use nginx because its small footprint is crucial on my limited-memory server.

Download the latest development release of nginx, currently version 8.1.14 [homepage | nginx-0.8.14.tar.gz]

Step 3. Installation

Unpack the nginx source, compile and install. The following commands work for me, on my fairly standard centos Linux box, but some of the paths may require tweaking on other systems.


$ tar zxvf nginx-0.8.14.tar.gz
$ cd nginx-0.8.14
$ ./configure \
--prefix=/usr \
--sbin-path=/usr/sbin/nginx-ssl \
--conf-path=/etc/nginx-ssl/nginx.conf \
--http-log-path=/var/log/nginx_ssl_access_log \
--error-log-path=/var/log/nginx_ssl_error_log \
--pid-path=/var/run/nginx-ssl.pid \
--http-client-body-temp-path=/var/tmp/nginx_ssl_client \
--http-proxy-temp-path=/var/tmp/nginx_ssl_proxy \
--http-fastcgi-temp-path=/var/tmp/nginx_ssl_fastcgi \
\
--with-http_ssl_module \
--with-http_stub_status_module \
\
--without-http_charset_module \
--without-http_gzip_module \
--without-http_userid_module \
--without-http_auth_basic_module \
--without-http_autoindex_module \
--without-http_geo_module \
--without-http_map_module \
--without-http_referer_module \
--without-http_fastcgi_module \
--without-http_limit_zone_module \
--without-http_browser_module \
--without-http_upstream_ip_hash_module
$ make
$ su
# make install


If any errors appear with the above commands, it probably means you need to install some required library. In most cases, using Google on an error message will provide the fastest solutions. Options may vary slightly between versions. Always check ./configure --help for the current list.

Step 4. Server Configuration

Download your server SSL certificate and private key, rename them to server.crt and server.key, and move them to your nginx config directory (/etc/nginx-ssl/). If you've bought a cheap SSL certificate like I did, the certificating authority probably won't be built into your users' browsers by default. You'll need to find the certificate of your CA and concatinate it with your server's SSL certificate. Your server.crt will then look something like this:


-----BEGIN CERTIFICATE-----
[your certificate here]
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
[your CA's certificate here]
-----END CERTIFICATE-----


Now, all that's left is to edit your config file /etc/nginx-ssl/nginx.conf. You'll want to play around with some of these setting, but they provide a good starting point.


# the number of worker processes should be
# equal to the number of processor cores in your server
worker_processes 4;

# disable error logging by uncommenting the next line
# error_log /dev/null crit;


events {
# max connections - 2048 is reasonable
worker_connections 2048;
}

http {
include mime.types;
default_type application/octet-stream;

# TCP tuning
sendfile on;
tcp_nopush on;
tcp_nodelay off;

# keepalive is highly beneficial for SSL
keepalive_timeout 10;

# disable logging
access_log off;

server {
listen 443;

# change this to your own website hostname
server_name f1a.me;

# these SSL options have been chosen to maximize
# throughput on small servers
ssl on;
ssl_certificate server.crt;
ssl_certificate_key server.key;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 5m;
ssl_protocols SSLv2 SSLv3 TLSv1;
ssl_ciphers RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;
ssl_prefer_server_ciphers on;

location / {
proxy_pass http://www.qatarliving.com;
proxy_set_header X-Forwarded-For $remote_addr;
}

# don't proxy static content, redirect instead
location ~* \.(gif|jpg|jpeg|png|ico|js)$ {
rewrite ^/(.*?)$ http://www.qatarliving.com/$1 permanent;
}
}
}


Start up your server by running nginx-ssl as root:


$ su
# /usr/sbin/nginx-ssl


Tail the error log (/var/log/nginx_ssl_error_log) if the server fails to start.

Further Reading

Nginx English documentation

Useful changes to default nginx configuration file

O3 Magazine - Open-Source SSL Acceleration

SSL optimization and security

HTTPS performance tuning

Installing an intermediate authority certificate under nginx

Why SSL?

Outstanding issues

Cookies that contain the original site's domain will not be stored by the user's browser. Most websites don't set a domain within the cookie, so this isn't usually a problem. However, if your site is affected, it can be fixed by filtering the headers from the insecure site and removing domains. I'll post a blog entry in a few days about this, as it affects most Drupal sites including QatarLiving.

Navigation within a proxied site will only be successful if most links do not include the original hostname.

Thursday, June 19, 2008

News site rankings

Before I started QatarJournal in April last year, I took a snapshot of the Alexa page views from local news providers. Looking at the changes over the past year, it's obvious that QatarLiving is now serving more pages than any of the other news sites, and Watan's website is declining in relative terms against the other papers. Permalink here. A similar decline can be seen in terms of Watan's market share, with only QL showing a noticeable increase.

When we look at only visitors from Qatar, the ranking of sites providing news becomes clearer:
* Qatar Living
* BBC
* Al Raya
* Al Jazeera
* Al Sharq
* India Times
* The Rest.

Thursday, June 05, 2008

QatarJournal.com sold

Today I sold QatarJournal.com to a local businessman. I probably wouldn't have sold the site in January - I was quite upset about it. Now, it just seems like someone has given me a surprise gift. I'm going to have a good holiday this summer.

Wednesday, March 05, 2008

Back!

Qatar Journal is finished, so I'm back blogging here.

At the moment, I'm working on a CGI-IRC gateway, a cleanup of another codebase. It's mostly just a matter of replacing bits of code with CPAN modules, but after it's fairly tidy I'm going to replace the guts of the code with POE::Component::IRC by Chris Williams. That's mainly because I don't trust the IRC code I'm working with, and the POE code looks like it's well maintained.

The idea is that eventually, I'll authenticate web users with OpenID and some sort of NickServ. I haven't quite worked it out yet. Should be a fun bit of work, and very, very web 2.0 ;)

Monday, June 18, 2007

This blog has moved

There's an idiom in the software industry: "eating your own dog-food". If you've built some software, it's not right to ask other people to use if you don't. Hence, I've moved my blog to Qatar Journal.

Every user of Qatar Journal can have a blog on the site, so if you'd like one, you need to visit the signup page, choose a username, enter your email address (twice) and click 'Create Account'. Users can also choose to receive a nightly summary of all the local news headlines.

Sunday, June 17, 2007

Is Qtel a bit simple?


I've been banned from QatarLiving. I'm sure the ban wasn't intentional, but I've been banned nonetheless. This morning, it was obvious that QatarLiving discussions were being attacked by someone with the intent of disruption. There was some speculation about who was responsible. Eventually, someone at the website banned the IP address of the attacker.

Unfortunately, Qtel employs a few Internet filters, and everyone behind a particular filter has the same public IP address. Thus, a system which is meant to make the web safer for Qatar's surfers has rendered one of the most popular discussion sites in Qatar completely inaccessible. This has happened before, of course. At the beginning of the year, Wikipedia saw a fair amount of abuse coming from a single IP address, and blocked the entire country from anonymous posting. It's not as if people don't know a solution to these problems - it's just that no one at Qtel is willing to make the fix.

Setting up a decent filtering proxy should be trivial, but Qtel's implementation masks users' true IP addresses from the sites they visit, and restricts the ability of site operators to punish abusers. How simple is Qtel? I'd say it's plain stupid.

Sunday, May 27, 2007

Best blogs in Qatar

<BLUSH> This blog has been chosen as one of the best blogs in Qatar. I'm choking back the tears, but will still manage to give a 10-minute acceptance speech, given half a chance.

Thursday, May 24, 2007

Qatar Journal - The Online Newspaper for Qatar

At the end of its first week, I'd say the launch of Qatar Journal has been a success. It's taken less than two months from the initial idea through to launching the site. I had hoped to launch a couple of weeks earlier, but I wanted to be in Qatar when it happened.

The response has been good from journalists and PR companies. I'm now getting a steady stream of stories, which is easing the administration of the site.

The technology has had some teething problems, with my server running out of memory on the first night, but it's now settled down. I've got a very long bug-list, which I'll start fixing in a couple of weeks. None of the bugs are serious, just annoying. Ben Figgis has pointed out a bug in the Qatar Journal feed that doesn't seem to be affecting slashdot, so I'll need to investigate. It's nice when people point out a problem, because noticing the bug is the difficult bit. Fixing is usually easy. Thanks Ben.

Thursday, May 10, 2007

Still in the UK

Sorry, I've been away from a PC for the past two weeks. I'm still in the UK because Sara has had surgery. She's OK, and is getting out of hospital today, but we can't return to Doha until at least May 20.

Wednesday, April 18, 2007

Qatar Journal

I had a leisurely coffee with Qatari from QL last Thursday. We were talking about the state of the IT sector in Qatar, and generally solving the world's problems. Anyway, he was hassling me saying I should have some sort of commercial project, because there was money to be made.

I'm a bit skeptical about making money from websites. There were a lot of good ideas during the dot-com boom that produced no cash. He started quoting some figures about google ads, and how little you need to spend on hosting. Anyway, I went home that night and had a chat with Sara about it.

There's a gap at the moment for serious local news. QL and others serve the demand for informal chat, but the local papers have abysmal websites and mainstream news sites don't cover Qatar. Thus, I'm going to start up a news site, named Qatar Journal. The official launch will be on 12 May 2007 (exactly one month after my chat with Qatari) - it will be rough around the edges, but it's better to launch quickly and fix things as we go along.

Basically, there's going to be no editorial. The easiest way to scare advertisers is to start getting cynical about press releases, so I'm going to leave all discussion to the readers. Obviously, there's going to be an editorial element in choosing stories, and here I have an advantage over the newspapers' sites: I only need to pick around 10 stories a day, whereas they are publishing (almost) everything.

Anyhow, I just wanted to blog this before anyone starts asking - "why are you trying to compete with QL?". I'm not. I'm a regular QL reader, and it's great. Qatar Journal is an online local newspaper, and when it generates its own community, it will have a very different tone to QL. Hopefully, some readers will visit both sites, but I would guess that most QL regulars will stick with what they know - I'm not trying to poach anyone.

Let me know what you think about the site, and keep an eye on progress. Everything should be working very soon, but it'll take a couple of weeks to get the site looking professional.



Also, discovered Alexa today. Here is an image snapshot of Qatar websites (April 2007), and a permalink for future comparison.

Sunday, April 15, 2007

First TC meeting

So, Gord and I are at the security conference for a few days. Here's a few things I've learned:
  • The current malware products are sophisticated - the HackDoor client looks a well-designed piece of software engineering. I'd always assumed they would be fairly naive. The server side looks simpler, but then server programming is simple ;)

  • Phishing scams have avoided those banks which have implemented two-way authentication on their websites.

  • Firewire ports and PCMCIA slots have direct memory access, so can be used to copy an image of your computer's RAM even if no one is logged in. This can recover useful forensic material even after a reboot cycle, as modern BIOS's don't clear RAM. Eek.

  • Vista has an option for encrypting the filesystem. However, files are cached in RAM in an unencrypted state. Eek.

  • There are no data, from anyone, on the scale of the IT security problem that exists in the gulf. Honestly.

There aren't many delegates from Qatar at the conference beyond the staff of Q-CERT. That's a shame, and I can't work out whether it's down to a lack of interest or poor advertising. I'm sure if they'd mentioned the free Ritz lunches, the attendance would have been much higher.

Monday, April 09, 2007

iGov ictQatar IT Forum

TOO MANY ACRONYMS! The Gulf Times (and probably every other paper) today announced there's going to be a meeting at the Ritz tomorrow to talk about developing government IT services so they can be accessed through the web. I could probably make some useful suggestions, if I were invited. But I'm not, and neither are you.

Why bother advertising a meeting if it's not open to the public? That's like me issuing a press release saying "Family meeting, 7.30, around the dinner table. We're going to talk about the state of the bathroom." Of course, Wednesday's papers will follow up the story with photos of Hessa al-Jaber shaking some guy's hand. There won't be any details of what was discussed - just a photo and a repeat of the purposes of the meeting. Grr.

Given that I'm not invited, here are my suggestions. The important ones are non-technical.

  1. Scatter the money around. Don't spend millions on a single project to do everything - it will fail, and all your money will have been wasted.

  2. All initial projects should be finished within three months. A two-year project will fail, and you will be two years behind schedule.

  3. After three months, see who's got something working - give them more money and more responsibility. Stop funding the projects that haven't achieved anything; look for new small projects instead.

  4. Replace the leaders of failed projects. Publicise the successful projects.

  5. Insist on compliance to internet standards. If someone is using an ActiveX control, Macromedia Flash, or non-validating HTML, take away their money. They don't understand the web, and eventually you will be embarrassed if people believe their solution is representative of the State of Qatar.

  6. Insist that people use a free database in their solutions. It saves on costly licenses for still-born projects, and the database code will be cleaner if programmers believe they need to swap from MySQL to Oracle at a later date.

  7. Apart from following internet standards, and using free databases, don't enforce any other standards. In particular, enforcing Microsoft products will halve your pool of talented programmers; enforcing project management and system analysis methodologies will double the bureaucratic overhead. Both of these will encourage failure.

Thursday, March 15, 2007

Natashia

Today, I saw a photo of my niece, Natashia. She looks like a happy little kid, and I'll get to meet her at the end of April when we take a trip to the UK. Because I've never seen her, I sometimes forget that I have a niece, so it was a nice surprise to get an email this morning full of photos.

Sunday, March 11, 2007

Interview hell

I've had a few interviews recently. Well, no one is prepared to call them interviews: they're just quick chats; a get-together over a coffee; an informal introduction. If you're not offered a job, then you can't be disappointed because it's never made clear whether there was any job to be offered. You're just left asking yourself - 'was that an interview?'

When Sara was looking around for jobs, she'd come back from interviews and say "I thought it went really well - like we agreed about a lot of things", but I've never had an interview like that: it's always excruciating. I leave the interviewer to do a lot of the digging and I'm pitiful when it comes to selling my CV. It's not that I have low self-esteem. In fact, quite the opposite - I have a very-high opinion of myself. It's just that, in front of strangers, I'm incapable of talking in glowing terms about why I'm so wonderful. At the back of my mind is a voice shouting 'come off it, big-head'.

On Wednesday, I had my worst interview to date. I'm never too hot on introductions - I speak too fast and never know what to do with my non-shaking hand, but in this instance I also managed to criticise the interviewer's choice of school for his child. We were chatting (the awkward bit after shaking hands, but before sitting down) and it just came out. He asked - 'I hear you have a son - have you got him settled into a school yet?' I replied, rambling, that we'd been applying to schools, and there were basically three that were any good, and hopefully we'd got into the one that was within easy driving distance of the compound. Inevitably, his child's school was not among the three. I dug myself deeper - 'well, not to worry' I said.

It went from bad to worse. The first interviewer was American, but his Aussie boss arrived after about 10 minutes and asked me about what I could do. "Well - I'm basically a coder. I'm not bad at Perl, but I can get by in most languages. Most of my work has been on Unix, but I hear that there's no jobs for Unix coders in Doha, so I'll do anything really." He gave me a look that said 'no way, mate' and then asked "when you say you're a 'coder', do you mean that you're a computer programmer". He then did a mental calculation of how long he'd have to talk to me before leaving. I got another 15 minutes, during which time I told him that I hadn't done any real work in three years, and didn't have any salary expectations. At one point I said "really, most of my skills are outdated, and it's difficult to match my CV with any current job". The most embarrassing part was that we both knew how awful it was, and I'm sure he was trying to avoid laughing until I'd left the room.

When I met up with Sara afterwards, she asked how it went. 'Not too bad,' I said. The horror was still sinking in.

Tuesday, March 06, 2007

Are sites ever unblocked?

To: censor@qatar.net.qa

Sir

The following website has been blocked by the filter run by Internet Qatar:
http://taipei.pm.org/
I feel that this blocking is in error. As far as I am aware, this site
is devoted to discussing the merits of the Perl programming language,
and supporting users who live in the Taipei area.

I assume the site was blocked because your filtering software was
designed for use by the Chinese government, and a simple rule has been
enforced that blocks all Taipei sites.

Many thanks

Nigel Gourlay

Monday, February 26, 2007

Creating PDFs for the Sony Reader

Chris sent me a few pretty pictures. He also reminded me about the ebook reader from Sony, which takes a (closed) format known as BBeB as well as a 600x800 PDF. Finally, sourceforge's shell service has been repaired, so I can now upload again to readng.com.

Tomorrow, there's the first meeting of QISF - the Qatar Information Security Forum. Although it's going to be an introductory talk, with a very general topic, it'll be interesting to see how many people turn up for a get-together.

Friday, February 23, 2007

Slow progress

I spent some of last week with a potential employer; they're into packet radio and need someone to look after a bunch of computers - some of which were familiar and some of which were new to me. Interesting stuff, but they haven't made me an offer, so I won't be counting my chickens just yet.

Sara's been out of the country most of the week, but I managed to get along to the first QSTP tech-talk. It was an entrepreneur who talked about Arab start-ups, venture capital and incubators. There wasn't much talk about the IT side of things, but the audience was full of technologists from the new universities. It was a good place to chat, and I met a couple of people who I'll try to keep in touch with.

Not much has happened with readng. I've been wrestling with font issues on my SuSe system, which are now solved, and I've been trying to put together some sample pages for the site. Unfortunately, mocking up a single page with groff is proving almost as difficult as typesetting a whole book. I keep telling myself that the second book will be easy.

Thursday, February 15, 2007

QSTP tech-talks

Ben Figgis has invited Qatar Perl Mongers to the QSTP tech talks. RSVP to info@qstp.org.qa.

Monday, February 12, 2007

Unix text processing

My new task is to read the relevent chapters of UTP (1.7MB; source available here) and TUM. In particular, I need to get my head around troff, the Unix typesetting system (now, mostly known as groff). I'm convinced that I can secure groff and use it in the typesetting chain for readng.

Here's what I think will be the sequence: (1) ASCII or HTML; (2) groff mm or mom format; (3) postscript/PDF; (4) PNG image.

I know it's a little nasty, using an image format to display pages of text, but that's just tough. HTML in any flavour doesn't give the necessary control for pretty typesetting, and readers want books to look pretty.

Sunday, February 11, 2007

ReadNG - it's like reading, but without the 'i'

So, out of the much nonsense that was spoken over the past week, the silliest is probably this. After several beers with the camel gang, we decided that readng was the next flickr, and that all we needed to succeed in the web 2.0 world was poor spelling. I've registered the domain name readng.com, and written a blurb - let the flood of offers begin. It was Sara who was the first to note that maybe avid book readers might find intentional illiteracy rather annoying. What does she know. Pah.