Site: Nginx Config for Tornado

08/31/14

Okay, as stated previously, there is a configuration for plugging a tornado server into nginx. The goal of this post is to translate that magic into something I understand, and alter it according to my needs. Another post will just have the end result and explain what it does without the extra linkage to documentation and forum posts that I will inevitably post here. This is how I learned it, the next post will simply be how to do it while gaining a minimal understanding.

So, I'm going to go through it line by line, figure out what's going on, and present alterations for my needs.

Okay, line one:

worker_processes 2;

Googling "worker_processes nginx" led me here. So, it defines the number of worker processes. Why 2 for tornado? I don't know. The documentation says the value "auto" will try to autodetect will try to autodetect a good number (equal to the number of available CPU cores). That's good enough for me right now. (Though, I guess setting it to one would also be good enough for me right now.)

error_log /var/log/nginx/error.log;

Self-explanatory, I think, but might as well check the documentation. Yeah, sets a log file for errors. Though, it seems there's lots of flexibility in error setup here. Something to learn about more deeply at a much later date, maybe. Should be fine to leave as is for now.

pid /var/run/nginx.pid;

Defines a file that will store the process ID of the main process. Should be fine to leave as is.

events {
worker_connections 1024;
use epoll;
}

"Provides the configuration file context in which the directives that affect connection processing are specified." Uh.... Wat? Okay, the documentation isn't cutting it here.

Okay, here's a beginner's guide to nginx. Let me read that for you.

Reading....

nginx consists of modules which are controlled by directives specified in the configuration file. Directives are divided into simple directives and block directives.

Oh, yes, hmmmm, interesting.

Reading....

I think I get it. Where were we?

events {
worker_connections 1024;
use epoll;
}

Right, right. This kinda defines connection processing settings. So, 1024 connections per worker, and using epoll, an efficient connection processing method used on Linux 2.6+.

http {

Http directive. Where server connection rules will be defined

charset utf-8;

Using unicode.

    # Enumerate all the Tornado servers here;
    upstream frontends {
        server 127.0.0.1:8000;
        server 127.0.0.1:8001;
        server 127.0.0.1:8002;
        server 127.0.0.1:8003;
    ;}

This was one of the main things really confusing me before. It's basically saying, "These are a bunch of proxy servers to use." Nginx will be the middleman sending and retrieving messages from user to proxy server (in this case, tornado) and vice versa.

I probably won't use this bit of code for my setup. I don't think I'll need multiple proxy servers anytime soon.

include /path/to/nginx.mime.types;
default_type application/octet-stream;

Okay, that include is linking to a table mapping filename extensions to MIME types of responses. This is setting up the proper communication channels to tell a user's browser how to deal with content that your site delivers. The default_type command sets any content of unknown MIME type to application/octet-stream.

Fun bit of wikipedia trivia: MIME stands for "Multipurpose Internet Mail Extensions", a specification used to compose non-ASCII parts of email messages. Also known as a Content-type.

access_log /var/log/nginx/access.log;

Access log. So, like, log of activity of access to server from users, I'm guessing.

keepalive_timeout 65;
proxy_read_timeout 200;

Timeout stuff. You know, when you try to connect to a server for a bit and then it eventually gives up. This might be related to that.

sendfile on;

Enabling sending of files.

tcp_nopush on;

Allows sending of file header and beginning of file in one packet, and/or sending a file in full packets.

tcp_nodelay on;

Affects behaivior on processes that transition into the keep-alive state.

    gzip on;
    gzip_min_length 1000;
    gzip_proxied any;
    gzip_types text/plain text/css text/xml
        application/x-javascript application/xml
        application/atom+xml text/javascript;

Compression file handling setup.

    # Only retry if there was a communication
    # error, not a timeout on the Tornado server
    # (to avoid propagating "queries of death" to
    # all frontends)
    proxy_next_upstream error;

This part is already documented. Awesome.

server {

The directive where we define location rules for content delivery and stuff.

listen 80;

Listen on port 80 for incoming traffic (from the user).

        location ^~ /static/ {
            root /path/to/app;
            if ($query_string) {
                expires max;
            }
        }

So, in Tornado, it's standard practice to keep your static HTML files in a local directory "static". But nginx is perfectly capable of serving static content to the user, so this setup is having nginx do just that. That "^~" is saying "do not use regular expressions to find content that should be found under the /static/ prefix".

That if block is telling any browser that hasn't gotten the content already that it can cache it for the forseeable future (static data need not be refetched or anything).

location = / {

Okay, we're getting into the meat here. This is what sends stuff to the Tornado server. This is saying anything trying to access the root site or anything beyond not matching static should go through this location block.

proxy_pass_header Server;

Let's the proxied server pass the Server header through nginx to the client.

            proxy_set_header Host $http_host;
            proxy_redirect off;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Scheme $scheme;

Header setting configurations.

proxy_pass http://frontends;

Use an available server that we defined in the upstream frontent block directive above.

}
}
}

End location block, server block, and http block. End file.

Whew! That was a lot more taxing than I thought it'd be! But I feel I learned a lot tackling it this way, and maybe someone else will find this little bit of writing useful.

Next site update blog post will be my version of this configuration file, where I made changes and why.