Nginx and Apache access logs: where to find them and how to read them

The nginx access log is usually at /var/log/nginx/access.log, and the Apache access log at /var/log/apache2/access.log on Debian and Ubuntu or /var/log/httpd/access_log on Red Hat-family systems. Both servers write one line per request in the same "combined log format" by default: client IP, time, request, status code, size, referrer and user agent. This guide shows how to find the log on any setup, read each field, change the format to add what you need, and answer common questions from the shell. It is the groundwork for analysing what search and AI crawlers do, which the guide to Googlebot and crawling puts in context.

Quick answer

• nginx: /var/log/nginx/access.log; the access_log directive in the nginx configuration sets the path and format.
• Apache: /var/log/apache2/access.log (Debian, Ubuntu) or /var/log/httpd/access_log (RHEL, CentOS, Fedora, Rocky, AlmaLinux); the CustomLog directive sets it.
• Shared hosting: cPanel → Metrics → Raw Access, or Plesk → Websites & Domains → Logs.
• Old logs are rotated to access.log.1, access.log.2.gz and so on; include them when you analyse.
• The combined format is IP - user [time] "request" status bytes "referrer" "user agent".
• Drop the file into the free log file analyser to read it without the shell; it runs in your browser and the file is not uploaded.

Where is the access log?

Every virtual host can have its own log. If one site on a server has a CustomLog or access_log line in its own configuration block, its requests go there instead of the default file, so search the configuration, not just the log folder.

Some platforms do not give you access logs at all. Shopify is one; many static hosts offer them only on higher plans. Behind a CDN, requests the CDN answers from cache never reach your server, so your log misses them.

Rotated logs

Logs are rotated by logrotate on most Linux systems: the current file is renamed to access.log.1, older ones are compressed to access.log.2.gz, access.log.3.gz and so on, and the oldest are deleted. How often and how many are kept depends on the settings in /etc/logrotate.d/nginx or /etc/logrotate.d/apache2. Copy what you need before it is gone.

# One month in one file, uncompressed
cat access.log access.log.1 > month.log
zcat access.log.*.gz >> month.log

The combined log format, field by field

Here is one request from Googlebot as nginx and Apache write it by default:

66.249.66.1 - - [26/Sep/2026:08:14:03 +0000] "GET /products/blue-mug?color=red HTTP/1.1" 200 18432 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.6668.100 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

The common log format is the first seven fields only, with no referrer or user agent. It is useless for bot analysis, because the user agent is how you tell Googlebot from a visitor; switch to combined if your server uses it.

Two small differences: Apache's %b writes - when no body was sent, while nginx writes 0; and Apache's %>s is the final status after internal redirects, where %s would be the original one.

The definitions in the configuration

Apache defines the formats with LogFormat in apache2.conf or httpd.conf:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
CustomLog ${APACHE_LOG_DIR}/access.log combined

nginx has combined built in; you cannot redefine it, but this is what it writes:

log_format combined '$remote_addr - $remote_user [$time_local] '
                    '"$request" $status $body_bytes_sent '
                    '"$http_referer" "$http_user_agent"';

Changing the log format

Default logs lack two things worth having: how long each request took, and which site it was for, if one server hosts several.

nginx, in the http block, then point access_log at the new format:

log_format timed '$remote_addr - $remote_user [$time_local] "$request" '
                 '$status $body_bytes_sent "$http_referer" "$http_user_agent" '
                 '$request_time $host';
access_log /var/log/nginx/access.log timed;

$request_time is the total time in seconds with millisecond resolution. Add $upstream_response_time too if nginx proxies to PHP or an app server and you want the back end's share.

Apache:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D %v" timed
CustomLog ${APACHE_LOG_DIR}/access.log timed

%D is the time taken in microseconds; %v is the virtual host name. Test the configuration (nginx -t, apachectl configtest) and reload the server. Keep the combined fields first and add new ones at the end, so tools that read combined logs still parse the lines.

For a log shipper or a data warehouse, nginx can write JSON lines with log_format name escape=json '{...}', which avoids problems with quotes in user agents and URLs.

Behind Cloudflare or a load balancer

If every line shows the same few addresses, the log is recording your proxy, not the visitor. Restore the real client IP before logging:

# nginx, ngx_http_realip_module: one set_real_ip_from line per proxy range
set_real_ip_from 173.245.48.0/20;
real_ip_header CF-Connecting-IP;

# Apache, mod_remoteip; then use %a instead of %h in LogFormat
RemoteIPHeader CF-Connecting-IP
RemoteIPTrustedProxy 173.245.48.0/20

Use your provider's full, current list of proxy ranges; Cloudflare publishes its ranges on its website. Without this, you cannot verify Googlebot, because every address you check belongs to the proxy.

Reading the log from the shell

With the combined format, awk splits each line on spaces, so $1 is the IP, $4 the time, $7 the path, $9 the status and $10 the size. The user agent contains spaces, so split on quotes to get it: awk -F'"' '{print $6}'.

# Watch requests as they arrive
tail -f /var/log/nginx/access.log

# Status codes, most common first
awk '{print $9}' access.log | sort | uniq -c | sort -rn

# The 20 most requested URLs that returned 404
awk '$9 == 404 {print $7}' access.log | sort | uniq -c | sort -rn | head -20

# The 20 most common user agents
awk -F'"' '{print $6}' access.log | sort | uniq -c | sort -rn | head -20

# Requests per hour for one day
grep '26/Sep/2026' access.log | awk '{print substr($4, 14, 2)}' | sort | uniq -c

# Search compressed rotated logs without unpacking them
zgrep 'Googlebot' access.log.*.gz | wc -l

For questions about crawlers, the Googlebot user agent guide has commands to split each Google crawler, and how to verify Googlebot shows how to check the addresses you find.

Reading it without the shell

The log file analyser reads Apache and nginx common and combined logs, including a virtual host in front and extra fields such as request time after the user agent, as well as JSON lines and IIS W3C logs. Gzipped files are unpacked in the browser. It groups every hit by bot, from Googlebot and Bingbot to GPTBot, ClaudeBot and SEO tools, with status codes, top URLs, wasted crawl and average response time when the log has it. Lines that do not parse are counted, so you can see if your custom format confused it.

What to do with the result, step by step, is in SEO log file analysis.

Privacy and retention

Access logs hold IP addresses, which count as personal data under the GDPR. Keep them only as long as you need them, limit who can read them, and mention server logs in your privacy policy. When you share a log with an agency or tool, prefer one that processes it locally, or remove the IP column first if the analysis does not need it.

Common mistakes

• Analysing one day. Rotation means access.log may cover hours, not weeks. Combine the rotated files.
• Using the common format. Without user agents you cannot tell bots from people.
• Logging the proxy's IP. Every request looks like it came from Cloudflare or the load balancer.
• Reading only the default file on a server where each site has its own access_log or CustomLog.
• Adding fields in the middle of the line. Parsers expect combined fields in order; append new ones at the end.
• Forgetting the server's time zone. +0200 in the timestamp matters when you compare with Search Console, which reports in Pacific Time.

Questions people ask

Where is the nginx access log?

On most Linux installs it is /var/log/nginx/access.log, with errors in /var/log/nginx/error.log. A site can set its own path with the access_log directive, so run sudo nginx -T | grep access_log to see every log file in the active configuration. In the official Docker image, the log goes to the container's standard output instead, readable with docker logs.

What is the combined log format?

It is the default access log format of Apache and nginx: client IP, identity, user, timestamp, request line, status code, response size, referrer and user agent, in that order. It extends the older common log format with the last two fields. Because it includes the user agent, it is the minimum you need to see which crawlers visit your site.

How do I add response time to the nginx access log?

Define a custom format with log_format in the http block that ends with $request_time, which is the total request time in seconds with millisecond resolution, then set access_log /var/log/nginx/access.log yourformat;. Keep the combined fields first so log tools still parse the line. In Apache, add %D for microseconds to the LogFormat line instead.

What does a dash mean in an access log line?

A dash means the value was empty or not available. The second and third fields are almost always dashes, because identd and HTTP authentication are rarely used. A dash in the referrer means the client sent none, which is normal for crawlers and direct visits. In Apache, a dash in the size field means no response body was sent, for example on a 304.

Why does my access log only show Cloudflare IP addresses?

Because Cloudflare connects to your server on the visitor's behalf, so the server sees Cloudflare's address. Restore the real one with nginx's realip module or Apache's mod_remoteip, reading the CF-Connecting-IP header and trusting only Cloudflare's published IP ranges. After a reload, new log lines show visitor and crawler addresses, which you need to verify Googlebot.