Web Log Monitoring with Nagios

Every request to a web server leaves a record. Access logs show who asked for what and how the server answered. Error logs show what failed. On one server, that is a file you can read. Across load-balanced web servers, reverse proxies, and application nodes, it is thousands of files on dozens of machines, rotating away on a schedule.

Nagios Log Server centralizes those logs. It receives access and error logs from Apache, Nginx, and Microsoft Internet Information Services (IIS), parses them into searchable fields, indexes them for retention, and alerts when a query crosses a threshold you set. Nagios XI covers the other half, checking whether each site responds correctly and holding the resulting Host and Service states next to the rest of your infrastructure. Log Server is licensed separately from Nagios XI, by the number of Instances in your Cluster, with no per-gigabyte fees.

What Is Web Log Monitoring?

A web log is the record a web server writes about its own activity. Access logs list each request with a timestamp, client IP address, requested path, HTTP method, response status code, bytes transferred, and often the referrer and user agent. Error logs record failures: a missing file, a backend timeout, a permissions problem, a certificate that would not load.

Web log monitoring collects those records off the servers that produce them, turns the raw text into structured fields, and watches those fields for conditions that matter. That means three things: shipping logs somewhere central before rotation deletes them, searching months of history for a specific path or status code, and being notified when a pattern appears more often than it should.

Why Web Log Monitoring Matters

Availability checks tell you a site is answering. Logs tell you what it is answering with. A server can return HTTP 200 for its home page and 500 for the checkout endpoint that actually earns revenue, and an uptime check alone will never mention it.

  • Errors surface before users report them: A climbing count of 500 responses in the access log is visible in minutes, well before the first support ticket arrives.
  • Incidents get shorter: When logs from every web server sit in one searchable index, you narrow a problem to a path, a client, or a single node without opening a terminal session on each machine.
  • History survives rotation: Log rotation is essential for disk space and fatal for investigations. Centralized collection keeps the record after the local file is gone.
  • Broken links and bad deployments become measurable: A spike in 404 responses after a release points at the paths that changed.
  • Retention supports audit and review work: Configurable index retention and automatic snapshots preserve the evidence trail your reviewers ask for.
  • Traffic patterns become visible: Request volume by path, by status code, and by client location are dashboard panels rather than one-off scripts.

Web Log Monitoring Capabilities in Nagios Log Server

Web Server Log Collection From Apache, Nginx, and IIS

Log Server receives data through Inputs, which are listeners configured in the Logstash pipeline. A default syslog Input ships with every installation, and you can add Inputs for other ports, protocols, and file paths from Admin > Logstash Configuration > Global Config > Inputs. Global Config applies the same pipeline to every Instance in the Cluster, so a new Input does not have to be repeated node by node.

  • Linux web servers: The Add a Log Source menu generates a setup-linux.sh script already populated with your Cluster address and port. Running it configures rsyslog to forward to Log Server.
  • Windows and IIS: NXLog collects Windows Event Logs, and an im_file Input block pointed at an IIS log directory extends it to the log files themselves. Adding the new input name to the NXLog route sends those records to Log Server under a SourceModuleName field you can query on.
  • Existing file contents: Setting SavePos FALSE and ReadFromLast FALSE imports a log file’s current contents on the first run. Reverting to SavePos TRUE afterward prevents the whole file being resent each time the service restarts.
  • Encrypted transport: A syslog Input can be configured with TLS so log data crossing untrusted networks is encrypted in transit.

Reference: Monitoring a New Log Source With Nagios Log ServerConfiguring Inputs in Nagios Log Server, and Configuring NXLog To Send Additional Files.

Web Log Parsing and Field Extraction

Raw log lines are text. Filters turn them into fields, which is what makes searching and thresholding possible. Filters are defined alongside Inputs in Global Config and use grok patterns to match the structure of a log format and name each captured piece.

  • Apache error logs are parsed by default: Log Server ships with a filter that matches the apache_error program name against the %{HTTPD_ERRORLOG} pattern, breaking each entry into its timestamp, severity, client, and message.
  • Other formats are added as filters: Nginx and IIS formats, combined and custom access log formats, and application logs written by your own code are handled by adding a filter block with a matching pattern.
  • Filters do more than split text: They add fields, add tags, resolve IP addresses to DNS names, and drop entries you do not want indexed at all.
  • Configuration is verified before it goes live: The Verify action checks a new filter for validity, and Save & Apply puts it into the running pipeline. A configuration snapshot is created automatically when changes are applied.

Reference: Configuring Filters in Nagios Log Server.

Searching and Analyzing Web Server Logs

Once indexed, log data is queried from Dashboards. Indexing usually completes within about five seconds of arrival, which puts near real-time analysis of a site’s current behavior within reach.

  • Query languages: Lucene and JSON queries, with saved queries you can reuse across dashboards and alerts.
  • Drill-down filtering: Expanding any log entry in the All Events panel lets you add a filter that must or must not match any parsed field, narrowing a search without retyping it.
  • Panels for shape and volume: Events over time, top values, and tabular panels arranged in rows on saved, shareable dashboards.
  • Client geography: The Map panel plots access log entries by client location. Log Server includes the Logstash GeoIP plugin, and a built-in GeoIP filter you enable by toggling GeoIP (Default) to On in Global Config, then clicking Save & Apply. Available in Log Server 2024R2.0.3 and newer.
  • Plain-language queries: In Log Server 2026R1.4 and newer, natural language queries convert an English description into a Lucene query. This experimental feature requires your own OpenAI or Anthropic key, or a self-hosted model, and incurs cost with your provider. Only the query text, the current date and time, and field metadata are sent to the provider. No log messages are sent.

Reference: Analyzing Logs With Nagios Log Server and Using the Map Panel in Nagios Log Server.

Monitoring 404 and 500 Errors With Thresholds and Alerts

An alert in Log Server runs a saved query on an interval, counts the matching events within a lookback period, and compares that count against thresholds. This is the mechanism behind HTTP error monitoring: a query that matches 500 responses in your access logs becomes an alert when more than an acceptable number appear in the window you chose.

  • Check Interval: How often the query runs.
  • Lookback Period: How far back in the indexed data each run examines.
  • Warning and Critical thresholds: The match count that raises each state. With a Warning of 0 and a Critical of 2, more than zero matches is Warning and more than two is Critical. Below the thresholds, the alert is OK.
  • Absence of events: Setting both thresholds to 1 alerts you when expected log entries stop arriving, which catches a silent shipper as well as a silent site.
  • Alert methods: Email with customizable templates and macros, SNMP traps using the Nagios MIB, a custom script executed on the Cluster, or a passive check submitted to Nagios XI or Nagios Core over Nagios Remote Data Processor (NRDP).
  • Suppressing recovery noise: The Only alert when Warning or Critical threshold is met option controls whether you hear about the return to OK.

Alerts based on a saved query copy that query at creation time. Editing the original query on the Dashboards page does not change the alert. Update the alert itself through Advanced (Manage Query).

Reference: Alerting On Log Events in Nagios Log Server.

Bringing Web Log Checks Into Nagios XI

Log data becomes ordinary monitoring once it reaches Nagios XI, where it sits next to CPU, disk, and HTTP response checks and inherits your existing notification and escalation rules.

  • Nagios Log Server Wizard: Navigate to Configure > Configuration Wizards and search for log server. The wizard takes your Log Server address, web interface URL, and API key from the Log Server My Profile page, then creates Service objects from existing or new queries with their own Duration, Warning, and Critical values.
  • Additive host naming: Entering an existing Host name in the wizard attaches the new checks to that Host, so a check for 404 status codes lands on the same web server object that already carries its HTTP and certificate checks.
  • Alerts as passive checks: NRDP alerts arriving from Log Server are passive check results. Nagios XI needs Service objects to receive them, and can create Services for results it has already received.
  • Log file watching on the host: Where you want a pattern match handled on the web server itself rather than centrally, Simple Log Watcher (Swatchdog) watches a file and submits a passive result to Nagios XI through the Nagios Service Check Acceptor (NSCA) add-on. Swatchdog is a separate Perl utility, installed on the monitored host rather than shipped with Nagios XI.

Reference: Monitoring Nagios Log Server Queries With Nagios XI and Log Monitoring With Swatchdog in Nagios XI.

Web Log Retention, Indices, and Archiving

Log Server writes one index per day into the OpenSearch database, which makes aging out old data a matter of policy rather than surgery. Retention is set under Maintenance Settings.

  • Close indexes older than: Closed indexes keep their disk space but release other system resources and must be reopened to be searched. The default closes indexes older than 60 days.
  • Open index ceiling: Each Instance supports a maximum of 1,000 shards worth of open indexes. Each daily index uses 5 shards, and a few indexes exist for settings and internal data, so a single Instance holds fewer than 200 days of open indexes. Adding Instances to the Cluster raises the ceiling.
  • Delete indexes older than: Permanent. A deleted index is recoverable only from an archived snapshot.
  • Snapshots: Snapshots are written to a Snapshot Repository that every node in the Cluster must be able to reach. Snapshots older than the configured age are deleted, with a default of 720 days.
  • Enable Maintenance and Snapshots: Scheduled maintenance jobs also create snapshots, and this setting defaults to No. Turn it on if you want either.
  • Redundancy: Each index is created with 5 primary shards and 1 replica. A single-node Cluster has nowhere to place the replica shards and therefore no redundancy. Three or more Instances give you mirrored copies spread across the Cluster.

Reference: Managing Indices in Nagios Log Server and Managing Snapshots and Maintenance in Nagios Log Server.

Security, Access Control, and Multi-Tenancy

Web logs contain client IP addresses, requested URLs, and session identifiers, so who can read them is a real question.

  • Multi-tenancy: Administrators define which log sources each user can see and what administrative capability they hold, so an application team reviews its own web servers without access to everything else in the Cluster.
  • Directory authentication: Users can be authenticated and imported from Active Directory and LDAP, with SSL or TLS on the directory connection.
  • Encrypted collection: Configure a syslog Input with TLS to encrypt log data in transit from the web server to the Cluster.
  • Dropping sensitive fields: A filter can drop entries or remove fields before indexing, so data you do not want stored is never written.
  • Audit trail: The audit log records administrative activity in Log Server and has its own retention setting.

Reference: Configuring Syslog With TLS in Nagios Log Server and Configuring Multi-Tenancy in Nagios Log Server.

Benefits of Nagios Web Log Monitoring

Answers instead of file paths. The question during an incident is rarely “what does this log say.” It is “which of our eleven web servers is returning 502, starting when, for which route.” One indexed repository with named fields answers that in a query. Eleven terminal sessions and a pile of grep answers it eventually.

Alerts you actually keep enabled. Thresholds are counts over a window rather than single-line matches, so a query for 404 responses does not page anyone the moment one crawler requests a missing favicon. Tuning the lookback period and the Warning and Critical counts is what separates a signal from notification fatigue.

One escalation path for the whole environment. Sending log alerts into Nagios XI over NRDP puts them under the contacts, time periods, escalations, and acknowledgements you already maintain. A web log condition reaches the on-call engineer the same way a failed disk does, which means nobody has to remember a second set of rules.

Evidence that outlives the incident. Retention is a setting, not a hope. Daily indexes, configurable closing and deletion ages, and snapshots to a shared repository mean the request history for last quarter is still there when someone asks about it, and the storage cost of keeping it is predictable.

Growth by adding Instances. Ingest capacity, open index headroom, and data redundancy all scale by adding Instances to the Cluster. Licensing follows Instance count rather than data volume, so a busy month of traffic does not change what you pay.

Web Log Monitoring Prerequisites

  • A licensed Nagios Log Server Cluster. One or more Instances, reachable from every web server that will send logs. Storage sizing should be planned before ingest begins.
  • Network path and open ports. Whichever Input you use must be reachable. The Linux setup script defaults to TCP 5544. Firewall rules are needed on each Instance for the ports your Inputs listen on, and on Inputs defined in Global Config that means every Instance in the Cluster.
  • Privileged ports. In Log Server 2024R2.0.3 and newer, and in any version of Log Server 2026, an Input on a port below 1024 such as 514 needs no extra configuration. Create the Input as usual.
  • Root access on Linux web servers. The setup-linux.sh script is run as root and writes /etc/rsyslog.d/99-nagioslogserver.conf. If the script populates a DNS name rather than an IP address, the web server must be able to resolve it.
  • Correct rsyslog working directory. When configuring rsyslog manually, $WorkDirectory must point at the real rsyslog spool directory, commonly /var/lib/rsyslog or /var/spool/rsyslog. An incorrect path stops the rsyslog service from starting.
  • Administrator access on Windows web servers. NXLog is installed and its configuration file at C:\Program Files (x86)\nxlog\conf\nxlog.conf edited, then the nxlog service restarted from services.msc.
  • Read access to the log files. The collecting service needs permission to read the IIS, Apache, or Nginx log directory you point it at.
  • A Snapshot Repository, if you want snapshots. A path accessible to all nodes in the Cluster, owned by the nagios user. Maintenance and snapshot processing is disabled by default.
  • For Nagios XI integration. Either an NRDP token configured in Nagios XI for inbound alerts, or your Log Server API key from My Profile for the Nagios Log Server Wizard. Never place a real key in a shared configuration note. Use a placeholder such as <API_TOKEN>.

How To Monitor Web Server Logs With Nagios Log Server

  1. Add the log source. In Log Server 2026, click the green + in the left menu. In 2024R2, use the green + Add Log Source button at the top of the Home page. Choose Linux or Windows and follow the generated setup, which is prepopulated with your Cluster address and port. Confirm arrival in the Verify Incoming Logs section, or by running a host query such as host.ip:<IP.address> on the Nagios Log Server Search dashboard.
  2. Point the collector at your web server log files. Syslog collection alone does not include Apache, Nginx, or IIS log files. On Windows, add an NXLog im_file input for the log directory and add that input name to the NXLog route. On Linux, add a file Input in Global Config or configure rsyslog to read the file, then restart the collecting service.
  3. Parse the logs into fields. Navigate to Admin > Logstash Configuration > Global Config and open the Filters tab. Apache error logs are handled by the default apache_error filter. For access logs and other formats, add a filter with a grok pattern that matches your log format. Enable the GeoIP (Default) filter here if you plan to use the Map panel. Click Verify, then Save & Apply.
  4. Build the queries you care about. On the Dashboards page, write and save queries for the conditions worth watching: 500 responses, 404 responses by path, requests exceeding a response time field, or specific error log messages. Add panels so the shape of normal traffic is visible before you set a threshold.
  5. Create the alert. Under Alerting, click Alerts, then New Alert. Select the saved query, set the Check Interval and Lookback Period, and set Warning and Critical counts based on what your dashboard showed as normal. Choose an alert method: email, SNMP trap, script, or NRDP to Nagios XI. To carry these checks into Nagios XI as Service objects instead, run the Nagios Log Server Wizard from Configure > Configuration Wizards.

Full procedures: Monitoring a New Log SourceConfiguring FiltersAlerting On Log Events, and Monitoring Log Server Queries With Nagios XI.

Where Nagios Fits for Web Log Monitoring

Nagios Log Server collects, parses, indexes, searches, retains, and alerts on log content. Nagios XI checks whether sites and services are responding and holds Host and Service state, including passive results and query-based Services fed from Log Server. Flow analysis using NetFlow, sFlow, J-Flow, and IPFIX, along with packet capture, Nmap scanning, and intrusion detection, belongs to Nagios Network Analyzer. A single view across multiple Nagios XI, Log Server, and Network Analyzer servers is Nagios Fusion. Log Server, Network Analyzer, and Fusion are licensed separately from Nagios XI.

Some things are outside this scope, and it is worth being direct about them:

  • Alerting compares counts to values you set. Log Server does not identify anomalies, spoofing, or hijacking on its own. It reports how many events matched your query in your window, against your thresholds.
  • It does not secure the application. Logging that a request looked hostile is not the same as blocking it. Request filtering and injection prevention belong to the application and to a web application firewall.
  • Retention supports compliance work rather than delivering compliance. Configurable retention, snapshots, exportable reports, and an audit trail give reviewers the evidence they ask for. The obligation remains with your organization.
  • Deep application performance tracing is a different category of tool. Per-request code profiling, distributed traces, and statement-level database timings are not what log indexing does. What Log Server gives you is the response the server recorded and any timing field your log format includes.
  • Marketing web analytics is a different category as well. Sessionization, funnels, attribution, and visitor identity are not derived from server logs here. Request counts, status codes, referrers, user agents, and client geography are.

Frequently Asked Questions

How do I monitor server logs with Nagios?

Install Nagios Log Server, add each server as a log source so it forwards its logs to the Cluster, add a filter that parses your log format into fields, save a query for the condition you care about, and create an alert on that query with Warning and Critical thresholds. For a single file on a single Linux host where central collection is not warranted, Simple Log Watcher (Swatchdog) can watch the file and submit a passive result to Nagios XI through the Nagios Service Check Acceptor (NSCA) add-on.

Can I get email alerts for 404 and 500 errors?

Yes. Save a query in Log Server that matches the status code in your parsed access logs, then create an alert on it with a Check Interval, a Lookback Period, and Warning and Critical counts. Email is one of the available alert methods, and you can build custom email templates using macros such as %state%. Sending the alert to Nagios XI over Nagios Remote Data Processor (NRDP) instead routes it through your existing contacts and escalations.

Does Nagios XI collect web server logs by itself?

No. Centralized log collection, indexing, search, and retention are Nagios Log Server capabilities. Nagios XI monitors whether sites and services respond, using the Website Wizard, Website URL Wizard, and Web Transaction Wizard for HTTP response validity and load time, DNS resolution, page content, SSL certificates, and multi-step transactions. Nagios XI then displays log-derived checks as Services once Log Server sends them or the Nagios Log Server Wizard creates them.

Does Nagios Log Server work with Nginx and IIS logs?

Yes. Collection is format-agnostic: Log Server receives whatever the shipper sends. What differs is parsing. Apache error logs are parsed by a filter that ships with Log Server. Nginx and IIS formats are handled by adding a filter with a grok pattern matching your configured log format. IIS log files are collected on Windows by adding an NXLog im_file input for the log directory.

Do I need a separate license for Nagios Log Server?

Yes. Log Server is licensed separately from Nagios XI, based on the number of Instances in your Cluster rather than data volume. Each license key covers two installations: a production install and a test or lab install. Deviating from those two use cases is a violation of the license terms. For questions about license levels, contact [email protected].

How long does Nagios Log Server keep web logs?

As long as you configure. Log Server creates one index per day and Maintenance Settings control what happens to older indexes. By default, indexes older than 60 days are closed, which keeps them on disk and out of active resources until reopened. Deletion is a separate setting and is permanent unless an archived snapshot exists. A single Instance supports a maximum of 1,000 shards worth of open indexes, which works out to fewer than 200 days of open indexes because each daily index uses 5 shards. Adding Instances raises that ceiling.

Is Nagios Log Server a SIEM?

No. Log Server centralizes, parses, indexes, searches, and alerts on log data. It does not include correlation rules, threat intelligence, behavioral analytics, or automated response. Security teams use it as the searchable record of what happened and as an alerting layer over queries they write themselves.

Can a Log Server 2026 Instance join an existing 2024R2 Cluster?

No. Log Server 2024R2 runs OpenSearch v2 and Log Server 2026 runs OpenSearch v3, so a 2026 Instance cannot join a 2024R2 Cluster until every node has been upgraded. Upgrade the master Instance first, then proceed one node at a time, and confirm all nodes appear correctly under Administration > System > Instance Status. Moving from Log Server 2024R1 is different again: 2024R1 uses Elasticsearch, so an inline upgrade is not possible and the migration tool built into 2024R2 is used instead. Snapshots created before 2024R2 cannot be converted and will not migrate.

What happens to my web logs when the log file rotates?

Entries already forwarded to Log Server are indexed and unaffected by rotation on the web server. That is the main reason to centralize. On Windows, keep SavePos TRUE in the NXLog input so the collector tracks its position and does not resend a file from the beginning after a service restart.

Related Solutions

Log Management

Web and Server Monitoring

  • Server Monitoring for the CPU, memory, disk, and service checks that explain why a web server started returning errors.
  • All Nagios Solutions for the full list of monitored technologies.

Network and Traffic Visibility

Start Monitoring Your Web Server Logs

The logs are already being written. The work is getting them somewhere searchable before they rotate away, naming their fields so they can be queried, and setting thresholds on the few conditions that should wake someone up. Nagios Log Server does that, and Nagios XI puts the result next to everything else you already watch.

The fastest way to see whether that fits your environment is to look at it with your own log formats, retention requirements, and Cluster sizing on the screen. Our team will walk through exactly that with you.

Centralize Your Logs with Nagios Log Server

Collect, analyze, and visualize log data from across your infrastructure.