How to Collect Syslog Messages to Cloud Storage (S3)

TABLE OF CONTENTS

The syslog archive feature supported by Syslog Watcher can integrate with cloud storage to duplicate collected logs in a more secure remote location. Following the steps below, you can easily collect syslog messages to an Amazon AWS S3 bucket in just a few minutes. We are using Amazon AWS cloud as an example, and this method can be applied to any other cloud storage service.

Prerequisite 1

A configured and running Syslog Watcher server that collects syslog messages.

Prerequisite 2

An Amazon AWS S3 bucket for syslog messages and an AWS IAM user keys to access that bucket. AWS configuration instructions are beyond the scope of this guide. We will be happy to assist you through technical support.

Step 1: Turn on Syslog Archive in Syslog Watcher

You also need to specify the path to the local directory where the archive files will be created. More details in Syslog Watcher User’s Guid.

Syslog Watcher settings with Syslog Archive enabled
Syslog Watcher server refreshes the syslog archive every day at midnight. You force Syslog Watcher server to synchronize it immediately by restarting it.

Step 2: Install AWS CLI

AWS CLI allows you to perform actions with Amazon cloud services from the command line. If the utility is not already installed, download the installer, run it, and follow the default steps: https://awscli.amazonaws.com/AWSCLIV2.msi

Step 3. Configure AWS CLI environment

Run the command aws configure in the command line (cmd.exe) to set up your cloud service access keys. Example of AWS CLI configuration steps

Step 4. Test: Upload syslog to the cloud storage

Run aws s3 sync . YOUR_BUCKET in the storage archive folder to upload storage archive files to the cloud (S3 bucket). Command to test syslog archive synchronization with cloud storage

Step 5. Schedule daily archive synchronization

Use the built-in Windows Task Scheduler to run the s3 sync command daily. The following screenshots illustrate the process of creating a daily task.

Windows Task Scheduler - How to create a basic task Windows Task Scheduler - How to run the task daily Windows Task Scheduler - Task Configuration

By default, the synchronization process does not remove files from cloud storage. This allows you to save local disk space by limiting the syslog archive, for instance, to 30 days while keeping a much larger archive in cloud storage.

UPCOMING FEATURE
The new version of Syslog Watcher will allow you to run the script immediately after refreshing the local archive, so you will not need to configure Windows Task Scheduler.