Logging with the AWS SDK for .NET

As we announced previously, the AWS SDK for .NET now supports easily-configurable logging to both log4net and .NET’s built-in System.Diagnostics logging. In this post, we cover how to enable and configure this new functionality, and how you can easily collect performance metrics for the SDK.

Both log4net and System.Diagnostics logging approaches have their own advantages and disadvantages, but we’re not covering those here. Which approach you take depends on your specific needs.

Turning on log4net or System.Diagnostics logging, or even both, is now very simple to do: just add the appropriate XML sections to your application’s config file (app.config or web.config, for example), and you’re done! As long as you are using version 1.5.n or higher of the AWS SDK for .NET, there is no need to change any code or recompile your application. (To use log4net, remember that you must download the necessary binaries from the download site and place those alongside the AWSSDK.dll assembly.)

Configuring Logging

To make logging as easy as possible, we have introduced an application setting key, AWSLogging, that configures logging for the entire SDK. The key can be set to one of the following values:

“log4net” – only use log4net
“SystemDiagnostics” – log using System.Diagnostics
“SystemDiagnostics, log4net” – log to both system
“None” – completely disable logging

In this post, we demonstrate configuring this setting and the associated loggers.

Configuring log4net

The simplest approach to logging is to direct the logs to a file on your local system. The following configuration shows how to use log4net and direct all logs to the file located at C:Logssdk-log.txt.

<configSections>
  <section name="log4net" 
           type="log4net.Config.Log4NetConfigurationSectionHandler,log4net"/>
</configSections>

<log4net>
  <appender name="FileAppender" type="log4net.Appender.FileAppender,log4net">
    <file value="C:Logssdk-log.txt"/>
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern 
             value="%date [%thread] %level %logger - %message%newline"/>
    </layout>
  </appender>

  <logger name="Amazon">
    <level value="INFO"/>
    <appender-ref ref="FileAppender"/>
  </logger>
</log4net>

<appSettings>
  <!-- Configure the SDK to use log4net -->
  <add key="AWSLogging" value="log4net"/>
</appSettings>

If you are considering using log4net logging full-time, use RollingFileAppender instead of FileAppender. Otherwise, your logs may grow too large in size.

System.Diagnostics

You can implement the same kind of local-file logging with System.Diagnostics as well. Here’s a comparable configuration:

<configuration>
  <appSettings>
    <!-- Configure the SDK to use System.Diagnostics -->
    <add key="AWSLogging" value="SystemDiagnostics"/>
  </appSettings>

  <system.diagnostics>
    <trace autoflush="true"/>

    <sources>
      <source name="Amazon">
        <listeners>
          <add name="text" 
               type="System.Diagnostics.TextWriterTraceListener" 
               initializeData="c:Logssdk-log.txt"/>
        </listeners>
      </source>
    </sources>

  </system.diagnostics>
</configuration>

Metrics

The AWS SDK for .NET has recently started logging performance metrics for most service calls. These metrics can also be enabled through a configuration update. This is done using a switch similar to AWSLogging called AWSLogMetrics. This value is a simple boolean field that accepts “true” and “false” as inputs. Here is an example:

<appSettings>
  <!-- Enable SDK metrics logging -->
  <add key="AWSLogMetrics" value="true"/>
  <!-- Logging must be enabled to collect metrics -->
  <add key="AWSLogging" value="log4net,SystemDiagnostics"/>
</appSettings>

When you include this setting in your configuration, the SDK will collect and log performance metrics for every service call. Below is a sample metric log from log4net:

2013-02-24 00:11:59,467 [1] INFO Amazon.DynamoDB.AmazonDynamoDBClient - Request metrics: ServiceName = AmazonDynamoDB; ServiceEndpoint = https://dynamodb.us-east-1.amazonaws.com/; MethodName = ListTablesRequest; AsyncCall = False; StatusCode = OK; AWSRequestID = RRAMEONOVS6EEMP2GVMVDF59DJVV4KQNSO5AEMVJF66Q9ASUAAJG; BytesProcessed = 132; CredentialsRequestTime = 00:00:00.0015676; RequestSigningTime = 00:00:00.0134851; HttpRequestTime = 00:00:00.7279260; ResponseUnmarshallTime = 00:00:00.0057781; ResponseProcessingTime = 00:00:00.1004697; ClientExecuteTime = 00:00:00.8906012;

As you can see, we identify the .NET client issuing the request, the service, the endpoint, the method, whether this is an asynchronous call, and the response. We also identify timings, including how long the HTTP request took to complete and the total SDK call time. Note that logging must be enabled in order for any metrics to be collected.

Logger Hierarchy

You can use logger hierarchies for both log4net and System.Diagnostics to choose which SDK services will log events. You do this by configuring the loggers based on the namespace of the source. The earlier examples we covered will capture logs for all service calls, as they reference “Amazon”, which is the hierarchical parent of all loggers in the SDK. If you were only interested in DynamoDB logs, for instance, you would reference “Amazon.DynamoDB”. And since there are multiple logging options and both log4net and System.Diagnostics support multiple listeners/loggers, it’s even possible to configure for different services to be logged to different destinations.

Below, you can see how easy it is to configure log4net to capture Route53 logs and System.Diagnostics to handle only DynamoDB messages.

log4net:

<logger name="Amazon.Route53">
  <level value="INFO"/>
  <appender-ref ref="FileAppender"/>
</logger>

System.Diagnostics:

<source name="Amazon.DynamoDB">
  <listeners>
    <add name="text" 
         type="System.Diagnostics.TextWriterTraceListener" 
         initializeData="c:Logssdk-log.txt"/>
  </listeners>
</source>

Summary

In this post, we covered the available logging approaches present in the AWS SDK for .NET. You learned how to configure logging with both log4net and System.Diagnostics, and how to configure both sets of tools to collect only the data you need. You’ve also seen how simple it is to gather metrics data for the SDK.

As a final note, both log4net and System.Diagnostics provide extensive approaches to logging, though we covered only a small subset of that in this blog post. We recommend checking out the log4net and TraceSource documentation to acquaint yourself with these technologies.

AWS Developer Tools Blog