Category Archives: .NET

New Relic Insights .NET Client

Fork me on GitHubNew Relic.NET

A lightweight C# client for New Relic Insights

  • Upload New Relic Insights events on-demand
  • Upload in batches, as a scheduled task
  • Minimal CPU usage (1 thread)
  • Proxy-aware

Upload a Single Event

Create the Event

Create a class that implements NewRelicInsightsEvent:

public class CustomNewRelicInsightsEvent : NewRelicInsightsEvent
{
	[JilDirective(Name = "name")]
	public string Name { get; set; }

	[JilDirective(Name = "count")]
	public int Count { get; set; }

	[JilDirective(Name = "unixTimeStamp")]
	public int UnixTimeStamp => 
           (int) DateTime.UtcNow
                 .Subtract(new DateTime(1970, 1, 1))
                 .TotalSeconds;

	///
	/// EventType is the New Relic Insights Event Grouping. It determines the
	/// database to which the event will persist.
	///

	///
	/// /// must be serialised
	/// in Camel case, in order to be correctly interpreted by New Relic
	/// Insights.
	/// /// /// Apply the following attribute to the
	///
	/// property in your implementation:
	/// /// /// [JilDirective(Name = "eventType")]
	/// ///
	[JilDirective(Name = "eventType")]
	public string EventType { get; set; }
}

Note: NewRelic.NET leverages Jil for serialisation. Though JilDirective attributes are not required for custom properties, the EventType property requires the specified JilDirective.

Initialise New Relic Metadata

NewRelicInsightsClient.Instance.NewRelicInsightsMetadata.AccountID = 
	"{New Relic Account ID}";
NewRelicInsightsClient.Instance.NewRelicInsightsMetadata.APIKey = 
	"{New Relic API key}";
NewRelicInsightsClient.Instance.NewRelicInsightsMetadata.URI =
	new Uri("https://insights-collector.newrelic.com/v1/accounts");

Upload the Event

New Relic Event Upload

    var customNewRelicInsightsEvent = new CustomNewRelicInsightsEvent
    {
        Name = "TEST",
        Count = id,
        EventType = "Test"
    };

Synchronously

    NewRelicInsightsClient.Instance.UploadEvents(
	new NewRelicInsightsEvent[]
	{ 
		customNewRelicInsightsEvent 
	}, 
	new HttpClientFactory());

Asynchronously

    NewRelicInsightsClient.Instance.UploadEventsAsync(
	new NewRelicInsightsEvent[]
	{ 
		customNewRelicInsightsEvent 
	}, 
	new HttpClientFactory());

Upload Events in Batches

New Relic Batch Upload

Start the Event Cache

if (!NewRelicInsightsClient.Instance.HasStarted)
{
	NewRelicInsightsClient.Instance.Initialise();
}

Add Events to the Cache

NewRelicInsightsClient.Instance
    .AddNewRelicInsightEvent(customNewRelicInsightsEvent);

Event Upload Frequency

Batch-upload occurs every minute, by default, unless a custom frequency is specified. The following example sets the upload-frequency to 10 minutes:

NewRelicInsightsClient.Instance.RecurringTaskInterval = 10;

Proxy Server

The following example indicates that a proxy should be leveraged when invoking HTTP requests to New Relic:

NewRelicInsightsClient.Instance.NewRelicInsightsMetadata
    .UseWebProxy = true;
NewRelicInsightsClient.Instance.NewRelicInsightsMetadata
    .WebProxy = new WebProxy(“127.0.0.1:8080”);

Custom HTTP Timeout

Outbound HTTP requests to NewRelic are restricted to a specific timeout (5 seconds) as follows:

NewRelicInsightsClient.Instance.NewRelicInsightsMetadata
    .UseNonDefaultTimeout = true;
NewRelicInsightsClient.Instance.NewRelicInsightsMetadata
    .NonDefaultTimeout = new TimeSpan(0,0,5);

Otherwise, the default C# HttpClient-timeout applies.

Restricting Batch Upload Size

The default batch-upload size is 1,000 events. That is, a batch consisting of no more than 1,000 events will be uploaded upon each cache upload-cycle. The following example indicates that the entire cache should be emptied upon every upload-cycle:

NewRelicInsightsClient.Instance.NewRelicInsightsMetadata
    .CacheUploadLimit = int.MaxValue;

Connect with me:

RSSGitHubTwitter
LinkedInYouTubeGoogle+

HSTS Supercookies with ASP.NET

Fork me on GitHub
HSTS, or HTTP Strict Transport Security is essentially a means of ensuring that your connection is secure. It is a feature of modern browsers that is designed to prevent, for example, man-in-the-middle attacks, where you request a secure resource, such as https://mybank.com, and are redirected by a malicious 3rd party over a non-secure connection, to http://mybank.com. Note the missing “s” in the 2nd URL Scheme.

How HSTS Works

Browsers typically solve this problem by storing security preferences in a small data structure. In its simplest form, this is a key-value pair index, where the key is the resource URL and the value is a boolean variable indicating whether or not the connection to the associated resource should be established in a secure manner:

_____________________
 google.com    | 1 |
 bing.com      | 1 |
 apple.com     | 1 |
 ____________________

Note that the above example indicates that all requests to google.com, bing.com, and apple.com should be made in a secure manner, over HTTPS. We can infer then, that entries that do not exist in the HSTS database can be said to allow non-secure connections. If we were to view this in tabular-format, it would resemble the following, where entries for both http://yahoo.com and http://wordpress.com do not exist in the browsers HSTS database:

_____________________
 google.com    | 1 |
 yahoo.com     | 0 |
 bing.com      | 1 |
 wordpress.com | 0 |
 apple.com     | 1 |
 ____________________

When read as a single value, the complete sequence of boolean values for this table reads as “10101”. It is therefore possible to leverage this table to store arbitrary binary values.

However, it is the sights themselves that determine whether or not they should be accessed over a secure connection or not. This is achieved by returning a HTTP 301 response to requests established over non-secure channels. The HTTP response also includes a reference to the secure URL. The browser will honour this response by redirecting to the secure URL, which returns the following HTTP Header:

Strict-Transport-Security: max-age=31536000

Note that the above max-age parameter may be set as required; the above is simply an example.

The browser, upon receiving this response, will add an entry to its HSTS database, indicating that all future requests should be established over a secure channel.

How to Hack HSTS

In order to “save” a binary value to the HSTS database, we need to control the URL entries that will reside within the database. Let’s assume that I own the following 4 domains:

1.supercookies.com
2.supercookies.com
3.supercookies.com
4.supercookies.com

I configure each of these sites to indicate the connections should only be established over secure channels. Imagine then, that I create a website that contains a JavaScript file that creates a random 4-digit binary value – in this case, “1010″.

In order to “save” this value, my JavaScript file should contain a function that connects to both 1.supercookies.com and 3.supercookies.com. This will create the following entries in the HSTS database:

_____________________________
 1.supercookies.com    | 1 |
 3.supercookies.com    | 1 |
_____________________________

We can infer from this that taking into account both other domains, out view of each domain expressed in tabular format might represent the following:

_____________________________
 1.supercookies.com    | 1 |
 2.supercookies.com    | 0 |
 3.supercookies.com    | 1 |
 4.supercookies.com    | 0 |
_____________________________

In other words, by implementing a custom endpoint in each domain that simply returns a boolean value indicating whether or not the inbound HTTP request is secure or not will indicate to us whether or not there is an entry in the browsers HSTS database for that domain. For example, if we invoke a connection to http://1.supercookies.com (note the non-secure HTTP Scheme) then we would expect the browser to force a redirect to the secure equivalent of that URL (https://1.supercookies.com). Thus, if out endpoint returns a positive boolean, we can infer that this domain is present in our browsers HSTS database. Otherwise, the domain is not present, and our endpoint will return a negative boolean. By establishing connections to each domain, we can build a series of boolean values; in this case, “1010“.

Practical Example with ASP.NET Web API

Add the following ASP.NET Web API Controller method to write an entry to the HSTS database for the domain that hosts your ASP.NET application:

public HttpResponseMessage Write()
{
    HttpResponseMessage response;

    if (Request.RequestUri.Scheme.Equals("https"))
    {

        response = Request.CreateResponse(HttpStatusCode.NoContent);
        response.Headers.Add("Strict-Transport-Security", "max-age=3153600");

        return response;
    }

    response = Request.CreateResponse(HttpStatusCode.MovedPermanently);
    response.Headers.Location = new Uri(Request.RequestUri.AbsoluteUri.Replace("http", "https"));

    return response;
}

In simple cases, the above method simply returns a HTTP 301, that indicates to the browser to redirect to the secure equivalent of the origin URL. Upon redirecting, the browser receives the HSTS Header that results in an entry in the HSTS database for the domain that hosts your ASP.NET application.

Add the following method in order to read the HSTS entry (if present) for the domain that hosts your ASP.NET application:

public class HSTSResponse
{
    public bool IsSet { get; set; }
}

public HSTSResponse Read()
{
    if (Request.RequestUri.Scheme.Equals("https"))
    {
        return new HSTSResponse
        {
            IsSet = true
        };
    }
    return new HSTSResponse();
}

This method returns a positive boolean value if the inbound HTTP request is secure, implying that the upstream browser contains an entry in its HSTS database for the domain that hosts your ASP.NET application.

Generating Tracking IDs


It is not necessary to compile or run the source code – simply browse to the included index.html file in order to demonstrate the process. You can, of course, run the application locally if you wish.

The complete code leverages 4 external websites, as per the above example, in order to generate a binary value and indirectly store it in the HSTS database. Leveraging 4 external websites yields a total of 24 possible unique values – hardly enough to constitute a unique tracking mechanism. However, consider that if we own 32 external domains we can now control over 2.6 billion unique tracking IDs using this method. Note that the tracking ID in the sample code is rendered as Base-36 for legibility.

Why use the HSTS database as a storage mechanism

Cookies can be removed, edited, and faked. Leveraging the HSTS database as a storage mechanism potentially reduces the possibility that your tracking ID will be deleted. While this style of design is generally considered unscrupulous, the purpose of this post is to educate; whether or not this mechanism should be implemented in the wild is a matter of opinion that I leave up to the reader.

Connect with me:

RSSGitHubTwitter
LinkedInYouTubeGoogle+

Building Stateful Services with Azure Service Fabric

Fork me on GitHub
Azure Service Fabric offers two modes of operation: stateful and stateless. Both implementations allow for microservice-style application development. This tutorial focuses on building a simple, stateful microservice; that is, a microservice that maintains a degree of state between calls such that underlying objects retain their properties’ state after a client applies changes to the object.

Why Stateful

Stateful services provide the best of both worlds in terms of scalability and reliability. Azure Service Fabric encapsulates the scaling process to a great degree, by handling virtualisation and hardware-provisioning. The framework also handles load-balancing, failover, and state-synchronisation. The following is a step-by-step guide to implementing the simplest stateful service possible with Azure Service Fabric. The guide assumes that you have completed the steps involved in configuring your development environment.

1. Create a new Azure Service Fabric project

Create a new Azure Service Fabric project called “MyApplication”, selecting the Stateful Reliable Actor type from the list of available templates:

Stateful Reliable Actor Template

Stateful Reliable Actor Template

2. Add Custom Method Stubs

Locate the ISimpleActor” interface and note the existing method stubs. This interface and associated implementation is created automatically and includes two built-in method stubs. Add the following stubs so that the interface includes a “Name” property accessor and modifier:

    public interface ISimpleActor : IActor
    {
        Task<int> GetCountAsync();

        Task SetCountAsync(int count);

        Task<string> GetNameAsync();

        Task SetNameAsync(string name);
    }

Note that all method stubs are asynchronous by default. We have added two new methods; a simple name property accessor and modifier.

3. Add Name Property to State

Locate the ActorState class and modify so that the class includes a “Name” property:

        [DataContract]
        internal sealed class ActorState
        {
            [DataMember]
            public int Count { get; set; }

            [DataMember]
            public string Name { get; set; }

            public override string ToString()
            {
                return string.Format(CultureInfo.InvariantCulture, "SimpleActor.ActorState[Count = {0}]", Count);
            }
        }

4. Add Custom Methods

Add the following implementation methods to the SimpleActor class so that it satisfies the ISimpleActor interface:

        public Task<string> GetNameAsync()
        {
            return Task.FromResult(State.Name);
        }

        public Task SetNameAsync(string name)
        {
            State.Name = name;
            return Task.FromResult(true);
        }

5. Manage in Service Fabric Explorer

Deploy the application using Visual Studio:

Build -> Deploy Solution

Note that the application is now running locally in Service Fabric Explorer:

Azure Service Fabric Cluster Manager

Azure Service Fabric Cluster Manager

6. Create Proxy Client

Add a new Console Application to the solution and install the Microsoft.ServiceFabric.Services NuGet package:

Azure Service Fabric NuGet Package

Azure Service Fabric NuGet Package

Note: You may need to change the Target Framework property to 4.5.1, and also the Platform Target to X64:

Modify Target Framework

Modify Target Framework

Modify Platform Target

Modify Platform Target

Finally, include a reference to the SimpleActor.Interfaces project, and add the following to the Main method in the Program class:

            var actorId = ActorId.NewId();

            var simpleActor =
                ActorProxy.Create<ISimpleActor>(actorId, "fabric:/MyApplication");

            simpleActor.SetNameAsync("Bob");
            var name = simpleActor.GetNameAsync();

            Console.WriteLine("Hello, " + name.Result);
            Console.ReadLine();

Summary

Run the program and observe that the text “Hello, Bob” is printed to the command window. The Name property retains its state across calls. Subsequent accessor calls to the SimpleActorService will return the value “Bob”, unless that value is explicitly modified. Note that the value is synchronised across all running instances and nodes.

Connect with me:

RSSGitHubTwitter
LinkedInYouTubeGoogle+

.NET CLR Explained

Inside the CLR

Overview

This post aims to provide a summary cheat sheet that outlines the .NET CLR, its key components and features, and inner workings as a frame of reference for developers of every calibre.

Key Points

  • Provides services such as garbage-collection, exception-handling, debugging, and security
  • Compilers output metadata in a format recognisable by the CLR
  • Executes code that has been compiled by a .NET-compliant compiler
  • Automatic memory-management – arguably the primary feature of the CLR
  • Once compiled, all CLR-based languages interface seamlessly

Managed Execution

Managed Execution consists of the following steps:

  • Loading source code
  • Executing source code
  • Providing automatic memory-management through garbage-collection

Managed Code to MSIL

C#
Microsoft Intermediate Language is a common denominator to which all CLR-compliant languages are transformed upon compilation. MSIL can be described as follows:

  • A set of CPU-independent instructions
  • Contains instructions for loading, storing, constructing, and executing methods
  • Removes the need for type libraries or registry entires
  • Exposes metadata, used to generate native code, through portable files

MSIL to Native Code

Native code is code that can be interpreted and executed by a specific OS architecture. Generally speaking, any given application will target one or more OS frameworks. In order to run on these frameworks, compiled MSIL must be compiled further to native code, or code that is specific to a particular environment, or collection of environments. Native code can be described as follows:

  • Runs on OS architecture supported by the underlying MSIL compiler
  • MSIL must be compiled to native code that targets OS architecture
  • This can be achieved either prior to (AOT compiler), or during (JIT compiler) code-execution

JIT Compiler

  • Convert MSIL to managed code at runtime
  • Stores native code in memory
  • Does not load entire assembly
  • Stubs un-called MSIL directives until invoked
  • Loads MSIL stubs to memory when invoked

AOT Compiler (NGEN.exe)

  • Convert MSIL to managed code prior to assembly-execution
  • Loads the entire assembly
  • Persists the generated code to disk in the Native Image Cache

Code Verification

MSIL is subject to a verification process, unless explicitly overridden by policy. The verification process ensures that MSIL adheres to and upholds the contract defined by the CLR in terms of allowing managed-execution. This helps ensure that code is type-safe, and protected from corruption.

Summary

The following topics define brief descriptions of CLR key features

Running Code

  • Provides infrastructure to facilitate managed-execution of source code
  • JIT compiles methods as they are invoked
  • Subscribes executing assemblies to garbage-collection, security, debugging services, etc.

Automatic Memory Management

One of the most seismic paradigm shifts in the field of software development has been the advent of frameworks that automatically manage memory. This feature shields developers from the complexities and pitfalls of memory-management, removing complexity and increasing productivity. The following outline key features of automatic memory management in the CLR

Allocating Memory

  • Objects are saved to memory in a contiguous manner (side-by-side)
  • Pointers provide access to Reference Types on the Managed Heap
  • Access to objects is fast and reliable

Releasing Memory

  • Garbage Collector releases memory assigned to objects that are no longer in use
  • Application Roots point to objects currently in memory
  • Unreachable objects are removed from memory
  • Heap-compaction occurs in cases where significant numbers of unreachable objects exist
  • Objects over 85KB in size are assigned to the Large Object Heap

Generations and Performance

  • Managed Heap is divided into 3 Generations
  • Segmentation allows for more effective Heap-compaction
  • New objects are assigned to Generation 0
  • Objects that survive garbage-collection are promoted upwards through Generations
  • Garbage-collection occurs when Generation 0 is full
  • Most objects live and die in Generation 0
  • Certain objects, such as database connections, tend to survive multiple garbage-collection cycles
  • Objects in Generation 2 remain there until they are deemed to be unreachable

Unmanaged Resources

  • Require explicit removal from memory
  • Examples include handles to the OS file system, network connections, etc.
  • Clients explicitly release unmanned resource via Dispose methods

Connect with me:

RSSGitHubTwitter
LinkedInYouTubeGoogle+