Intro to .NET Core at Music City Code

A few months ago we ported our .NET Web API application to .NET Core.  We started with .NET Core after RC1 came out and then migrated to RC2 and 1.0 as they came out.  It's been interesting to say the least.  With .NET Core 1.0 out now things are starting to calm down a bit and it's a great time to start looking at .NET Core if you have been holding off in the past.

Since we were early to jump on the .NET bandwagon, today I gave a talk on .NET Core at Music City Code.  If you are interested in the slides or the source code to my demo project I used during my talk the links are below.  If you are looking to get started with .NET Core download it and check out the documentation.

Download the slides

View the source on GitHub

MongoDB Quarter Aggregation

Since last spring I've been working on a new project using MongoDB as the backend data store.  We're using MongoDB's aggregation framework for our data analysis and one thing we are frequently doing is slicing data by date parts, such as months, days, weeks, etc, for further aggregation and analysis.  The aggregation framework includes date operators to help with extracting date parts from date fields, but there is no operator to extract a three-month business quarter.  This may be in part because the definition of when quarter starts and ends can vary.  For us, the first quarter starts on January 1st and the last quarter ends on December 31st.

By joining some math and string operators together you can extract the business quarter from a MongoDB date field.  Be warned, however, that this returns the quarter number as a string, not as a number as other aggregation framework date operators do.

db.dates.aggregate([
 {$match: {date: {$exists: true}}},
 {$project: {dateQuarter: {$substr: [{$add: [{$divide: [{$subtract: [{$month: "$date"}, 1]}, 3]}, 1]}, 0, 1]}}}
])

For this example, the query could be executed on a collection of documents containing a single date field:

{
    "_id" : ObjectId("54baa73bfccd5e75836d0c6f"),
    "date" : ISODate("2015-05-20T01:20:13.551Z")
}

The first $match in the pipeline ensures that the document actually has a date field. This is important because without this you'll get a nasty error that MongoDB "can't convert from BSON type EOO to Date" coming from any documents in the collection missing the date field. It would be nice if the aggregation framework was smart enough to exclude any documents without the field on its own.

The second operation in the pipeline is the $project (this could be a $group operator as well) which extracts the quarter from the date.  The first step is to extract the month from the date field, then subtract 1 from the month number.  The result is then divided by 3 and 1 is added to the quotient.  This is where some trickery is involved.  Since there is no way to round a number in the aggregation framework, we are left with a few places to the right of the decimal we need to get rid of.  I used the $substr operator to remove everything but the first number in the result, the downside of this is the resuling dateQuarter field is now a string.  In our application having the quarter as a string wasn't a problem, so this approach worked for us.  You could potentially cast the quarter in your application code if necessary.

Testing RavenDB Indexes

RavenDB prefers performance over accuracy, meaning Raven will return results as fast as it can rather than waiting for other operations to complete.  Because of this testing out indexes can be a little tricky.  If you set up an integration test inserting some data into Raven and then begin querying data back out with an index, you might find that your index returns nothing!  How can it return nothing when you've just inserted data?  This is where you will need to instruct Raven to wait for indexes to finish rebuilding before returning results.

In my last post, I added

store.Conventions.DefaultQueryingConsistency = ConsistencyOptions.QueryYourWrites;

to the SpecsForRavenDb<T> class so that we can query data we have just inserted into Raven.  The problem with this is DefaultQueryingConsistency property does not apply to indexes.

To instruct Raven to wait for non-stale results when querying indexes, we can add add a listener our SpecsForRavenDb<T> class.

public class NoStaleQueriesListener : IDocumentQueryListener  
 {  
    public void BeforeQueryExecuted(IDocumentQueryCustomization queryCustomization)  
    {  
      queryCustomization.WaitForNonStaleResults();  
    }  
 }  
   

This simple class tells RavenDB before each query is executed wait for non stale results.  To hook it into our test base class, we just need to add one line to SpecsForRavenDb<T> before we initialize our EmbeddableDocumentStore:

.RegisterListener(NoStaleQueriesListener())

Since we are only applying our listener when testing, Raven will wait for non stale results in a test scenario, but will still use the default behavior of returning results as fast as possible when running in production.

Now our final SpecsForRavenDb<T> class looks like this:

    public abstract class SpecsForRavenDb<T> : SpecsFor<T> where T : class 
    { 
        private IDocumentStore _store; 
        public IDocumentSession RavenSession { get; set; } 
 
        public override void SetupEachSpec() 
        { 
            _store = new EmbeddableDocumentStore 
                { 
                    RunInMemory = true 
                }
                .RegisterListener(NoStaleQueriesListener())
                .Initialize(); 
 
            _store.Conventions.DefaultQueryingConsistency = ConsistencyOptions.QueryYourWrites;
            IndexCreation.CreateIndexes(typeof(MyProject.MyIndex).Assembly, _store);  
 
            RavenSession = _store.OpenSession(); 
             
            base.SetupEachSpec(); 
        } 
         
        protected override void ConfigureContainer(StructureMap.IContainer container) 
        { 
            container.Configure(cfg => cfg.For<IDocumentSession>().Use(RavenSession)); 
        } 
 
        protected override void AfterSpec() 
        { 
            base.AfterSpec(); 
            RavenSession.Dispose(); 
            _store.Dispose(); 
        } 
    }

Testing RavenDB

Lately I've been doing some development with RavenDB.  It's my first venture in to a "NoSQL" database and I've found it quite enjoyable to work with once you get out of the "relational" mindset.  One of the areas where Raven really shines is how easy it is to test.  Rather than mock out the RavenDB document store, you can spin up an in-memory version of RavenDB server, testing everything from Raven on up.

I usually test with the SpecsFor framework.  When I spin up my test I just create a new RavenDB in-memory EmbeddableDocumentStore and test with it.  Below is the base class I use with SpecsFor to handle spinning up and tearing down a new in-memory instance of RavenDB.

namespace MyProject.Test.Helpers 
{ 
    public abstract class SpecsForRavenDb<T> : SpecsFor<T> where T : class 
    { 
        private IDocumentStore _store; 
        public IDocumentSession RavenSession { get; set; } 
 
        public override void SetupEachSpec() 
        { 
            _store = new EmbeddableDocumentStore 
                { 
                    RunInMemory = true 
                } 
                .Initialize(); 
 
            _store.Conventions.DefaultQueryingConsistency = ConsistencyOptions.QueryYourWrites;
            IndexCreation.CreateIndexes(typeof(MyProject.MyIndex).Assembly, _store);  
 
            RavenSession = _store.OpenSession(); 
             
            base.SetupEachSpec(); 
        } 
         
        protected override void ConfigureContainer(StructureMap.IContainer container) 
        { 
            container.Configure(cfg => cfg.For<IDocumentSession>().Use(RavenSession)); 
        } 
 
        protected override void AfterSpec() 
        { 
            base.AfterSpec(); 
            RavenSession.Dispose(); 
            _store.Dispose(); 
        } 
    } 
}

Line 16 tells Raven to wait for writes to complete before returning results, more on this in my next post.
Line 17 tells Raven to go find the assembly we are testing and create all the indexes defined in the assembly.

The rest handles spinning up and disposing of the store after each test. Happy testing!