Thursday, 23 June 2011

Magic Unicorn and attachment disorders

As previously noted, I’m a big fan of Entity Framework 4.1 – but it does have a few shortcomings/quirks that can take a bit of head scratching to overcome.  One of its features that is useful and annoying in equal measure is the ability to persist whole layers of complex objects in a single call.  This is great because…it can do it in a single call!  But there are scenarios where automatic persistence of everything is less than desirable.  A good example of this is lookups – let’s take Northwind’s Products table:

northwindproducts

The SupplierID and CategoryID fields are foreign keys (to the Supplier and Category tables).  If we were to create an object model around the product table, then we would probably create separate classes for the suppliers and categories:

public class Supplier
{
    public int Id { get; set; }
    public string CompanyName { get; set; }
    // ...other fields
}

public class Category
{
    public int Id { get; set; }
    public string Name { get; set; }
    public string Description { get; set; }
}

public class Product
{
    public int Id { get; set; }
public string Name { get; set; } public virtual Supplier Supplier { get; set; } public virtual Category Category { get; set; } public string QuantityPerUnit { get; set; } public decimal? UnitPrice { get; set; } public short? UnitsInStock { get; set; } public short? UnitsOnOrder { get; set; } public short? ReorderLevel { get; set; } public bool Discontinued { get; set; } }

However if we run this through Magic Unicorn and set the Supplier and/or Category property of the Product object to a supplier and/or category that isn’t already in the database, then EF 4.1. will helpfully create new rows in the appropriate tables for us:

using (var context = new NorthwindContext())
{
    var supplier = new Supplier()
    {
        CompanyName = "Random Co"
    };

    var category = new Category()
    {
        Name = "Expensive stuff",
        Description = "Things that cost a lot of money"
    };

    var product = new Product() 
    { 
        Name = "Money burner",
        Category = category,
        Supplier = supplier, 
        QuantityPerUnit = "lots of boxes", 
        UnitPrice = 20,
        UnitsInStock = 500,
        UnitsOnOrder = 0,
        ReorderLevel = 10,
        Discontinued = false
    };

    context.Products.Add(product);
    context.SaveChanges();
}

Whether this is a good thing depends on whether the “dependent” object being created is specific to the parent or just a general lookup value.  In this case it’s a one-to-many with category as the principal (i.e. there can be many products for each category) so it’s much better practice to separate the processes of persisting products, suppliers, and categories.  As an aside I should highlight what happens if we use values already in the database for the category and supplier:

using (var context = new NorthwindContext())
{
    var supplier = context.Suppliers.FirstOrDefault();
    var category = context.Categories.FirstOrDefault();

    var product = new Product() 
    { 
        Name = "Money burner",
        Category = category,
        Supplier = supplier
        // other fields...
    };

    context.Products.Add(product);
    context.SaveChanges();
}

As one would expect, Entity Framework patches up the references correctly and everything saves fine with no duplication of the ‘lookup’ values.  This is because when entities are retrieved/deserialised, Entity Framework actually returns a proxy (rather than the POCO).  It does this so that it can do all sorts of clever interception to support change tracking.  It uses this ‘special magic’ to work out whether objects already relate to rows in the database so that it doesn’t duplicate them – in fact it’s key (no pun intended) to the way that a good OR/M should work.

So far, so good.  What happens if we stick on a few data contract attributes and expose those POCOs/entities over WCF?  Well, once we get past the data contract resolver problem (for getting the objects out of the database and over SOAP), we hit the issue of sending back real POCOs (instead of proxies) and trying to match them up with database rows.  This can lead to some interesting errors:

Violation of PRIMARY KEY constraint 'PK_xxx'. Cannot insert duplicate key in object 'MyTable.

What this is saying is that there is already an object with the same key, but the one being persisted isn’t it.  So EF tries to create the object again but fails because it already exists.  So what we need to do is tell EF that “the object I’m giving you is the same as the database row with the same key”.  After unsuccessfully trying to use the Attach() method of the data context, which produced this error:

An object with the same key already exists in the ObjectStateManager. The ObjectStateManager cannot track multiple objects with the same key

I still don’t understand why this happens – I thought that the Attach() method was supposed to make the DbContext recognise the object.  After much gnashing of teeth and wailing around stuff to do with the entity key, I realised that the easiest thing to do was to replace the object with the actual object from the database context.  But in cases whether the object I’m trying to retrieve doesn’t exist, I want to know about it.  To this end (and with a little help from Kevin McNeish’s blog) I wrote the GetEntityForObject() extension method:

public static class DataContextExtensions
{
    #region Public methods

    public static T GetEntityForObject<T>(this DbContext context, T value)
        where T : class, new()
    {
        T castEntity = null;

        if (value != null)
        {
            var adapter = context as IObjectContextAdapter;
            Debug.Assert(adapter != null);

            var objectContext = adapter.ObjectContext;
            string entitySetName = context.GetEntitySetName(typeof(T));

            try
            {
                var key = objectContext.CreateEntityKey(entitySetName, value);
                object entity = null;
                bool isExists = objectContext.TryGetObjectByKey(key, out entity);

                if (isExists)
                {
                    Debug.Assert(entity != null);
                    castEntity = entity as T;
                    Debug.Assert(castEntity != null);
                }
            }
            catch { }
        }

        return castEntity;
    }

    public static string GetEntitySetName(this DbContext context, Type type)
    {
        var adapter = context as IObjectContextAdapter;
        Debug.Assert(adapter != null);

        var objectContext = adapter.ObjectContext;
        string entitySetName = GetEntitySetName(type, objectContext);

        return entitySetName;
    }

    #endregion

    #region Private methods
        
    private static string GetEntitySetName(Type type, ObjectContext context)
    {
        // Thanks to Kevin McNeish for this little gem

        string entityTypeName = type.Name;

        var container = context.MetadataWorkspace.GetEntityContainer(
            context.DefaultContainerName, DataSpace.CSpace);

        string entitySetName = (from meta in container.BaseEntitySets
                                where meta.ElementType.Name == entityTypeName
                                select meta.Name).First();

        return entitySetName;
    }

    #endregion

Basically this tries to get the “same” entity from the data context, but returns null if there is no match.  So in the Northwind example we could use it to set the value of a property, so that if it already existed in the database then it would use the database version:

product.Category = context.GetEntityForObject(product.Category);

If it can’t find a matching version in the database then nothing (or more accurately null) gets persisted for that field.  Obviously this could be handled as you see fit (e.g. throw a validation error back to the user if the field gets nulled).

I’ve thrown together a demo that shows the whole process against a WPF client.

Wednesday, 18 May 2011

Entity Framework 4.1: Riding the Magic Unicorn

UPDATE: there is now a VS2013 version of the sample project.

Although it was released back in April, it’s taken me until now to actually try out Entity Framework 4.1 (aka Magic Unicorn).  It’s very impressive and does a surprisingly large amount of the work of getting data from databases and putting it into objects (and vice versa) without the need for visual designers (although they’re still there) or field-by-field mapping.  The only bad thing I can say about it is that for something so feature-rich, documentation is thin on the ground, so I shall add what I’ve learnt to the scant body of knowledge out there.  Other than ADO.NET team blog, most of what I didn’t work out myself came from the invaluable Morteza Manavi’s blog – so I won’t repeat what’s already there.
To provide a little context (no pun intended), what drew me to EF 4.1 was the possibility of mapping an existing database to an existing object model.  This, to my mind, is the way that ORM should be done – i.e. you create a data schema according to the best practices of relational design, create an object model based on the domain requirement, then use ORM to get them to talk to one another.  Basically I think you should design object models with your OOP developer hat and design data schemas with your DBA hat.  Although there may be similarities between the two models, it’s rare that they’ll be exactly the same.
Anyway, I’ve spent this week mapping a complex object model to an equally complex data structure and despite thinking (at times) that Magic Unicorn didn’t support the type of mapping I was attempting, it proved me wrong on all counts and happily transformed my data schema into my object model.  I’ve also done some tinkering with EF across WCF, but I won’t bring that into the mix just yet as it comes with a whole load of caveats and scariness.

The basics

It’s worth mentioning that if your objects have the same name as the tables they map to and your properties have the same name as the fields, then you don’t need to do much at all.  EF will automatically infer the mappings based on a set of conventions – all you have to do is create the DbContext.  This is all covered by Scott Gu’s blog post.  The only caveat is that all POCOs you want to persist must have a key field.
In order to do custom mapping you have to override the OnModelCreating method of the DbContext and use EF’s fluent API to set up the mappings.  Most of the examples out there put all the mapping rules in the OnModelCreating method like so (I’ve included the DbContext setup for clarity):
public class MyDatabaseContext : DbContext
{
 public MyDatabaseContext()
  : base("data source=(local);initial catalog=MyDatabase;integrated security=True")
 { }

 public DbSet<MyClass> MyClasses { get; set; }

 protected override void OnModelCreating(DbModelBuilder modelBuilder)
 {
  modelBuilder.Entity<MyClass>()
   .Property(c => c.SomeFieldOrOther)
   .HasColumnName("StrangelyDifferentFieldName");
  modelBuilder.Entity<MyClass>()
   .Ignore(c => c.AFieldIDontWantMapped);
 }
}
I prefer to use an EntityTypeConfiguration class as it’s cleaner and more maintainable.  Basically you create one EntityTypeConfiguration for every class on which you want custom mapping and put the mapping code in the constructor:
public class MyClassEntityTypeConfiguration : EntityTypeConfiguration<MyClass>
{
 public MyClassEntityTypeConfiguration()
 {
  Property(c => c.SomeFieldOrOther).HasColumnName("StrangelyDifferentFieldName");
  Ignore(c => c.AFieldIDontWantMapped);
 }
}
Then add a new instance of that class to the DbContext’s configurations (via the OnModeCreating method), like so:
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
 modelBuilder.Configurations.Add<MyClass>(new MyClassEntityTypeConfiguration());
}
You can add in as many entity type configurations as you like, but if you don’t add one for a class then the default mapping rules will apply.
As an aside, something worth noting in the above examples is the use of the Ignore() method. This is used because any property in a class that doesn’t map to a field in the database must be explicitly ignored (except for enumerations which are ignored by default).

The example models

The sample project for this can be downloaded here.
For illustration I will use the scenario of a product sales LoB system.  Let us imagine that we have a company that uses field sales people to sells widgets.  The sales people can make either a sales or support visit to a customer – the model for this is shown below:
Visit model
Although the data model is probably close to what I’d actually use in such a situation, I have intentionally designed it to be ‘awkward’ via the use of differently named fields/properties, schemas, odd relationships etc:
Sales sample ERD

The code

In this section I’ll just explain a couple of the tricks I’ve used to get the mapping done.  Although the sample project includes mapping of object hierarchies I won’t bother explaining this as it’s covered better elsewhere.
  • Enumerations
    UPDATE: Later versions of EF do this natively. Don't use the code below.
    If you want to coerce a numeric value into an enumeration then just expose the database field as a numeric property and get the property setters to do the work for you.  In the sample, I’ve done this on the Visit class using the following enumeration:
    public enum VisitTypes : short
    {
    Sales = 1,
    Support = 2
    }
    The enumeration property ‘Type’ gets its value from the ‘VisitTypeId’ property (which maps to the field of the same name in the database).
    public class Visit
    {
     private short? _visitTypeId;
     private VisitTypes? _type;
    
     public VisitTypes? Type
     {
      get
      {
       return _type;
      }
      set
      {
       _type = value;
       _visitTypeId = null;
    
       if (_type.HasValue)
       {
        _visitTypeId = (short)_type.Value;
       }
      }
     }
    
     public short? VisitTypeId
     {
      get
      {
       return _visitTypeId;
      }
      set
      {
       _visitTypeId = value;
       _type = null;
    
       if (_visitTypeId.HasValue)
       {
        VisitTypes visitType;
    
        if (Enum.TryParse<VisitTypes>(_visitTypeId.ToString(), out visitType))
        {
         _type = visitType;
        }
       }
      }
     }
    }
    By using the Enum.TryParse method we can ensure that any values in the database that don't match the ones available in the enumeration force the enum field to null.
  • Multiple tables into one object
    This has been done in the SupportCheckList class in the sample. Simply call the Map method for every table that you want to include, using the ToTable and Properties methods in the lambda expression:
    public SupportChecklistEntityTypeConfiguration()
    {
     HasKey(s => s.Id);
    
     Map(m =>
      {
       m.ToTable("SupportCheckList", "Data");
       m.Properties(s => new
       {
        s.Id,
        s.WasIssueFixedDuringVisit,
        s.HasCustomerBeenGivenFollowUpDate
       });
      });
    
     Property(s => s.Id)
      .HasColumnName("SupportCheckListId")
      .HasDatabaseGeneratedOption(DatabaseGeneratedOption.Identity);
    
     Map(m =>
      {
       m.ToTable("SupportCheckListSafety", "Data");
       m.Properties(s => new
       {
        s.HasProductBeenMadeSafe,
        s.ProductFailureCausedIncident
       });
      });
    }
    The Properties method takes an anonymous type which contains a list of the fields which are mapped from the given table. You can then do column mapping as per normal (outside of the Map method) and EF will know where to look for the field.
    If you want to get multiple objects into one table then have a look at this post.
I won’t explain the various cardinality mappings because it’s quicker just to look at the sample.  There’s a mass of odd relationships and strange keys that I’ve put in there purposely to show how to use the Map() method of the fluent API.

Monday, 16 May 2011

MSMQ and WCF across the Internet

I've learned two new things in the last week about WCF or, to be more accurate, I have had two of my foolish assumptions corrected. The first assumption was that MSMQ provides support for disconnected interoperable services, while the second was that I would never need to self-host a WCF service in a Windows Forms/WPF application. Happily I now have a test application that does both of these things, thus giving me a pattern for what used to be called a "smart client".

Both of these discoveries came about as part of a project to deliver an occasionally-connected application for tablet devices. Basically, the client application would collect a load of data, package it up into a DTO, then persist it via a WCF service. Given the never-ending CTP state of the Sync Framework and my preference for SOA over the client/server database approach, I thought MSMQ would facilitate a single programming model regardless of the connectivity status of the client - simply designate the appropriate WCF service methods as one-way calls and expose over an MSMQ endpoint.

The problem with this is that MSMQ doesn't do what I thought it did. I assumed that I could expose a service endpoint somewhere "out there", then the MSMQ local queue on the client would monitor the availability of a network connection and post messages in the local queue off to the service when connectivity was achieved. This works very nicely in public queues, but they require an Active Directory installation and can't be exposed outside of the Intranet. A private queue (in a workgroup installation) can only post to/monitor an MSMQ service on the same machine. I did consider a bespoke solution using Isolated Storage, serialization, and a timer control - but it didn't have the transactional reliability of MSMQ (and had all the hallmarks of a badly reinvented wheel).

The answer to my problem came from the oft-quoted bible of WCF: Juval Lowy's "Programming WCF Services", where he addresses the issue of MSMQ over the Internet by means of the HTTP bridge pattern (p.518) and also the bit where he discusses responsiveness when hosting WCF services in a UI application (p.416). My solution is to post all messages into a local private queue, then trap the network connectivity events in the System.Net.NetworkChange class and launch/shutdown (as appropriate) a local MSMQ service host in the UI application (which must have the UseSynchronizationContext value on the ServiceBehavior attribute set to true so as not to lock up the UI thread). When the local MSMQ service starts up it sends all messages to the distant (HTTP) service using the bridge pattern. If the send fails (or any other unhandled exception occurs) then the local MSMQ service gets faulted and shuts down. It will start up again either the next time the network status active event is raised or when the application is opened again (the only problem with this approach being that if the call to the distant service fails because of a problem with the service itself then the call could well fail again the next time the local MSMQ service gets launched - but this doesn't lead to lost messages so I can live with it).

The sequence of activity is something like:

  • Form startup
  • Create queue (if it doesn't exist) on a background thread
  • When background worker returns set up event handler for NetworkAvailabilityChanged
  • If a network connection is available then launch the MSMQ service
  • If the network connection becomes unavailable then shut down the service
  • If the service faults then shut it down
  • If the network connection becomes available then launch the service

Getting an exception to fault the service host is done in the binding configuration:

<bindings>
<netMsmqBinding>
<binding receiveErrorHandling="Fault" 
receiveRetryCount="0">
<security mode="None"/>
</binding>
</netMsmqBinding>
</bindings>

This will stop any retries after the fault occurs. Subscribing to the Faulted event on the service host will provide a hook for taking action after a fault occurs.

The sample project can be downloaded here.  It requires MSMQ workgroup installation, which can be reached by the following route:

  1. On Windows XP this can be enabled by going into Control Panel, Add Remove/Programs, Add/Remove Windows Components.
  2. In Vista/Windows 7 this can be enabled by going into Control Panel, Programs, Turn Windows features on or off.

Run the sample, disable your network adapter, make a call, then enable your network adapter – it should send the messages queued locally.  It’ll need to be run in debug mode as the distant service simply writes out to the Debug/Output window, but it should be sufficient implementation for illustration.

Wednesday, 23 March 2011

Single-click editing for checkboxes in WPF and Silverlight datagrids

Possibly the single most annoying thing about the (otherwise excellent) datagrid in WPF and Silverlight is that the check box column requires two clicks to actually change state.  There’s a perfectly good reason for this – the cell must be selected before editing – but the user experience is a bit irritating.  I first came across this problem in WPF and found a solution, only to discover that the same thing wouldn’t work in Silverlight.  So this post is more of an aide memoire to myself for the next time this happens.

I should add that this isn’t completely original code, more of a piecing together of code from much cleverer people on the Silverlight forums and Stack Overflow, but it works happily with MVVM/data binding.  Both versions use the FindVisualChildren method mentioned in this post:

public static IEnumerable<T> FindVisualChildren<T>(DependencyObject depObj)
    where T : DependencyObject
{
    if (depObj != null)
    {
        for (int i = 0; i < VisualTreeHelper.GetChildrenCount(depObj); i++)
        {
            DependencyObject child = VisualTreeHelper.GetChild(depObj, i);
            if (child != null && child is T)
            {
                yield return (T)child;
            }

            foreach (T childOfChild in FindVisualChildren<T>(child))
            {
                yield return childOfChild;
            }
        }
    }
}

The WPF version:

public static class DataGridCellHelper
{
    #region IsSingleClickInCell

    public static readonly DependencyProperty IsSingleClickInCellProperty =
        DependencyProperty.RegisterAttached("IsSingleClickInCell",
        typeof(bool), typeof(DataGrid),
        new FrameworkPropertyMetadata(false, OnIsSingleClickInCellSet));

    public static void SetIsSingleClickInCell(UIElement element, bool value)
    {
        element.SetValue(IsSingleClickInCellProperty, value);
    }

    public static bool GetIsSingleClickInCell(UIElement element)
    {
        return (bool)element.GetValue(IsSingleClickInCellProperty);
    }

    private static void OnIsSingleClickInCellSet(DependencyObject sender,
        DependencyPropertyChangedEventArgs e)
    {
        if (!(bool)(DesignerProperties.IsInDesignModeProperty.GetMetadata(typeof(DependencyObject)).DefaultValue))
        {
            if ((bool)e.NewValue)
            {
                var dataGrid = sender as DataGrid;
                Debug.Assert(dataGrid != null);

                EventManager.RegisterClassHandler(typeof(DataGridCell),
                    DataGridCell.PreviewMouseLeftButtonUpEvent,
                    new RoutedEventHandler(OnPreviewMouseLeftButtonDown));
            }
        }
    }

    private static void OnPreviewMouseLeftButtonDown(object sender, RoutedEventArgs e)
    {
        DataGridCell cell = sender as DataGridCell;

        if (cell != null && !cell.IsEditing && !cell.IsReadOnly)
        {
            var checkBoxes = ControlHelper.FindVisualChildren<CheckBox>(cell);

            if (checkBoxes != null && checkBoxes.Count() > 0)
            {
                foreach (var checkBox in checkBoxes)
                {
                    if (checkBox.IsEnabled)
                    {
                        checkBox.Focus();
                        checkBox.IsChecked = !checkBox.IsChecked;

                        var bindingExpression = checkBox.GetBindingExpression(CheckBox.IsCheckedProperty);

                        if (bindingExpression != null)
                        {
                            bindingExpression.UpdateSource();
                        }
                    }
                    break;
                }
            }
        }
    }

    #endregion
}

The Silverlight version:

public static class DataGridCellHelper
{
    #region IsSingleClickInCell

    public static readonly DependencyProperty IsSingleClickInCellProperty =
        DependencyProperty.RegisterAttached("IsSingleClickInCell",
        typeof(bool), typeof(DataGrid),
        new PropertyMetadata(false, OnIsSingleClickInCellSet));

    public static void SetIsSingleClickInCell(UIElement element, bool value)
    {
        element.SetValue(IsSingleClickInCellProperty, value);
    }

    public static bool GetIsSingleClickInCell(UIElement element)
    {
        return (bool)element.GetValue(IsSingleClickInCellProperty);
    }

    public static void OnIsSingleClickInCellSet(DependencyObject sender,
        DependencyPropertyChangedEventArgs e)
    {
        if (!(bool)(DesignerProperties.IsInDesignModeProperty.GetMetadata(
            typeof(DependencyObject)).DefaultValue))
        {
            if ((bool)e.NewValue)
            {
                var dataGrid = sender as DataGrid;
                Debug.Assert(dataGrid != null);

                dataGrid.MouseLeftButtonUp += OnDataGridMouseLeftButtonUp;
            }
        }
    }

    private static void OnDataGridMouseLeftButtonUp(object sender, MouseButtonEventArgs e)
    {
        var dataGrid = sender as DataGrid;
        Debug.Assert(dataGrid != null);

        var hitPoint = e.GetPosition(null);
        var controlsInRange = VisualTreeHelper.FindElementsInHostCoordinates(
                                    hitPoint, dataGrid);
        var dataGridCellInRange = (from element in controlsInRange
                                    where element is DataGridCell
                                    select element).FirstOrDefault();

        if (dataGridCellInRange != null)
        {
            HandleSingleClickCheckBox(dataGridCellInRange);
        }
    }

    private static void HandleSingleClickCheckBox(UIElement dataGridCellInRange)
    {
        var checkBox = ControlHelper.FindVisualChildren<CheckBox>(
            dataGridCellInRange).FirstOrDefault();

        if (checkBox != null)
        {
            if (checkBox.IsEnabled)
            {
                checkBox.Focus();
                checkBox.IsChecked = !checkBox.IsChecked;

                var bindingExpression = checkBox.GetBindingExpression(CheckBox.IsCheckedProperty);

                if (bindingExpression != null)
                {
                    bindingExpression.UpdateSource();
                }
            }
        }
    }

    #endregion
}

Usage is the same in both (where p is the reference to the namespace that the DataGridCellHelper class is in):

<DataGrid p:DataGridCellHelper.IsSingleClickInCell="True" />

This should ensure that any checkbox columns respond to a single click, even if data bound.

Tuesday, 8 March 2011

Custom role authorisation with Windows authentication in WCF

I’ve just published my first project on CodePlex, which is a WPF/MVVM tool based on the IDesign credentials manager, so it seemed an appropriate time to blog about using ASP.NET providers for authorisation while using Windows for authentication with WCF services.  Admittedly this isn’t an original idea, in fact ScottGu blogged about it back in 2006, but for ASP.NET web applications instead.  This is simply a variation on that.

Aside from the fact that SharePoint has done this since WSS 3.0/MOSS 2007, why would you want to do this?   the main reason is because you may not have control over Active Directory (if you don’t regularly see any of the people who run the servers that you work on, then this is probably true of your situation) but you want to provide granular permissions on an application or service.  Eventually you’ll want to delegate role management to an end-user so the product owners can be responsible for their own security.  In my experience, the larger a company gets, the longer it takes to get a system access change request done – so this type of requirement is quite common.  There are technical and business advantages to using Windows authentication (instead of the ASP.NET membership provider); such as being able to rely on the ambient security of your business domain for user authentication, but also that it removes the need for certificates in development or production.  Often, network security administrators don’t like to create AD groups to meet the requirements of specific applications, so this is a “best of both worlds” solution that should keep everyone happy.

Implementation is done in the web.config.  Basically it’s no different from using ASP.NET providers for everything (authentication and authorisation) except that you can miss out the authentication bits as all the main bindings default to Windows authentication.  The only caveat is for the basicHttpBinding (which is insecure by default) where the security and transport client credentials have to be explicitly defined to use Windows authentication.  The important bits are below:

<?xml version="1.0"?>
<configuration>
  <connectionStrings>
    <!--
      This is the connection string to the aspnetdb/credentials store database
      Update as appropriate
    -->
    <add name="AspNetDb" connectionString="Data Source=MyServerName;Initial Catalog=aspnetdb;Integrated Security=True;enlist=false" providerName="System.Data.SqlClient"/>
  </connectionStrings>
  <system.web>
    <!--
      Replace the application name with the name of your application in the aspnetdb/credentials store database
    -->
        <roleManager enabled="true" defaultProvider="SqlRoleProvider">
          <providers>
            <add
              name="SqlRoleProvider"
              type="System.Web.Security.SqlRoleProvider"
              connectionStringName="AspNetDb"
              applicationName="MyApplication"
                      />
          </providers>
        </roleManager>
    </system.web>
  <system.serviceModel>
    <bindings>
      <basicHttpBinding>
        <binding>
          <security mode="TransportCredentialOnly">
            <transport clientCredentialType="Windows" />
          </security>
        </binding>
      </basicHttpBinding>
    </bindings>
    <behaviors>
      <serviceBehaviors>
        <behavior>
          <serviceAuthorization
                        principalPermissionMode="UseAspNetRoles"
                        roleProviderName="SqlRoleProvider" />
        </behavior>
      </serviceBehaviors>
    </behaviors>
  </system.serviceModel>
</configuration>

The client-side config for basicHttpBinding is:

<system.serviceModel>
<bindings>
  <basicHttpBinding>
    <binding>
      <security mode="TransportCredentialOnly">
        <transport clientCredentialType="Windows" proxyCredentialType="None"
            realm="" />
      </security>
    </binding>
  </basicHttpBinding>
</bindings>
</system.serviceModel>

Tuesday, 8 February 2011

Crosstab, pivot, matrix, WPF DataGrid

When I first started out doing IT professionally (having done it amateurishly for about 15 years at that point), Microsoft Access was my proverbial bread and butter (prior to that I had done a lot of work with accountant-types on Excel and its forebears).  So I was more than a tad familiar with the crosstab query, which has since survived in the Excel Pivot table and the SSRS Matrix/Tablix control (among others).  While I’d love to wax lyrical about the joys of Access, the reason I mention this is because in the last few days I came across a problem that required editing tabular data in a matrix, inside a WPF application.  The problem concerns a security matrix: we have a list of application roles and functions, with a boolean value indicating which roles can carry out which functions.  For example:
  Pending Active Dormant Locked
Monkey X
Organ grinder X X X
Evil Overlord X X X X
As the list of roles and functions may increase or (possibly) decrease dynamically, this data needs to be stored in a database in a format that accommodate this.  Typically:
SystemStatus RoleName CanEdit
Requested Monkey X
Pending Monkey  
Active Monkey  
Dormant Monkey  
Locked Monkey  
Requested Organ Grinder  
Pending Organ Grinder X
Active Organ Grinder X
Dormant Organ Grinder X
Locked Organ Grinder  
Requested Evil Overlord X
Pending Evil Overlord X
Active Evil Overlord X
Dormant Evil Overlord X
Locked Evil Overlord X
So the problem is that we need to be able to get the data from the tabular format into the matrix.  But I don’t want to be rewriting the DataGrid, so here’s where the ADO.NET DataTable comes to the rescue.  In these days of LINQ to SQL, Entity Framework, et al; the DataTable seems a bit of a legacy curiosity and something that I would normally discourage the use of in data-binding scenarios due to its excess of plumbing (and proprietary serialization).  Back in the (.NET 1) day however, it was the preferred means of data access and persistence; supported in Windows Forms data binding controls and even after all these years is natively supported in WPF list controls.  The good thing about the DataTable is that it can be built up on the fly by adding (Data)columns and (Data)rows as needed, then bound to a WPF DataGrid with column auto-generation.
So using a combination of reflection and generics, we can easily build up a pivoted bindable version of an object in a DataTable.  The only thing to be aware of is that DataTables pre-dated nullable types by some years – thus instead of null use DbNull.Value.  So we need some extra code in there to get the underlying primitive from any nullables and set null values appropriately.  Note also that the column pivot values are applied to the DataColumn’s Caption property because the Name property has to follow variable naming conventions (so there’s another method to coerce the column name into a variable-friendly format).
Here’s the pivoting class:
/// <summary>
/// Class for crosstabbing/pivoting a list of objects
/// </summary>
/// <typeparam name="T">The type of object to pivot</typeparam>
public class ObjectPivoter<T>
where T : class, new()
{
#region Private member variables

private PropertyInfo _columnProperty;
private PropertyInfo _valueProperty;
private PropertyInfo _rowProperty;
private string _rowSourceHeader;

#endregion

#region Constructor

/// <summary>
/// Creates a new instance of this class with the given row, column, and value settings
/// </summary>
/// <param name="rowSource">The name of the field in type <typeparamref name="T"/> to use for the row headings in the pivot</param>
/// <param name="columnSource">The name of the field in type <typeparamref name="T"/> to use for the columns headings in the pivot</param>
/// <param name="valueSource">The name of the field in type <typeparamref name="T"/> to use for the values in the pivot</param>
/// <exception cref="System.ArgumentNullException">Thrown if any of the arguments are null, empty, or whitespace only</exception>
/// <exception cref="System.ArgumentOutOfRangeException">Thrown if any of the values used for pivot fields do not exist in type <typeparamref name="T"/></exception>
public ObjectPivoter(string rowSource, string columnSource, string valueSource) :
this(rowSource, rowSource, columnSource, valueSource)
{ }

/// <summary>
/// Creates a new instance of this class with the given row, column, and value settings
/// </summary>
/// <param name="rowSource">The name of the field in type <typeparamref name="T"/> to use for the row headings in the pivot</param>
/// <param name="rowSourceHeader">The caption to use for the row heading of the row pivot</param>
/// <param name="columnSource">The name of the field in type <typeparamref name="T"/> to use for the columns headings in the pivot</param>
/// <param name="valueSource">The name of the field in type <typeparamref name="T"/> to use for the values in the pivot</param>
/// <exception cref="System.ArgumentNullException">Thrown if any of the arguments are null, empty, or whitespace only</exception>
/// <exception cref="System.ArgumentOutOfRangeException">Thrown if any of the values used for pivot fields do not exist in type <typeparamref name="T"/></exception>
public ObjectPivoter(string rowSource, string rowSourceHeader, 
string columnSource, string valueSource)
{
ThrowExceptionIfFieldNullOrEmpty("rowSource", rowSource);
ThrowExceptionIfFieldNullOrEmpty("rowSourceHeader", rowSourceHeader);
ThrowExceptionIfFieldNullOrEmpty("columnSource", columnSource);
ThrowExceptionIfFieldNullOrEmpty("valueSource", valueSource);

_rowSourceHeader = rowSourceHeader;
_columnProperty = typeof(T).GetProperty(columnSource);

if (_columnProperty == null)
{
ThrowExceptionIfFieldNotExists("columnSource", typeof(T), columnSource);
}

_valueProperty = typeof(T).GetProperty(valueSource);

if (_valueProperty == null)
{
ThrowExceptionIfFieldNotExists("valueSource", typeof(T), valueSource);
}

_rowProperty = typeof(T).GetProperty(rowSource);

if (_rowProperty == null)
{
ThrowExceptionIfFieldNotExists("rowSource", typeof(T), rowSource);
}
}

#endregion

#region Public methods

public DataTable CreatePivot(IList<T> source)
{
if (source == null)
{
throw new ArgumentNullException("source");
}

var pivotTable = new DataTable();

CreatePivotColumns(source, pivotTable, _columnProperty, 
_valueProperty, _rowProperty);

AddPivotRows(source, pivotTable, _columnProperty, 
_valueProperty, _rowProperty);

return pivotTable;
}

public ObservableCollection<T> Unpivot(DataTable table)
{
if (table == null)
{
throw new ArgumentNullException("table");
}

var unpivotedData = new ObservableCollection<T>();

foreach (DataRow row in table.Rows)
{
foreach (DataColumn column in table.Columns)
{
if (column.ColumnName != _rowProperty.Name)
{
var dataItem = CreateObjectFromRowColumn(row, column);                    
unpivotedData.Add(dataItem);
}
}
}

return unpivotedData;
}

#endregion

#region Private methods

private void AddPivotRows(IList<T> source, DataTable pivotTable, PropertyInfo columnProperty,
PropertyInfo valueProperty, PropertyInfo rowProperty)
{
var rows = (from s in source
select rowProperty.GetValue(s, null)).Distinct();

foreach (var row in rows)
{
var newRow = pivotTable.NewRow();
newRow[rowProperty.Name] = Convert.ChangeType(row, rowProperty.PropertyType);

foreach (var sourceRow in source)
{
object rowValue = rowProperty.GetValue(sourceRow, null);

if (rowValue != null && rowValue.ToString() == row.ToString())
{
string columnName = columnProperty.GetValue(sourceRow, null).ToString();
columnName = GetCleanColumnName(columnName);

var columnValue = valueProperty.GetValue(sourceRow, null);

if (columnValue == null)
{
columnValue = DBNull.Value;
}

newRow[columnName] = columnValue;
}
}

pivotTable.Rows.Add(newRow);
}
}

private void CreatePivotColumns(IList<T> source, DataTable pivotTable, 
PropertyInfo columnProperty, PropertyInfo valueProperty, PropertyInfo rowProperty)
{
var columns = (from s in source
select columnProperty.GetValue(s, null).ToString()).Distinct();

Type rowColumnType = GetUnderlyingPrimitive(rowProperty.PropertyType);
pivotTable.Columns.Add(rowProperty.Name, rowColumnType);
pivotTable.Columns[rowProperty.Name].Caption = _rowSourceHeader;

foreach (var column in columns)
{
Type columnType = GetUnderlyingPrimitive(valueProperty.PropertyType);

string columnName = GetCleanColumnName(column);
pivotTable.Columns.Add(columnName, columnType);
pivotTable.Columns[columnName].Caption = column;
}
}

private T CreateObjectFromRowColumn(DataRow row, DataColumn column)
{
var dataItem = new T();

_rowProperty.SetValue(dataItem, row[_rowProperty.Name], null);
_columnProperty.SetValue(dataItem, column.Caption, null);

bool isNullable = IsNullable(_valueProperty.PropertyType);

if (row[column.ColumnName] == DBNull.Value && isNullable)
{
_valueProperty.SetValue(dataItem, null, null);
}
else
{
_valueProperty.SetValue(dataItem, row[column.ColumnName], null);
}

return dataItem;
}

private void ThrowExceptionIfFieldNullOrEmpty(string argumentName, string argument)
{
if (string.IsNullOrEmpty(argument) || argument.Trim() == string.Empty)
{
throw new ArgumentNullException(argumentName);
}
}

private void ThrowExceptionIfFieldNotExists(string argumentName, 
Type type, string fieldName)
{
throw new ArgumentOutOfRangeException(argumentName,
string.Format("The field '{0}' does not exist in the '{1}' class",
fieldName, type.Name));
}

private string GetCleanColumnName(string columnName)
{
string cleanColumnName = Regex.Replace(columnName, @"[^\w]", @"_");

if (!Regex.IsMatch(cleanColumnName, @"^[a-zA-Z_]\w*$"))
{
cleanColumnName = string.Concat(@"_", cleanColumnName);
}

return cleanColumnName;
}

private bool IsNullable(Type type)
{
bool isNullable = (type.IsGenericType &&
type.GetGenericTypeDefinition() == typeof(Nullable<>));
return isNullable;
}

private Type GetUnderlyingPrimitive(Type type)
{
if (IsNullable(type))
{
return Nullable.GetUnderlyingType(type);
}

return type;
}

#endregion
}


Usage as follows:


_pivoter = new ObjectPivoter<SecurityMatrixEntry>("RoleName", "Role", "SystemStatus", "CanEdit");
var pivotedData = _pivoter.CreatePivot(dummyData);

dgMatrix.ItemsSource = pivotedData.DefaultView;


Sample (VS2010) project here, which also contains some attached properties for the DataGrid which enable the use of the DataColumn’s Caption property for the column headers as well as locking the first column.

Wednesday, 2 February 2011

Plumbing-for-free: WCF, MVVM, and the WPF DataGrid

I have come to love MVVM, but every now and then WPF comes back and reminds us that it often wants to be used like Windows Forms with its event-based code-behind model.  Nowhere is this more pronounced than in the WPF DataGrid -which comes with all sorts of helpful plumbing for filtering, sorting, and tracking dirty states.  Unfortunately a lot of this functionality is tied to events on the control, which means that at some point you’ll either need to put View-related code into the ViewModel or else use code-behind.  I hit one such problem today where I wanted to know if an ObservableCollection, that I was using as an ItemSource for a DataGrid, was dirty or not.  By ‘dirty’ I mean that either the contents of the list, or any of the objects in the list, had changed.  However if all changed properties were returned to their original state then I would consider that the data was no longer dirty.

Essentially what I wanted to implement was a ‘trackable’ observable collection similar to Ludwig’s one here.  Unfortunately the requirement to ‘un-dirty’ the collection if data returns to its original state, makes it a bit trickier.  Now this is where using WCF, even for non-distributed systems, can really pay off.  There are two reasons for this:

  1. SvcUtil can be used to automatically implement INotifyPropertyChanged on every public property of a data contract (i.e. model class) when creating proxy classes.  This means that with automated proxy rebuilds (see this post) you get property notifications guaranteed on all data contracts with no extra effort.
  2. WCF ‘model’ objects in SvcUtil generated proxy classes are deep-cloneable by default.  This is because all ‘model’ objects are DataContracts and all fields are DataMembers (otherwise they wouldn’t be picked up from the WSDL by SvcUtil in the first place), so serializing and then de-serializing an object will give you an exact copy that isn’t the same object.  This is a technique used in IDesign’s SerializationUtil helper class.

The net effect is that we have an easy way of being notified when a property has changed, but also of checking whether the object (i.e. data contract) as a whole has changed.  So, taking ‘inspiration’ from Ludwig’s example (i.e. trapping the property change notification per-object), we can include something that also compares objects every time a property on the object changes – but only if the object was in the original collection (because additions/inserts to the list are dirty-by-default).  For this we can use the aforementioned IDesign helper classes, to which we can add a few extra methods for comparing objects.  Rather than attempt to compare byte or ASCII streams, we can compute an MD5 hash of the object and compare that instead.

My solution is simply a leaner modification of the TrackableObservableCollection<T> class.  Internally it wraps every ‘original’ object in the list in a TrackableObject<T> class, which handles the plumbing of notifying the parent list of changes via the OnDirtyStatusChanged event.  The TrackableObservableCollection<T> inherits from ObservableCollection<T> which saves us having to recode a whole raft of plumbing.

Despite what you may think, given the constant serialization/deserialization, the object comparison is very quick (longest time taken is 6ms).  What matters though is that the solution performs well enough.  I’m sure that someone clever has a more efficient way of doing this, but what I like is that it’s all made so easy and plumbing-light by the use of WCF.

To use, simply pass your existing ObservableCollection to the constructor, then bind to the DataGrid:

// get the data
ObservableCollection<TestClass> myCollection = GetData();

// convert to a trackable collection
TrackableObservableCollection<TestClass> myTrackableCollection =
new TrackableObservableCollection<TestClass>(myCollection);

Debug.Assert(!myTrackableCollection.IsDirty);

// store a value
DateTime? originalValue = myCollection[0].DateOfBirth;

// change the value
myCollection[0].DateOfBirth = DateTime.Now;
Debug.Assert(myTrackableCollection.IsDirty);

// change the value back
myCollection[0].DateOfBirth = originalValue;
Debug.Assert(!myTrackableCollection.IsDirty);

Rather than detail every single aspect of the solution, it’s easier just to download the (VS2010) example.

Points to note:

  • For the sake of simplicity I’ve used ‘Add Service Reference’ rather than SvcUtil in the build events.
  • Be careful when using EF-derived data contracts with circular navigation properties as updating a value may not apply the update to the respective navigation property (a problem that can be averted by not using lazy loading needlessly)

Sunday, 30 January 2011

MVVM-friendly hideable menus in WPF

I’ve been working on adding keyboard accessibility features to an existing MVVM WPF application - during the course of which I decided that while a menu bar would be a great way to aid navigation, it wasn’t in keeping with the aesthetic style.  A number of Microsoft applications (e.g. IE8+, Windows Live Messenger) solve this by having a menu bar that hides until the Alt or F10 key is pressed, then hides again when keyboard focus is lost.

This functionality is available OOB in MFC with the SetMenuBarVisibility member of the CFrameWnd class (also described in this article).  Despite searching for some time I couldn’t find how to surface this easily in WPF but I did find an answer on Stack Overflow, however it’s a bit too intrusive for MVVM.  In the end I came up with a solution that uses attached properties, which means no code-behind or view-based code in the view model.

The property is called HideMainMenu and it’s attached to a Window.  If set to true it will hide the first menu marked as the main menu.  It does this by hooking up code via an attached property to the Window.KeyDown and Menu.IsKeyboardFocus events.  I also borrow a little code from this question on Stack Overflow to help navigate the visual tree (the FindVisualChildren<T> method):

public static class MainMenuHelper
{
public static readonly DependencyProperty HideMainMenuProperty =
DependencyProperty.RegisterAttached("HideMainMenu",
typeof(bool), typeof(Window),
new FrameworkPropertyMetadata(false, OnWindowLoaded));

public static void SetHideMainMenu(UIElement element, bool value)
{
element.SetValue(HideMainMenuProperty, value);
}

public static bool GetHideMainMenu(UIElement element)
{
return (bool)element.GetValue(HideMainMenuProperty);
}

private static void OnWindowLoaded(DependencyObject sender,
DependencyPropertyChangedEventArgs e)
{
if (!(bool)(DesignerProperties.IsInDesignModeProperty.GetMetadata(typeof(DependencyObject)).DefaultValue))
{
if ((bool)e.NewValue)
{
var window = sender as Window;
Debug.Assert(window != null);
window.Loaded += hostingWindow_Loaded;
}
}
}

private static void hostingWindow_Loaded(object sender, RoutedEventArgs e)
{
var window = sender as Window;

var menus = FindVisualChildren<Menu>(window);

foreach (var menu in menus)
{
if (menu.IsMainMenu)
{
window.KeyDown += hostingWindow_KeyDown;
menu.LostKeyboardFocus += mainMenu_LostKeyboardFocus;
menu.Visibility = Visibility.Collapsed;
break;
}
}

window.Loaded -= hostingWindow_Loaded;
}

private static void hostingWindow_KeyDown(object sender, KeyEventArgs e)
{
if (e.SystemKey == Key.LeftAlt || e.SystemKey == Key.RightAlt || e.SystemKey == Key.F10)
{
var window = sender as Window;
Debug.Assert(window != null);

foreach (Menu menu in FindVisualChildren<Menu>(window))
{
if (menu.IsMainMenu)
{
if (menu.Visibility == Visibility.Collapsed)
{
menu.Visibility = Visibility.Visible;
}
else
{
menu.Visibility = Visibility.Collapsed;
}

break;
}
}
}
}

private static void mainMenu_LostKeyboardFocus(
object sender, KeyboardFocusChangedEventArgs e)
{
var menu = sender as Menu;

if (menu.IsMainMenu && !menu.IsKeyboardFocusWithin)
{
menu.Visibility = Visibility.Collapsed;
}
}

private static IEnumerable<T> FindVisualChildren<T>(DependencyObject depObj) where T : DependencyObject
{
// thanks to http://stackoverflow.com/questions/974598/find-all-controls-in-wpf-window-by-type for this method

if (depObj != null)
{
for (int i = 0; i < VisualTreeHelper.GetChildrenCount(depObj); i++)
{
DependencyObject child = VisualTreeHelper.GetChild(depObj, i);
if (child != null && child is T)
{
yield return (T)child;
}

foreach (T childOfChild in FindVisualChildren<T>(child))
{
yield return childOfChild;
}
}
}
}
}


Usage example as follows:



<Window x:Class="MenuSample.MainWindow"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:ap="clr-namespace:MenuSample"
ap:MainMenuHelper.HideMainMenu="True"
Title="MainWindow" Height="350" Width="525">
<Grid>
<Menu IsMainMenu="True">
<!-- menu items here -->
</Menu>
</Grid>
</Window>


The VS2010 sample project can be downloaded here.