Thursday, 31 December 2009

Nearly the 2nd decade of the 21st century but…

So much for the technological age.  It’s nearly 2010 and I need to make a payment that will arrive in a bank account the next working day.  So I phone up the bank, who inform me that it has to be made by CHAPS at a cost of £25.  Which is a tad annoying given that light travels around the world about 7 times a second and a typical enterprise database can serve several thousand concurrent users on a single processor, but obviously my maths is flawed.

Anyway, all they need for this transaction to take place is a fax.  Excuse me?!?!?  It’s the 21st frickin’ century!!  Why do I need to send a fax?  Surely email is easier?  It’s not like it’s even more secure!  I don’t have a fax machine and none of my PCs have telephone modems in them because it’s like…er…the 21st century and all that - I have ADSL and very quick it is too.  So I look for a fax by email service that doesn’t require me to sign up to a contract or monthly fee but to no avail.  I only want to send a 1 page fax for goodness sake!

So next time my bank asks me for information perhaps I should say something like “yeah, that’s fine.  Just send it to me on chiselled stone or Papyrus scrolls in ink made from Dodo blood”.

Friday, 18 December 2009

Maintainable code and Boolean logic

After months of doing BI-related stuff, I’ve found myself doing ‘proper’ coding again – specifically Windows Forms based work in C# working against a WCF data source. Other than the realisation that Forms seems primitive compared to WPF, I’ve found myself referring back to Juval Lowy’s excellent tome “Programming .NET Components” which includes the IDesign C# coding standard. This is the definitive .NET coding standard and one I thoroughly recommend adopting (if you haven’t done so already), however it does contain some guidance whose value cannot be fully appreciated until one starts to follow it. In particular I’m thinking of rule 30 in the “Coding Practices” section:

“Avoid function calls in Boolean conditional statements. Assign into local variables and check on them”

Among other things this makes it much easier this makes to debug nested conditional statements, but it’s also helped me to realise the value of using Boolean functions. By factoring out as much code as possible to such functions, code becomes self-descriptive. To illustrate this point, look at the following code:

if (
      !(lastRowRequested <= (m_CentrePoint + (m_ToleranceWindow / 2))
      &&
      lastRowRequested >= (m_CentrePoint + (m_ToleranceWindow / 2)))
  )
{
  if (
          lastRowRequested > GetAbsoluteHalfwayPoint()
          &&
          (m_Cache.Count + m_FirstRowAbsoluteIndex)!=m_TotalCount
      )
  {
      // Do something
  }
  elseif
      (
          lastRowRequested < GetAbsoluteHalfwayPoint()
          &&
          m_FirstRowAbsoluteIndex != 0
      )
  {
      // Do something
  }
  else
  {
      // Do something else
  }
}

It’s part of a cache manager component that keeps a ‘window’ in memory on a much larger dataset. Knowing that, does the above code make sense? It certainly doesn’t to me and I wrote it! At this point I refer to rule 8 of “Coding Practices”:

“Avoid comments that explain the obvious. Code should be self-explanatory. Good code with readable variable and method names should not require comments.”

Now this is one of the many potentially controversial rules in the standard, but only because some people take it to mean “don’t write comments”. What it is actually saying is “don’t state the bleedin’ obvious”. How can we rewrite the above code to give it context and clarity? Something like this:

bool isInToleranceWindow = IsInToleranceWindow(lastRowRequested);

if (!isInToleranceWindow)
{
  bool isInBottomHalfOfCache = IsInBottomHalfOfCache(lastRowRequested);
  bool isInTopHalfOfCache = IsInTopHalfOfCache(lastRowRequested);
  bool isFirstRowOfDatasetInCache = IsFirstRowOfDatasetInCache;
  bool isLastRowOfDatasetInCache = IsLastRowOfDatasetInCache;

  if (isInBottomHalfOfCache && !isLastRowOfDatasetInCache)
  {
      // Do something
  }
  elseif (isInTopHalfOfCache  && !isFirstRowOfDatasetInCache )
  {
      // Do something
  }
  else
  {
      // Do something else
  }
}

Now the beauty of this will be evident to any BI practitioners out there, in that it gives a high-level overview of what’s going on but (thanks to the F12 key in Visual Studio) one can drill down to a lower level of detail if required (i.e. look at the function definitions). I realise that this is a trivial example, but it does demonstrate how refactoring code can make it easier to understand (and maintain).

Incidentally, as an antithesis to coding standards there is always the obfuscator’s bible, whose adherents I have to thank for providing me with a good living for several years.

Tuesday, 15 December 2009

Custom paging with MDX in SSRS

As I’ve mentioned before, one of the things I really like about QlikView is scrollbars on the bar charts. Too much data to display on the screen at once? Not a problem – you can scroll the axes! Now this can be sort-of-replicated in SSRS by using dynamically sized charts and hosting the reports in a dashboard-friendly environment like SharePoint, but can require a bit more custom work if you’re either using SSRS Report Manager or are hosting in a single ASP.NET Report Viewer.

I came across such a problem recently and decided to use custom paging. Given that I’ve quite successfully implemented before in reports based on a T-SQL source (but not an MDX one), the T-SQL solution seemed a fairly good place to start. Basically you need 2 data sources – 1 for the pager and another for the actual data being displayed. Returning data in pages is a common requirement for ASP.NET developers (although less so since the advent of LINQ) so I’ll point you to this article for more information. The SSRS T-SQL pager requires that you:

  1. Include a page parameter in your report
  2. Have a page size
  3. Calculate how many pages are in the data
  4. Return a table of sequential numbers from 1 up to the amount of pages

Calculating the number of pages is pretty trivial (assuming that decimals are rounded up):

Number of pages = Dataset size / page size
(e.g. for a dataset of 150 rows with a page size of 10, the number of pages would be 15)

So to populate the pager with page numbers we need some T-SQL that will return the numbers 1 to 15. We could use a loop but that would be very ugly – a better alternative is to use Common Table Expressions as you can use recursion to create an incremental list. To cut a long story short, here’s some T-SQL that shows it (without explaining why):

DECLARE @StartNumber int, @EndNumber int;

SET @StartNumber = 1;
SET @EndNumber = 15;

WITH PagerNumbers
(
 PageNumber
)
AS
(
 SELECT
     @StartNumber [PageNumber]
 UNION ALL
 SELECT
     X.PageNumber + 1
 FROM
     PagerNumbers X
 WHERE
     PageNumber < @EndNumber
)
SELECT
 *
FROM
 PagerNumbers
OPTION
(
   MAXRECURSION 0
);

Set that as the data source for a table control where each cell hyperlinks through to the respective page in the report and that’s about it (actually there’s a few less-than-minor implementation details to be taken care of but you get the idea). What we really want to know is how to do the same thing easily in MDX? The answer lies in the RANK function. The principle is simple:

  • Use RANK to give each row in the entire dataset a unique (sequential) number
  • Return only the first row for each page from the dataset
  • Er…that’s it

We also need to know the total number of records in the dataset, as it may not be divisible exactly by the page size – which would lead to the number of the last page not being included. To illustrate this point, consider a dataset with 153 items and a page size of 10 items. If we return all rows where:

row number MOD page size = 0

then we will get 15 pages back where there should be 16 (the last page will contain rows 151-153). To get round this we need to make sure that the last item is displayed even if it doesn’t divide exactly by the page size. In the MDX below (based on the AdventureWorks cube), this can be seen in the use of the PagePoint and PageNumber measures – which are then used in combination with the Round function to work out if the item is the last one in the dataset:

WITH
 SET OrderedActivity
     AS
     Order
     (
         NonEmpty
         (
             StrToSet('[Product].[Product].[Product].ALLMEMBERS'),
             [Measures].[Internet Sales Amount]
         ),
         [Measures].[Internet Sales Amount],
         BDESC
     )
 MEMBER [Measures].[Row rank]
     AS
     Rank
     (
         StrToMember('[Product].[Product]').HIERARCHY.CURRENTMEMBER,
         OrderedActivity
     )
 MEMBER [Measures].[MaxPage]
     AS
     Max
     (
         OrderedActivity,
         [Measures].[Row rank]
     )
 MEMBER [Measures].[PagePoint]
     AS
     [Measures].[Row rank] / StrToValue(10)     
 MEMBER [Measures].[PageNumber]
     AS
     IIf
     (
         [Measures].[Row rank]=[Measures].[MaxPage],
         Round
         (
             (
                 [Measures].[PagePoint] + 0.5
             )
         ),
         (
             [Measures].[PagePoint]
         )
     )
SELECT
 {
     [Measures].[Row rank],
     [Measures].[PageNumber],
     [Measures].[MaxPage]
 } ON COLUMNS,
 Filter
 (
     Order
     (
         NonEmpty
         (
             {
                 StrToSet('[Product].[Product].[Product].ALLMEMBERS')
             },
             [Measures].[Internet Sales Amount]
         ),
         [Row rank],
         ASC
     ),
     [PageNumber] = Round([PageNumber])
 ) ON ROWS
FROM
 [Adventure Works]

Incidentally, the Round function is actually a VBA function that is in the VBAMDX assembly that should be installed by default with SSAS (it definitely is with 2008 but I haven’t checked previous versions).

Also note that I’ve used StrToxxx functions – this is to highlight the places where variables can be used in lieu of the literal strings so that this works generically in SSRS (the StrToValue value in the PagePoint measure is the page size). Finally, note the use of the NonEmpty() function in the main SELECT statement – if the dataset that you’re paging through doesn’t remove empty values then don’t remove them here.

So that’s the pager, which in combination with the SubSet() function will facilitate custom paging. As a general purpose pager it will need some tweaking as it doesn’t scale very well (try running it against the Customer dimension in AdventureWorks and you’ll see what I mean), but for limiting the amount of data displayed on a chart it works.

I’ve created a sample report based on AdventureWorks which can be downloaded from my SkyDrive (note: you will need to change the connection to point at your Analysis Server if it isn’t localhost).

Tuesday, 24 November 2009

SSRS gauge custom labels

I’m not an artistic person – hardly the best attribute for a sometime report designer, so most of my dashboard efforts are primarily functional and secondarily aesthetic. For this reason I have thus far avoided the gauge control on the grounds that if something business critical (e.g. expenditure exceeding income) needs to be spelt out using pretty colours and dials then the consumer of that information probably shouldn’t be in a decision making capacity. Back in reality, where the world does not revolve around my dogmatic prejudices and the fact is that gauges are a very useful way of indicating critical success factors in a business without frying the mind of the busy executive.

So after years of avoidance I found myself having to use these dreaded controls in a SSRS 2008 project against an SSAS 2008 data source. All I had to do was display cost v income – 2 measures. This should be simple thought I foolishly only to be still working on the same task 2 days later. Much of this time was spent getting the range of the gauges to be dynamic according to the upper and lower limits of income and expenditure – but one of my biggest bugbears was displaying anything other than numbers on the scale.

Basically, I wanted numbers to be displayed in the most concise format possible so that very large values don’t crowd the scale - for example:

2500 should be displayed as 2.5k

1,000,000 should be displayed as 1m

1,000,000,000 should be displayed as 1bn

First thing I did was create an SSRS (VB) function that converted a double value to this concise representation:

Public Shared Function DoubleToConcise(valueToConvert As Double) As String

Dim negator As Double = 1
Dim value As Double = Math.Abs(valueToConvert)

If valueToConvert < 0 Then negator = -1

If value >= 1000 Then
   If value < 10000 Then
       Return ((value/1000)*negator).ToString() & “k”
   Else
       If value < 100000000 Then
           Return ((value/1000000)*negator).ToString() & “m”
       Else
           Return ((value/1000000000)*negator).ToString() & “bn”
       End If
   End If
End If

Return (value*negator).ToString()

End Function
(Note the use of the negator value and Abs function to save having to include negative integer ranges in the If statements) The only problem with this function is that the gauge still expects to be given (and display) numerical ranges, which means I’d have to return a double (rather than a string) value – which is fine except for the fact that 100,000 is now being shown as 0.10 with no qualifying suffix. After much gnashing of teeth and wailing I realised the solution was actually quite simple – basically split the function into two:
  • one to return the numeric value
  • another to return the suffix

Like so:

Public Shared Function DoubleToConciseValue(valueToConvert As Double) As Double

Dim negator As Double = 1
Dim value As Double = Math.Abs(valueToConvert)

If valueToConvert < 0 Then negator = -1

If value >= 1000 Then
   If value < 10000 Then
       Return (value/1000)*negator
   Else
       If value < 100000000 Then
           Return (value/1000000)*negator
       Else
           Return (value/1000000000)*negator
       End If
   End If
End If

Return value*negator

End Function

Public Shared Function DoubleToConciseFormat(valueToConvert As Double) As String

Dim value As Double = Math.Abs(valueToConvert)

If value < 1000 Then
   return String.Empty
Else
   If value < 10000 Then
       Return "k"
   Else
       If value < 100000000 Then
           Return "m"
       Else
           Return "bn"
       End If
   End If
End If

End Function

We then apply the DoubleToConciseValue function to all values passed into the gauge (pointer, scale limits, etc) so instead of:

=Fields!MyValue.Value

We do so:

=Code.DoubleToConciseValue(Fields!MyValue.Value)

But what about the suffix on the scale? This is where that old chestnut the format string comes in. Basically we can pass literal values into format strings and they are displayed verbatim – e.g. applying the format string “0.00hello” to a value of 156 would give us “156.00hello”. So we put the following formula in the FormatString property of the scale’s LabelStyle property:

=”£0.00” & Code.DoubleToConciseFormat(Fields!MyValue.Value)

Which gives us the finished gauge:

There are limitations to this approach in that it pretty much requires that at least some of the output is numerical, although even this limitation could be overcome with innovative use of IFormatProvider in a custom assembly.

Monday, 23 November 2009

Wonderful Windows 7

As I’ve been using it since the RTM became available on MSDN in August, it’s high time I put some praise Windows 7’s way.  Apart from all the snazzy UI features, much more efficient use of hardware (than either Vista or XP), and the oh-so-easy ad-hoc networking - the feature that I’m currently in love with is Windows XP mode.

Some time ago I bought a copy of Paint Shop Pro X2 for my copy of Vista.  It would run first time and then crash without fail thereafter.  Something to do with ntdll apparently – although given that (apart from PSP) I never had a single application compatibility problem under Vista, I suspect it’s more to do with Corel than Microsoft.  As I did buy X2 because it was compatible with Vista I found this very annoying to say the least.  Anyway, I decided to try it under Windows XP mode and it works a treat.

What I like about Windows XP mode is that it’s a classic case of using existing technologies in a different way to solve an otherwise difficult problem.  The integration is flexible and (almost) seamless.  Not only that, but it’s a great way to sandbox applications of less than stellar quality.

Microsoft get a lot of criticism (oddly enough from people who still choose to use their products despite the prevalence of alternatives) simply because they’re Microsoft.  I think they deserve some kudos for Windows 7 because it really is a pleasure to work with.

Multicoloured column charts in SSRS 2008

Perhaps I’m being as dumb as a bag of spanners but I’ve really struggled with getting multi-coloured column charts in Reporting Services 2008 without putting the category data on the series axis, which then creates all sorts of problems with sizing and grid lines. It seems that Dundas type charts don’t do this easily, although the Nevron ones do (if you know where to look). However I’m not forking out for third-party charting controls just to get nice colours.

This problem first came to light when doing the QlikView vs SSRS development (QlikView charts are pretty by default). Since then I’ve looked in vain for a multicolored (sic) property similar to charts in Excel.

To illustrate my point, I’ve included a screenshot from Excel where I’ve put together a simple column chart with a category and value axis:

By default this renders all columns in a single colour. However if we right-click any of the columns and ‘Format Series Data’ then we can vary the colours by data point:

Anyway, after much gnashing of teeth and wailing I chanced upon this article which got me thinking about a simple VB.NET function that could be used in the fill property of the columns. Thereafter the solution was quite simple. Basically you need to add the following function to your report code (remembering to add a reference to the System.Drawing assembly):

Public Shared Function GenerateRandomColor(ByVal seed As Integer) As String 

Dim randNum As New Random(seed)
Return System.Drawing.ColorTranslator.ToHtml( _ System.Drawing.Color.FromArgb( _ randNum.Next(0, 255), _ randNum.Next(0, 255), _ randNum.Next(0, 255)))

End Function

Then right click the columns in design view to bring up the Series Properties dialog and go to the Fill section:

Enter the following expression for the Color formula (obviously remembering to replace the fieldname with the field being used on the category axis):

=Code.GenerateRandomColor(RunningValue(Fields!MyFieldName.Value,Count,Nothing))
 

That’s about it really. A bit convoluted just for random colours but it works. Despite the fact that I will kick myself if this is already a supported feature (probably a checkbox marked ‘multicolored’) I would be really grateful if anyone could enlighten me.

Sunday, 22 November 2009

QlikView and Reporting Services

I recently downloaded the Office 2010 and PowerPivot Betas so I could compare Microsoft’s current and near-future BI offerings against the competition – which in this case is QlikView. As it is, I haven’t actually had a chance to muck around with PowerPivot yet so I’ll just summarise what I found about QlikView vs SSRS.

Firstly, I should qualify this by pointing out that although it may not be comparing apples with oranges, it could be considered comparing Granny Smith with Cox’s. In other words, while QlikView and SSRS do pretty much the same thing, they do not have parallel feature sets or business contexts. However, there are a lot of BI implementations taking place where the people making the decision will see it as either/or.

As a long-time SSRS user it was going to take a lot to impress me, but QlikView has achieved this. More because of its reporting tools than its in-memory database. Anyone who claims it negates the need for a data warehouse should be dismissed with a withering glance as this simply isn’t true. Like any application, if you put crap in you will get crap out. The fact is that you need to put as much effort into preparing your data as you would when building as SSAS cube. However slightly less effort is required thereafter and it’s very quick with the data processing. But it looks pretty and makes configuration of most common charting features (e.g. median lines, multi-coloured bars) a simple matter of selecting a checkbox. Dashboards can be put together in minutes – good ones in less than an hour.

There are two killer features: chart scrollbars and parameter selection.

Now this may sound trivial, but if you’ve every tried to present a bar chart with 100 different data points then you’ll realise how clever the first of these is. Basically all charts that have data points extending beyond their defined area will put a scrollbar on the axis. Not the chart, the axis. So the legend stays fixed and you can scroll through all the data items within the small space allocated to the chart on the dashboard. There are some clever implementations of this in Excel but they are non-standard and you sure as heck can’t do it out of the box in SSRS. The SQL Server and Office teams need to sort this out pronto!

Parameter selection in QlikView is in a completely different league to SSRS. Basically the list of parameters have mutual filters and allow Ajax-style selection. So a list of 100,000 parameters doesn’t slow things to a snail’s pace or make it difficult to find items. It’s most impressive although it does highlight one of the major shortcomings of QlikView – namely that it doesn’t understand recursive hierarchies (although if anyone would care to dispute this I’d be most grateful). Feeding in a table with recursive keys leads to somewhat unexpected and unpredictable results. and even does recursive hierarchies (see comments).

The only other drawbacks I’ve found so far to QlikView are around security and customisation. Basically it isn’t too easy to do either, certainly nothing to match the logic-based security of SSAS or flexible security of SSRS. Also the drill through isn’t quite as intuitive, even if it is logical (charts drill through to more charts).

Overall I’d say QlikView is a very impressive product but it won’t preclude the need to get your data straight first. If Microsoft want to stay ahead then they need to get scrollbars on the chart axes and much better parameter selection than SSRS currently provides.

I haven’t covered QlikView as a server product (we were working with the desktop edition) so I suspect this will not be my last post on the topic.

Thursday, 19 November 2009

Hierarchies, ParallelPeriod, and the Tablix

I’ve had an interesting week or so doing some proof-of-concept work trying to replicate some QlikView reports in SQL Server Reporting Services 2008 (which is a story for another day).  One of the things that came up was a tabular report that should have been oh-so-simple but wasn’t.

To cut a long story short here’s roughly what it needed to look like (except that there would be values in more of the month cells):

Country 0506 0607 1 2 3 4 5 6 7 8 9 10 11 12
United States 3,079,806 19,471,989 1,321,439 2,497,472                    
Canada 13,208,634 5,615,169                        

The easy way to do this would have been to return a dataset with 14 columns – 2 for the annual totals and 12 for the monthly totals – and put them into a table.  However, this would have been a maintenance nightmare as well as quite slow to render.  The alternative is a matrix-like report.

We need the measures for each month by country so we can feed it into the matrix.  We can use a row total in SSRS to produce the current year total, but we need some way of producing the previous year total.  Because we’re grouping the previous year annual total by country, it makes sense to repeat it for each occurrence of country in the dataset regardless of the actual month.  So our dataset should look something like this:

Country Month Reseller sales amount Previous year total
Canada 1 438,822 3,079,806
Canada 2 658,166 3,079,806
United States 1 1,321,439 13,208,634

Incidentally, this is a very good example of why the Tablix is reason enough to upgrade to SSRS 2008.  A matrix would allow us to dynamically expand the columns to include all the months (without having to create 12 columns), but a Tablix makes it easy to add the two annual total columns.

The problem I struggled with was how to get the previous year total for that country to appear in every row.  It didn’t take long to head in the direction of the ParallelPeriod function, but it would only return data for the month in the previous year (rather than the whole year).  In the end the answer was obvious – somehow I needed it to work not against the current member (which was a month) but against the year to which it belonged, in other words it’s parent year.  Actually the year is the great-grandparent or in MDX parlance, the Parent.Parent.Parent.

So here's the MDX:

WITH
    MEMBER [Measures].[LastFiscalYearResellerSalesAmount]
        AS
        (
            ParallelPeriod
            (
                [Date].[Fiscal].[Fiscal Year],
                1,
                [Date].[Fiscal].CURRENTMEMBER.PARENT.PARENT.PARENT
            ),
            [Measures].[Reseller Sales Amount]
        ), FORMAT_STRING = "Currency"
SELECT
    {
        [Measures].[Reseller Sales Amount],
        [Measures].[LastFiscalYearResellerSalesAmount]
    } ON COLUMNS,
    NonEmpty
    (
        (
            [Date].[Fiscal].[Month].ALLMEMBERS,
            [Geography].[Country].[Country]
        ),
        [Measures].[Reseller Sales Amount]
    ) ON ROWS
FROM
    (
        SELECT
            [Date].[Fiscal].[Fiscal Year].&[2003] ON COLUMNS
        FROM
            [Adventure Works]
    )

It selects against the 2003 fiscal year using a sub-select.  The advantage of this is that you can easily replace it with an SSRS parameter (by using StrToSet).

I’ve put together a sample report which can be downloaded from my SkyDrive here.

Friday, 30 October 2009

Great myths of database design

I have a really pressing deadline and consequently very little time to spare – but I’ve found something sufficiently annoying that not to rant about it would be illogical.  Basically it’s big denormalised tables.  There, I feel better now…almost.

Many times over the course of my IT career when enquiring as to why someone has not bothered to design their database properly, I hear something along the lines of “we decided against a normalised design for performance reasons” or “it’s not appropriate to the domain model”.  I have yet to see a case where this is actually true (and I’ve been round the block a few times).  The real reason is usually either a lack of competence or simply because things were done in such a hurry that good design didn’t seem a priority.

Of course right now if anyone is actually reading this, at least one person will be thinking “but my case is special!”.  To these people I say: “no it isn’t”.  Unless you’re working for Google, the SharePoint team at Microsoft, or a similar mass-market product/service vendor with a requirement to scale out to massive volume and throughput – then it is highly unlikely that your project requires that level of ‘innovation’.  Let me make a bold assertion right now: you are probably not as smart as Edgar Codd (and if you are then it’s likely that you agree with me).

The relational model is a rare thing in the IT world: a methodology that has lasted more than 5 years.  Dr Codd also coined the term OLAP (amongst other things) so what he didn’t know about managing data probably wasn’t worth knowing.  That’s not to say that things haven’t moved on since then (e.g. the removal of business logic from the data layer, moving the conceptual layer into the domain model, etc) but the core principles of normal form have not changed.  Every database should exists in normal form somewhere, even if application or task specific abstractions need to be used (and in my experience they rarely do).

My recommended metrics for table design are that tables should ideally have between 3 and 12 fields.  The practical limit is 20 – anything more than that and you need to break the tables and review whether your view of the data accurately reflects the entity boundaries (if these metrics sound familiar it’s because they are similar to Juval Lowy’s metrics for service design in WCF applications).

By keeping table sizes reasonable it becomes much easier to manage indexes.  In most cases I would strongly recommend that all foreign keys are indexed, as well as any fields which are likely to form the basis of search criteria.  There is nothing whatsoever wrong with indexing every field in a table.  Further, I would recommend not using natural keys as primary keys – use surrogates instead (for further information see Jamie Thompson’s article here).  Implicit in this is that all foreign keys should reference the surrogate key.

One further thing I should clarify is that the conceptual model does not have to match the logical model.  To put this in context: normalised databases do not preclude domain-driven design.  This is what we have data persistence frameworks for (i.e. to act as a broker between the two models).

There, I feel much better now.