Pensieve

Saturday, April 15, 2017

I've Moved

I've moved!

Now that I'm blogging again, I am writing new posts over at http://blog.jeffgabriel.com

Wednesday, December 19, 2012

ASP.NET MVC 4 Execution Sequence

download files

I recently took a deep dive into the internals of the ASP.NET MVC code to facilitate a design discussion around web application extensibility. MVC is extremely well suited to extensible use, providing an opening at just about every step in the execution life-cycle

I created a detailed sequence diagram of the action execution path, and something of an overview of the request life-cycle. I have glossed some details - especially concerning alternate paths for asynchronous vs. synchronous execution.

I don't have a lot to add beyond the notes I took while I created a couple of sequence diagrams, but I have shared both the images and the original UML diagrams project for VS 2012 if you may find them useful. If you improve or expand upon them, please let me know. Enjoy.

Wednesday, December 15, 2010

Script Block Type and Dynamic Script Loading

Obviously I haven't written in a while, but after spending a few debugging hours on this simple issue I hope to save somebody some time.

I am re-writing a widget framework for a client so that all widgets are loaded dynamically client side after page load. Widget content may be complex and include all manner of scripting.

I was having problems getting javascript blocks in the dynamically loaded content to execute in any other browser than Firefox. I was also seeing some blocks execute in the other browsers, but not all and couldn't figure out what was special about the syntax, methodology, or source of those that would work.

I finally figured out that it was the 'type' attribute on the script blocks themselves. We had some script tags with 'ecmascript' and others with 'javascript'. The 'ecmascript' type is not dynamically executed by webkit browsers, or IE; 'javascript' type blocks are. This includes omitting the type attribute altogether as these browsers assume javascript when not otherwise indicated.

Perhaps this script type exclusion is well documented somewhere, but most discussions surrounding problems loading scripts dynamically center on getting around same source issues. Here it is simply a matter of type.

Take home point: always indicate 'text/javascript' if you include the type attribute.

You can test for yourself with the code below. Change the type in the content page to see the background behind 'HI' turn red in browsers other than Firefox.

Simple script loading page...

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">

<head>

<script type="text/javascript" src="//ajax.googleapis.com/ajax/libs/jquery/1.4.3/jquery.min.js"></script>

<script language="javascript" type="text/ecmascript">

    $(document).ready(function() {

        $("#target").load("/testcontent.htm");

})

</script>

</head>

<body>

<div id="target">

</div>

</body>

</html>

Simple Content Page:

<script type="text/ecmascript">

    $("#test").css("background-color", "red");

</script>

<div id="test" style="height:20px;width:20px;">

HI!

</div>

Wednesday, September 16, 2009

Building General Purpose Lambda Expressions

I was fascinated a while back to find the use of the Expression class to build general purpose lambda expressions while researching the use the Repository pattern and the Entity Framework. I didn’t look into it much at the time, but have recently had the time to dig a little deeper.

By general purpose lambda I mean using classes in the System.Linq.Expressions namespace to build a run-time lambda from parameters that let you manipulate which class, method, and properties are used to build the expression executed as a delegate.

Let’s say you have an entity with several properties by which a user may want to sort a list. For example a Person with FirstName, LastName, DateJoined, and UserRank.

Person
FirstName	string
LastName	string
DateJoined	DateTime
UserRank	Int32

You have an instance of a generic List and want to sort it according to a user’s selection on a column. Of course you can probably think of some other ways to get this done, but it’s a good opportunity for a general purpose lambda (and actually just a neat thing to do).

We can start to understand what we need if we decide that we’re going to use the Sort operation on List, and specifically the overload that takes a Comparison delegate. If we were to hard-code our delegates we would have:

    public enum SortBy
{
FirstName
,LastName
,DateJoined
,UserRank
}

Comparison<Person> fnameCompare =
(x, y) = x.FirstName.CompareTo(y.FirstName);
Comparison<Person> lnameCompare =
(x, y) = x.LastName.CompareTo(y.LastName);
Comparison<Person> dateCompare =
(x, y) = x.DateJoined.CompareTo(y.DateJoined);
Comparison<Person> rankCompare =
(x, y) = x.UserRank.CompareTo(y.UserRank);

We could do a switch statement and sort with the correct delegate. However, with the availability of a "CompareTo" method on every type on which we want to sort, and a common delegate type with basically the same expression syntax, it looks like we can do better.

The goal of our expression building will be to build the expression of the form (x,y) = x.[FieldName].CompareTo(y.[FieldName]) where x and y are Person type input arguments and the CompareTo operation is being executed against the underlying field type.

In order to build such an expression we will need parameters for the delegate, which in this case is an x and y parameter of the type Person since the Comparison delegate signature is

public delegate int Comparison<T>(

 T x,
T y
)

var xparam = Expression.Parameter(typeof(Person), "x");
var yparam = Expression.Parameter(typeof(Person), "y");

We are also going to need references to the field on each delegate parameter so we can run the CompareTo operation on fields involved:

var xprop =
Expression.Property(xparam, this.SortBy.ToString());
var yprop =
Expression.Property(yparam, this.SortBy.ToString());

All 4 of these statements are creating expressions. In the first case we are creating instances of ParameterExpression, and in the second case MemberExpression. I found it was most helpful to my understanding that each concrete type of Expression is adding a node to the parse tree. As such, picking the right type of Expression to represent what you want is key to building the pieces of your overall lambda expression.

Note in the given expression, a local field with the SortBy enum has values equal to the field names on the target type – this is a stand-in for expressing a string field name.

Since we now have our delegate parameters, and the participating fields from those parameters, we can build the call to CompareTo. Since we want to add a method call to the tree, we will build a MethodCallExpression using the static Expression.Call helper method:

BindingFlags flags = BindingFlags.FlattenHierarchy |
        BindingFlags.ExactBinding |
        BindingFlags.NonPublic |
        BindingFlags.Public |
        BindingFlags.Instance;

MethodInfo info = xprop.Type.GetMethod("CompareTo", flags, null, new Type[] { xprop.Type }, null);

var methodCall = Expression.Call(xprop,info,yprop);

I used an overload that takes a MethodInfo because the underlying FindMethod used in the Expression class does not specify "ExactBinding" and fails because it also finds the CompareTo overload with the object type parameter. Also note that the ParameterExpression instance xprop has a handy Type field that provides the type of the referenced field on the MemberExpression type to which it was bound.

With these 5 expressions ready to go all we need to do is generate a lambda expression of the appropriate delegate type and compile it:

var sortExpression = Expression.Lambda<Comparison<Person>>(
        methodCall
        , xparam
        , yparam);

return sortExpression.Compile();

Place this code in a method with a return type of Comparison<Person> which could be called to retrieve the delegate and sort your list:

myList.Sort(GetSortExpression());

Some good discussions and samples on the topic:

http://rogeralsing.com/2009/03/23/linq-expressions-from-and-to-delegates/

Nate Kohari’s use in Community Server 5’s REST API: http://kohari.org/2009/03/06/fast-late-bound-invocation-with-expression-trees/ and the related Rick Strahl entry: http://www.west-wind.com/weblog/posts/653034.aspx

http://www.codeproject.com/KB/recipes/Generic_Sorting.aspx

http://www.codeproject.com/KB/architecture/linqrepository.aspx

Monday, August 17, 2009

VS 2008 SharePoint Workflow Templates on x64

As acknowledged by MS in the readme file, the SharePoint workflow project templates for sequential and state machine workflows do not work on an x64 machine.

The error is blunt and unhelpful: “A 32-bit version of SharePoint Server is not installed. Please install a 32-bit version of SharePoint server.”

The hurdle to developing a SharePoint workflow in no way requires such involved effort.

Instead:

Create a standard WF workflow from the given .Net 3.0 template of your choice
Add a file called feature.xml with the following content:
<?xml version="1.0" encoding="utf-8" ?>
<Feature xmlns="http://schemas.microsoft.com/sharepoint/"
         Id="My generated unique GUID"
         Title="My Workflow"
         Description="This feature creates my workflow."
         Version="1.0.0.0"
         Scope="Site">
    <ElementManifests>
        <ElementManifest Location="workflow.xml"/>
   </ElementManifests>
</Feature>
Add a file called workflow.xml with the following content:
<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
    <Workflow
        Id="726DC49B-B3DE-455d-8ACB-A8CA86894F87"
        Name="Expense Report Approval Workflow Template"
        Description="This workflow template enables expense report approvals and payment"
        CodeBesideClass="MyWorkflowNamespace.MyWorkflowClass"
        CodeBesideAssembly="MyWorkflow,Version=1.0.0.0, Culture=neutral, PublicKeyToken=MySigningKeyToken">
        <Categories></Categories>
        <MetaData></MetaData>
    </Workflow>
</Elements>
Create a folder structure in your project for TEMPLATE\FEATURES\MyFeatureFolder
Add a file called install.bat which automates the tasks of gac registration, feature file copying, feature installation, and IIS resetting. Set this file to be called by a post-build action. The file should contain something like:

@SET TEMPLATEDIR="C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\TEMPLATE"
@SET GACUTIL="C:\Program Files (x86)\Microsoft Visual Studio 8\SDK\v2.0\Bin\gacutil.exe"
@SET STSADM="C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\BIN\STSADM"

Echo Installing in GAC
%GACUTIL% -if bin\debug\MyWorkflow.dll

Echo Copying files
xcopy /e /y TEMPLATE\* %TEMPLATEDIR%

Echo Installing feature
%STSADM% -o installfeature -filename MyFeatureFolder\feature.xml -force

ECHO Restarting IIS Worker process
"c:\windows\system32\inetsrv\appcmd" recycle APPPOOL "SharePoint Apps"
Add an OnWorkflowActivated activity to your workflow from the SharePoint activities in your toolbox.
Optionally change the base class on your workflow program class to SharePointSequentialWorkflowActivity
Add the following field declarations to your workflow class:
public Guid workflowId = default(System.Guid);
public SPWorkflowActivationProperties workflowProperties = new SPWorkflowActivationProperties();
Set the WorkflowProperties property on your OnWorkflowActivated activity equal to your new workflowProperties field.

There is another workaround posted.

I won’t hold my breath, but you can also go vote on the bug in hopes that perhaps a patch will be released. I also imagine that packages for VS ‘10 will not have this problem.

Monday, July 06, 2009

LINQ, Persistence Ignorance, and Testing with Data Context/Entity Framework

One thing I really try to avoid when writing tests is testing things that just aren’t going to break; or at least are of no concern to the application code I wrote. Database IO certainly fits this description, and yet I have found myself testing it by consequence of the desire to test Linq-based statements embedded in the data access code.

Since other people have encountered and attempted to solve the same problem, a few hours of reading showed that in order to remove database IO testing from my tests, and to improve the design of the data access layer, it is necessary to implement some pattern of persistence ignorance like the Repository.

In short form a repository allows business code to ask questions of the data layer without knowing anything more than the interfaces of data-storage objects. Want all the customers with outstanding invoices? Get a list of ICustomer objects back from the CustomerRepository.CustomersWithOutstandingInvoices method. As a business layer developer I have no idea what you did to return this list of customers, and don’t care. You don’t expose your DataContext (or other DAL implementation) to me, and I don’t write LINQ directly against the Customers table because the management of the DataContext and particulars of locating outstanding invoices may be too wonderful for me to know. Pretty standard separation of concerns, and very simple to implement. There are a few unique issues in a LINQ-centric world:

How can I take advantage of delayed execution?
How can LINQ best be supported outside the Repository
How can I mock-up data-centric tests when I don’t have a database to execute against?

If you hide LINQ execution behind a repository you don’t really want to provide access to ObjectQuery objects or hand out Table<T> references, as you are tied immediately back into what you were hiding. This being the case you really need to execute the queries before they leave the repository*. Performance should come from good caching techniques, and by limiting the scope of queries through tight definition of access to the data.

*update* It isn't really necessary to disconnect the query in any way that breaks delayed execution. You can maintain all the benefits and still hand back an IQueryable<T>. However, it would then be up to developer discipline to not do further manipulation that takes advantage of the underlying data provider's specific types.

LINQ can work on any IQueryable, so handing back objects that can be used in LINQ syntax queries outside the context is as easy as handing back List<T>, or IQueryable<T> references. Referring back up to issue 1 above, these post-repository queries will be disconnected from the database.

When it comes to testing, we played around with a number of different mocks and tricks to fool LINQ into executing disconnected; fake data contexts, entities without db connection strings, overrides and events that modify queries, etc. It turns out that the simplest way (in my view) is to use the repository to simply remove the database from the equation – that was after all the goal of this whole exercise. However, rather than go with a repository directly exposed to business logic code, I prefer to put what I will call a Provider in front of the repository. This ‘Provider’ is really just another sort of repository as some envision the concept, except that it contains all the LINQ queries and specific entity related methods and takes an IRepository implementation that simply has CRUD operations exposed with internal knowledge of the DAL implementation contained in this simple repository. Why?

It’s easier to create a general purpose IRepository interface for use in any project.
With a simple IRepository interface you can create a simple general-purpose FakeRepository for use in testing.
Once you can have LINQ statements dependent upon a single interface, you can inject the dependency and thereby replace the source of data during testing. You can test your LINQ query logic without going to the database.
The logic used to obtain entity objects from methods like “CustomersWithOutstandingInvoices” is still removed from knowledge of the DAL, moving persistence ignorance as far up as we can.

How does this all come together?

You create a concrete repository that has a concrete ObjectContext or DataContext.

With EF ObjectContexts (my preference) you implement the Get<T>() functions on your repository by using CreateQuery methods:

public IQueryable<T> Get<T>() where T : class
{
  return _context.CreateQuery<T>(typeof(T).Name)

 as IQueryable<T>;
}

You create a concrete ‘Provider’ that takes an inject-able IRepository object in the constructor.

Write your Provider LINQ queries against Get<T>() statements from the repository

Write tests against your provider, inject a FakeRepository that has lists as it’s data source. Add objects to your Provider in test setup and confirm proper retrieval/manipulation in tests.

You can download our assembly with an IRepository/FakeRepository implementation from:

http://atgitesting.codeplex.com/

Inspiration and understanding from:

The Repository Pattern Explained

Andrew Peters’ Blog » Blog Archive » Fixing Leaky Repository Abstractions with LINQ

Diego Vega - Unit Testing Your Entity Framework Domain Classes

Dynamic Queries and LINQ Expressions - Rick Strahl's Web Log

Friday, April 17, 2009

Working with detached entities and Linq to Sql

On my current project we are using Linq to Sql, and as it is a web application are working with the DataContext in limited scope only during data operations. This means that objects generally need to be attached to the context in order for all operations to succeed (all related parent objects must be attached when inserting children, changed objects before updates/deletes)

One annoyance I have is that I can't really tell if I should attach the object, insert it, or just update it. If every operation is done in an entirely atomic fashion, or you associate objects by key values only; this can be avoided. Yet that robs me of the features I like such as adding any number of new objects, associate object instances together and let the context figure out the order of operations. With fairly atomic data operations defined in our data class we end up blind to the state of the objects when dealing with a new context.

To help out with this I have created the following extension on DataContext...

public static class DCExtensions{

 public static void SafeAttach<T>(this DataContext context
                                             , T entity)
     where T : class
{
   Table<T> entityTable = context.GetTable<T>();
   if (entityTable == null)
       return;
   T instance = entityTable.GetOriginalEntityState(entity);
   if (instance == null)
   {
       if (!entityTable.Contains(entity))
           entityTable.InsertOnSubmit(entity);
       else
           entityTable.Attach(entity);
   }
}
}

I can then have a method like the following that isn't concerned with entity state:


internal ChildObject AddAssociation(ParentObject1 parent1
                                     , ParentObject2 parent2)
{
   using (var dc = new MyDataContext())
   {
      dc.SafeAttach(parent1);
      dc.SafeAttach(parent2);
      ChildObject child = new ChildObject();
      child.Parent1 = parent1;
      child.Parent2 = parent2;
      dc.ChildObjects.InsertOnSubmit(child);
      dc.SubmitChanges();
      return child;
    }
}

Tuesday, March 24, 2009

IIS 7 Wildcard mapping issues

I recently ran into an issue while converting a site to use the managed pipeline in IIS 7 that took a while to fix but turned out to be one single attribute in web.config. This also turned out to be the answer to something we had simply hacked around before.

I was converting this asp.net website into an application, and configuring it to run under iis 7. The previous developers had setup a url rewriting scheme in global.asax intended to run for extensionless urls such that http:/example.com/placename would properly respond with the information for said place name.

I moved the rewriter logic into a new IHttpModule implementation and configured it in system.webserver. This helped to get my requests mapped properly but the page came up with an error stating: "Session state can only be used when enableSessionState is set to true, either in a configuration file or in the Page directive.". This was an unfortunate error as I had everything already in place for session to work. After a good bit of digging I found that the problem is that when you have a custom module in place to handle all requests you lose the functionality of any other configured module which has a pre-condition for managedHandler. Session is one of these, as is FormsAuthentication.

You can either go through and remove each module and replace with a config that has an empty preCondition attribute, or you can just force all with the following:

<modules runallmanagedmodulesforallrequests="false">

This turned out to be the best option in my case. As I said, this also helped solve an old problem where we would get authentication failures when going to extensionless urls. The behavior is a bit perplexing - logged in when on default.aspx, logged out when at /. It turns out this is the source of that problem too.

The answers are out there, I just had a hell of time finding them so I thought I would re-share.

Tuesday, February 17, 2009

Community Server Training Class Announced

I haven't been into writing much content here lately as I have been focusing on class materials and labs for our upcoming Community Server training class.

I am really excited about this, and even more so today as we finally announce the date and costs and open up registration! The class will be April 27th-29th and will cover everything from basic site configuration to creating custom chameleon controls. At ATGi we're very excited, and a little bit nervous, but I think the content is excellent and can't wait to give the class.

If you are working with Community Server as a customized community platform, check it out:

http://www.atgi.com/training.aspx

Tuesday, February 10, 2009

Bright Cove Controls for Community Server

I have published a project on code plex with a set of tools and controls for working with Bright Cove video inside Community Server.

It includes

A Bright Cove video File Viewer
A Bright Cove video Content Fragment
List, data, and image Chameleon controls that work directly against the BC Media API.

Use it if you need it, join me in updating it if you need more features.

Tuesday, November 25, 2008

SEO and Community Server

I am not an SEO expert but have been working with one to optimize Community Server urls and pages for a client's install. I recently read this post on CS forums and it deals directly with many of the same issues surfaced by the expert we have been working with.

It seems that one of the major problems for SEO experts is the forum post perma-link urls which come out as duplicated content urls. Which is to say that threads have links, and so do posts:

http://mycommunitysite.example.com/t/2323.aspx (the thread)

and any number of post urls like this:

http://mycommunitysite.example.com/p/2323/83839.aspx

Where the post ID as the page name is the individual post in the thread.

The simple solution is to use only thread references, right?. This isn't quite so simple if you have multiple pages to display posts. How do you indicate which page of the thread a post should link to? Posts can also be deleted or moved, which could affect page position for every post in a thread.

You can solve these problems in dynamic site links by changing post perma-links in a new ForumUrls provider:

public override string PostPermaLink(int threadID, int postID)
{
int _pageIndex = Posts.GetPageIndex(postID, PageSize, 0,
(int)CSContext.Current.User.PostSortOrder);
return ThreadPaged(threadID, _pageIndex) + "#" + postID.ToString();
}

This methodology has the mild inconvenience of requiring a set page-size for display of posts in a thread.

However, this doesn't solve the problem of what may be indexed. Once a link is stored, it may end up being on the incorrect page if posts are moved or deleted. Setting all of your post pages to noindex can help here. You could also set all of your thread display pages to list a very large number of posts at a time and virtually eliminate pages.

To really deal with posts and threads I think the best way would be to implement a script-based solution. Most crawlers don't implement script, which allows you to hide content from crawlers while still having a user-friendly implementation.

In a thread-centric implementation you could have only the thread page with all content on it, and implement paging via script. Post references are handled via anchors (as they can be in ootb CS).

My preference would be for a post-centric implementation where every post is in fact its own page and you implement the threaded view via client script. Links to 'threads' are in fact links to the thread-starter post.

When a post page is rendered you can dynamically 'locate' the post within the thread based on paging configuration.

Tuesday, October 28, 2008

CS Blog post dates and links

I have been working on another Community Server installation for a client concerned with SEO (more on lessons here later). Duplicate urls are one of the major issues we have tried to elimate in his site. CS publishes lots of urls for the same content, and in particular will display blog post urls that are date based where the date is dependent on the publishing user's time zone or the server time. These probably don't cross over too often, but we needed to eliminate different date-based urls.

Here are the things I learned about CS blog urls while investigating how to change this behavior:

1. During post creation two dates are stored – the server date/time as PostDate and the user’s date/time as UserTime.

2. When posts are retrieved as IndexPost through the SearchBarrel (search, tags/topics) the UserTime is set directly from the PostDate field before requesting the url.

3. When posts are retrieved as WeblogPost through the WeblogPosts component (weblog archive lists, googlesitemap) the UserTime is populated from the UserTime property.

4. The BlogUrls provider uses the UserTime property to createthe url. Based on 2 and 3, this could vary depending on execution path.

5. WeblogPost picks up a third property called CurrentUserTime which is a manipulation of the PostDate to the viewing user’s timezone and is dynamic. This property does not appear to be used anywhere in the SDK.

6. Any object property that is of the type DateTime will be formatted by the default property formatter to be displayed in the current user’s time zone manipulated date and time. In the case of blog post display, this property is supposed to be "PostDate" but actually comes out to be CurrentUserTime because of this property formatting. While this seems like a failure of consistency to me on the part of Telligent, it is not 'dangerous' in terms of SEO because it is simply the text displayed to the user. Changing this behavior would require new IndexPostData and BlogPostData controls to properly override the FormatProperty method and a global replace of their use in the site’s theme files. Not a prohibitive change if required, but probably not necessary.

Our fix here was to create our own BlogUrls provider, override the Post(WeblogPost, Weblog) method, and standardize UserTime to PostDate before getting the url from the base provider.

Friday, August 15, 2008

Section 8.5.2 of CSS 2.1 and the border-color property

I realize this is minutea, but I wanted to get this down for later recall.

First of all, when working with Firefox, I usually find that it is reliably strict, and so developing for FF will allow you to get a result that transfers to other browsers. Of course, sometimes the strictness is annoying but at least it's reliable. In this case I thought I had found something it just wasn't doing right, but giving it the benefit of the doubt on just being a PITA, I looked at the spec and of course FF was proved right.

The problem was that I was setting some border colors for various containers on the page like so (heavily simplified):


.boxtype1
{
     border-color:blue;
     ...other properties...
}
.boxtype1-alert
{
     border-color:red;
     ...other properties...
}

What I was seeing was that boxtype1 was getting a black border color (the color of the 'color' property in the body tag), and the -alert type box was getting the proper red border - only in FF; IE (6 & 7) and Safari were 'fine' in that they displayed my chosen color as the border color. After cursing FF for it's lame implementation I decided to check the spec and found the answer as related to the use of some border-side specific styling and the order of styles implemented.

The fact is that other declarations subsequent to the style I was using to set color were setting other properties on left/right/top/bottom borders without setting the color:


.boxtype1
{
 border-left:solid 2px;
}

This doesn't leave the color as previously set, but makes it work as "inherit" or default because it was not set specifically.

The solution is to either make sure the order of execution is correct (a fragile waste of time) or set each border specifically, making sure it will take on the style no matter the order.

You can see for yourself that if you have the following html:


    <div class="boxtype1">
        <p>
            Some text with a border around container.
        </p>
    </div>

and apply the following styles:


body
{
    color:Blue;
}
.boxtype1
{
    border:solid 1px;   
}
.boxtype1
{
    border-color:Red;
}
.boxtype1
{
    border-left:solid 2px;   
}

You will get a red border all the way around in IE, but will have a blue, 2px border on the left in FF.

Monday, August 11, 2008

Asp.Net AJAX Client Side Templated Data Bound Control

Source Code Available from Hydrus Software

I recently read these two articles by Dino Esposito about ajax templates (part1, part2) and was intrigued by the possibilities. I spend a lot of time working on customizing Community Server which is one page after another of templated data-bound lists. In a number of situations we have needed to customize lists to respond to client input. This can present performance problems when lists are in tabs, or the page is very heavy.

In any case, I had some time available and worked out a client script control that implements the template builder Mr. Esposito showed in part 1 of his article.

I created a server control that allows you to specify header, item, and footer templates for a basic data list that will be bound on the client based on either a given web service method, or on a data-source provided client side. The templates are rendered server-side before being passed to the client behavior which allows you to use other server-side controls in the development of the templates.

I had not built an asp.net ajax client script control before, but the magic lies in the IScriptControl interface. There are plenty of articles out there on this, so I won't go into it. At a high level, this interface provides a way for you to instantiate your client-side object (behavior) with properties set on your server side control.

Finally, in order to make the server-side declarative coding a bit cleaner, I implemented the client-template replacement string as a control. Once you have this nifty server-side control, you can setup the client repeater with code like this:


<HydrusClient:ClientRepeater runat="server" ID="CustomerList" ServiceMethod="GetCustomers"
        ServicePath="Service.svc" DataBindingFunctionName="MyFunction">
        <HeaderTemplate>
            <table>
                <thead>
                    <tr>
                        <td>
                            <asp:Label runat="server">Full Name</asp:Label>
                        </td>
                        <td>
                            <asp:Label runat="server">Role</asp:Label>
                        </td>
                    </tr>
                </thead>
                <tbody>
        </HeaderTemplate>
        <ItemTemplate>
            <tr>
                <td>
                    <HydrusClient:ClientBoundDataProperty runat="server" PropertyName="FullName" />
                </td>
                <td>
                    <HydrusClient:ClientBoundDataProperty runat="server" PropertyName="Role" />
                </td>
            </tr>
        </ItemTemplate>
        <FooterTemplate>
            </tbody>
            </table>
        </FooterTemplate>
    </HydrusClient:ClientRepeater>

A few words about getting the control to work properly...

While we instantiate the templates into containers in CreateChildControls, the templates are rendered into strings in the RenderChildControls method. These strings must have insignificant whitespace removed.

Because the template strings must be properties on the client-side behavior, we cannot call RegisterScriptDescriptors until after the children have been rendered, thus we make the call in the overridden Render method.

The ScriptManager must have the id of a rendered client element with which to register the behavior when creating the ScriptControlDescriptor. In this case my server control renders a placeholder. You could write this control as a rendered control extension and force the user to identify the rendered element.

Wednesday, October 03, 2007

Community Server and Extended Profile Properties

Anyone who has done much customization for Community Server will eventually run into the need to add more profile properties for users. These properties are really easy to add. The problem is that you cannot easily search on these fields because they are stored as two text columns in the database. There's an old add-on that offered one way to get at these values. Another would be do your searching in code since the values are easy to get at once the User objects have been loaded.

While it isn't efficient enough for regular site-based use, and it certainly isn't pretty; you can query the database for these values using text manipulation procedures in Sql Server. Adding first-class columns to the profile table requires a major effort in CS, so this approach can work if you want to add ad-hoc usre reports.

<disclaimer>I really can't recommend the following for adding functionality to your CS search page - but if you manage to use it and have some performance measures, please share them.</disclaimer>

I added three functions to my CS database*:

GetUserPropertyValue - takes the UserId, property name, property type (b,s,i), and settingsID and comes back with the named property value.

GetPropertyStartIndex - takes the property name you're searching for, and the list of property names. Used by GetUserPropertyValue.

GetPropertyLength - takes the same params as above, and returns the text length of the property value.

The start index, and length are what is stored in the property names string. You then go to the property values string to get the value only. I broke this operation up to into separate procedures for readability.

Finally, I created a modified version of cs_vw_Users_FullUser to include values for the fields I was interested in by adding this function call into the select query:

dbo.GetUserPropertyValue(cs_UserID, N'MySpecialProperty', N'S', SettingsID)

This way, I could write a report against all users and search, sort, and filter by custom values.

Download the sql code.

This hack will work for asp.net based profiles too since the methodology used to store these key/value strings in cs_UserProfile (PropertyNames | PropertyValues) is the same as is used for the base aspnet_Profile table for asp.net membership profiles.

*my sql-guru friend John wrote the original manipulation query.

Monday, September 24, 2007

Earned Value Early Results

We have our first reporting period under our belts, and I am very excited about the EV results.

In this fictionalized representation of our first project, we see the period % complete reporting by the team leader:

This evaluation of our status leads to the following period values for EV:

You'll notice that while 145 hours were spent instead of only 120, we also saw 8.2% EV, instead of the planned 6.63%. However, the question we have is, was this enough extra EV to justify the hours spent? In our calculations we have the answer:

We're still all green because the EV was indeed enough to overcome the overspending. Notice that the CPI is roughly $1.02 - this means that the EV just barely covered the cost overrun for the period (we got $1.02 effort out of our team for each project $1 planned). Thus, while we spent over $17K in the period on labor instead of just over $14K, the project is projected to end early, and still come in under budget. What this also shows, is that we'll need to watch this team's hours closely over the coming weeks to make sure this level of efficiency is maintained.

Our second team had a more modest hours overage, but still managed a healthy start on EV:

Leading to an even rosier bottom line report:

So far, this seems like a realistic view of real project progress. Obviously, there's still a ways to go. BTW - it didn't take even close to 15 minutes to enter and analyze the data.

Friday, September 21, 2007

Earned Value Management

One session of a conference I attended way back in March was a discussion about managing projects presented by Juval Löwy. One of Juval's excellent claims was that he had used earned value analysis to keep his management time to 15 minutes every Friday afternoon. He explained he used the rest of his time to become a better architect, and when he was finally too bored with the simplistic task of management, he moved on to architecture.

A great story, and certainly a compelling one for someone that needs to manage on less than 40 hours a week. I have been waiting to use this magical method, so as a recent project was ramping up, I pulled out my notes and hit a few websites to try and prepare for the 15 minute management job. Going on scant examples of exactly the type of project analysis I wanted caused this task to take somewhat more than 15 minutes. I found some helpful examples here and here.

If you aren't familiar with EV, the basic idea is that actual work is tracked against the amount of value accrued by the work completed against the plan of value to have been accrued according to the plan milestones. The notion of Earned Value has to do with tracking accomplished activities, rather than just time spent (I think this works well with my management objectives - I don't care how much time you spent, I care that the work gets done). Each activity has some percentage of value to the overall project. A simple way to get at this in software development is to divide the hours for one task against the total hours estimated for the project. Choosing the proper level of task granularity to track is a matter of balancing ease of reporting with meaningful levels of detail for catching problems before they go out of control. Once the earned percentage values are assigned to milestone tasks, you can either work with dollar figures or just work with hours - it doesn't matter. The important thing is to calculate how much earned value should be accrued at various key intervals along the project path. For instance, if I am checking in on my developers every Friday night, then I calculate EV earned to each Friday. When my developers report in, they tell me how many hours they worked this week, and what % complete they are for each milestone task.

Planned EV will be the number of developer hours available in the time period divided by the total number of project hours. When each developer reports a percent done, you multiply that by the percentage EV calculated for each task against which you are reporting. Summing each EV gives you the period EV which you compare against your goal. You can then use the total actual hours reported as a measure of how much real effort it took to get actual EV. This calculation comes into play as you forecast the future effort expenditure of the project. If you thought you could complete 10% EV with 80 hours, and got 8% EV with 80 hours, your SPI will give you a multiplier for estimating future period effort. This is where the 15 minutes comes in. Each Friday you must inspect your planned EV and and planned Costs against your actual's. You then analyze what variances in the charts mean to you:

PEV way ahead of AEV? You underestimated and should add resources
AEV way ahead of PEV? - You overestimated and may be able to divert resources
AEV=PEV but costs overrun? - You can meet the deadline, but your resources are inefficient - you need to figure out what's bogging down your resources.

You get the idea. Manage next week's resources based on your 15 minutes of EV and cost analysis.

Without a tool to provide EV analysis for me, I turned to Excel. The only gotchas I had in setting up the calculations was discerning the best path for tracking - either project to date sums, or per period values; and getting graphs to show the EV and cost trends. In the end I went with per period values and some extra calculations to show trends and two separate graphs for EV and Cost.

With the plan in place, I simply communicated the project milestone tasks to the team along with target dates and requested weekly reporting of progress. Hopefully this leaves the devs with schedule freedom and a sense of autonomy, while giving me a way to know which adjustments are necessary before we have a big problem.

Orcas Beta 1 and other new stuff

It seems to me that the development world is experiencing another period of rapid evolution again, much as it did in '99-'01 around web application servers, XML, and all the XML spinoff technologies. Interestingly, during the last evolution many people left javascript in the dust; but it has had a massive resurgence to the forefront this time around. I think it also interesting to note that the lastest tools and technologies are clearly evolutionary in that most are still new ways of utilizing the XML breakthrough. Well, I am sure someone more knowledgable wrote about this years ago; but it's clear that it is time to dig deep into the recent mountain of information.

As one who is primarily an MS-based developer, a lot of that mountain I am facing is coming out Redmond lately.

.net 3.0
- Windows Communication Foundation
- Windows Workflow Foundation
- Windows Presentation Foundation
Silverlight
C# 3/3.5 (lambdas, extensions, implicit types, anonymous types, type initializers, expression trees, and LINQ)
AJAX Framework
PowerShell scripting
Vista/Longhorn Server (IIS7 being the big one here so far)

Along with this has been the birth of Ruby and the resurgence of javascript.

Of course, many of these things have been around or brewing for years, but the pressure has mounted as more of the MS stuff nears release.

I've had the opportunity to play with Orcas the last couple of days, and it has been fun. Silverlight programming isn't really something I think I'll do a lot of, but it provided a good way to mess around with XAML and learn about what's available. It also provided a way to get into VS 9, and the new Blend tool.

First, the XAML stuff is cool, especially the web delivered Silverlight. Updating markup, and then watching visuals automatically update was great. However, my artistic skills approach those of a first grader so the appeal is limited. I can see a potential upside in using Silverlight for rich administrative interfaces. I hate all-flash sites because they are usually more about style than substance (i don't need to hear sounds when I drop down menus or wait for transition effects when I go to a new page), but if used carefully and mixed with AJAX it could be good.

VS 9 has some fairly exciting innovations too. I especially like the javascript editor has improvements (though it could be a lot better - see Aptana).

C# 3.5 has some great stuff going on. Scott Guthrie's coverage of LINQ features has been excellent. It does take some time to get used to the LINQ programming model (which utilizes most of the new language features), but it's well worth it. Start investing the time now.

Tuesday, June 05, 2007

Private Workspace Config Mgmt w/Team System

On a recent project our team has been using Microsoft's Team System from within Visual Studio. We are using the built-in capabilities for:

Requirements Definition
Task Assignments
Source Control
Build Server
Bug Tracking
Code Coverage
Unit Testing

So, we're involved in it up to our necks. We've been using pieces of the system for quite a while, but this is our first immersion. The SCM is a little funky sometimes, and really annoys my SCM-geeked teammate.

Honestly, I think it has been a pretty good experience overall. I like the reporting we get out of the box (these only come into play if you do use TS for everything), and there is no question that as an extension of the IDE, having everything available in one familiar place is a big bonus.

The main point I wanted to bring up here is an early issue we had with getting our private workspaces* and build server to have all the right files from the repository. We do not share an overall folder structure, and I don't like forcing developers to have their files in any one place on disk on their machine. I am just too much of a libertarian for that.

Because this project included a couple of frameworks, we decided to install those on every machine, and depend upon the GAC for references. However, we wanted to share a signing key, a shared AssemblyInfo file, a test run configuration file, team build types, along with multiple assemblies not included in the installed frameworks.

For the shared files we decided to use the solution items folder for the solution. In order for the items here to be source controlled, and therefore downloaded from the repository, they had to exist beneath the root workspace folder. We decided that we would mimick the folder structure exactly on disk as it was in the solution items folder. Specifically, we created solution items folders, and then created their counterparts in the root solution folder on disk.

Assemblies were placed in an Assemblies directory and other files in a Common folder (unfortunately, this is still one area where the SCM can still be flakey and occasionally get latest version doesn't seem to pull new solution items).

Now, in each project we referenced the assemblies in the solution items folder rather than locally. We also needed to put links for the common key and sharedassemblyinfo.cs into each project. In order to do this, each csproj file was hand edited with a new "Compile" element:

<Compile Include="..\Common\SharedAssemblyInfo.cs">
<Link>Properties\SharedAssemblyInfo.cs</Link>
<Visible>true</Visible>
</Compile>

The Link sub-element tells the project where to display this file relative to the project root.

We added a "None" element for the key:

Finally, in order for the .testrunconfig file to work, and to see the team build files they had to be included in the solution via the context menu 'include existing item' - for some reason these references did not come along with updated versions of the solution file.

With this all in place, every developer can reliably get latest and build without issue.

*I mean this in the sense of the Configuration Management pattern given by Berczuk which I think is probably normative for most developers - do your dev in a private workspace where you control the versions of the components you are using and the version of the code you developing against. You control when and how your environment changes. This is basically done by downloading a complete environment from the repository to your local workstation.

Generic Mock Factory

I've been involved in VS integration work lately, and within the unit testing framework provided by the integration SDK is an excellent mock factory for any interface based type.

If you are involved in unit testing where you need mocks, this is an excellent generic resource for any project - not just VS integration.

GenericMockFactory.cs