Skip to content
Apr 14 12

Search Engine APIs. Contrast and Compare

by Phil

Finding a search engineIt was with quite a bit of interest that I received an email today from Bing advising that “the Bing Search API will transition to an offering made available on the Windows Azure Marketplace”. Within the PR hype were two pretty crucial pieces of information:

  • Developers can expect subscription pricing to start at approximately $40 (USD) per month for up to 20,000 queries each month.
  • After the transition period [completed in the next few months], Bing Search API 2.0 will no longer be available for free public use.

That’s a big deal. Here’s the full announcement. Why? Because it’s a lot of money for any business or amateur / hobbyist sites or applications to front up with at very short notice. Exchange rates may make this cost even more significant (for example, at time of writing $40 US dollars equated to $48.20 NZ dollars) and a few months may well not be enough time to compare different alternatives, test them and deploy them to a production environment.

So along that line of thinking, what are some of the different options for search APIs? How do their pricing structures compare and what limitations are imposed with each?

Search Engine Cost Per Cap (in queries) Link Notes
Google Free Day 100 Google API All additional requests over the 100 daily request cap are charged at the rate of $5 per 1000 queries, for up to 10,000 queries per day.
Baidu Free Week Quota Baidu API The quota system is not clear on the initial allotment of allowed API calls
DuckDuckGo Free Always None DuckDuckGo API Search engine attribution requested in results
Bing US $40 Month 20000 Bing API

Yahoo! search APIs were deprecated in April 2011 in favour of Bing when Microsoft finally secured a stake in the search giant.

DuckDuckGo is the darling child of bleeding-edge web enthusiasts and power users with its !bang syntax, strong keyboard shortcut support and quirky UX.

Baidu appears on this list because despite being almost exclusively Cino-centric, it commands a very high ranking in global share purely by virtue of the population it serves. However I found it extremely difficult (using the translated English description) to find concrete numbers regarding the Baidu quota system and the complexities around the calculation of API query totals.

A Bung Move for Bing?

With most of the main competitors offering a free service for low-end users or, in the case of DuckDuckGo, a completely free service, it’s difficult to understand what Bing is trying to achieve with this decision. It seems on the one hand that Bing would want to drive as many potential customers away from the “not evil” Google in favour of its own services but charging from the outset across the board for all users would likely achieve exactly the opposite.

Maybe Microsoft are trying to position themselves in a niche within the search market that targets higher-end enterprise customers. Their API is solid, their results generally good and the options around the delivery of those results are very broad but the pricing model still sets a barrier for entry that will in all probability make Bing a second choice at best behind its competitors.

So there it is. Do you know something I don’t? If I’ve overlooked or misunderstood some details around the Bing or other search APIs, add a comment below or tweet me. I’ll be really interested to see where this is going.

Feb 14 12

Putting the “You” in Community – Why You Should Care to Get Involved

by Phil

Speaking as a communityI was very fortunate to have Scott Hanselman agree to stop by The Office for a casual meet and greet with some of our development folks yesterday and it was a really valuable couple of hours. We chatted about general stuff, asp.net stuff, Microsoft management layers (Way more than three degrees of separation between himself and Ballmer. Who knew?) and – resonating particularly with us – community. Scott feels quite strongly about the .net community but thinks it has quite some way to go before we – collectively – are really “there yet”.

Scott is in Wellington this week for the 2012 Webstock conference (great conference, by the way. Really valuable speakers, workshops and networking opportunities. Get along if you can). An interesting observation he made was that people would ask where he was from and when they heard the answer, “Microsoft”, they usually reacted with indifference or borderline sympathy. This is a shame. Microsoft has moved a long way since those days and he’s quite right to say there are more evil companies than this now. There’s like a “Technology Stack of Evil” that puts Microsoft way down the list. They’re only quasi-evil. The diet coke of evil, as the quote goes. Just one calorie; not evil enough.

Microsoft: the margarine of evil

Microsoft: the margarine of evil

I wanted to do some kind of inverted pentagram, but couldn’t visualise how to arrange the members hierarchically (submissions or suggestions gratefully received). The Fox network and large bank CEOs probably fit somewhere in the middle there, by the way.

Anyway, the point – and there is one – is that the .Net Community (that’s probably you if you’re reading this) needs to follow Microsoft’s lead in the open source and community spaces. As a community the observations seem to be that we’re isolated into pockets, not constructive with our interactions with each other or not as engaged with sharing information, code or effort as, say, the Rails community.

That might seem a bit cynical. You look at Stack Overflow and you’d think that any .net question was filled quickly with good, helpful and constructive answers, but when you think about it, that’s an artificial construct that actively drives and promotes that sort of behaviour. Actual contributions by way of blogs, site content or (and this is a big one) open source participation are actually much lower in practice.

Possibly this is because the Microsoft stack makes it so easy to build your own projects and – for the most part – makes it so fun, that people do exactly that: spend their time learning, refining and growing their craft by projects that are personally significant to them. Possibly it’s because people aren’t familiar with how open source works and are reticent about dipping their toes in the water. Or possibly it’s because people are afraid to get involved with an open source .net community project because they haven’t found one they feel passionate enough about and are probably a bit scared they’d only get in the way or be shot down for making mistakes.

Whatever the reason, this is why we as a community need to get behind events such as the Webstock workshops, speaking or networking events and local area user groups. Individuals aren’t going to learn, improve and grow their skills and their craft by staying at home and coding in their living room (although that helps). Get involved! We need to support events that are being organised for .net users; get passionate and contribute to projects that are community-facing and above all be constructive and supportive of those of us who are stepping out of our comfort zones and trying to engage.

This is why Hanselman is so highly regarded in the .net community: because he always makes the time to encourage, lead and support. But that’s a lot of responsibility for one guy. We need more leaders like that who we can look up to in our own regions. Hanselman says “Tweet less, blog more” in his post from early January titled “Your blog is the engine of community”, I’d argue tweet more if that means you are trying to engage in your community and widen your network. Meet people. Talk to them and find out what sort of common ground you have in the .net space. Find ways to collaborate or share knowledge. Argue your case to attend those conference opportunities. Everyone benefits: you learn new stuff, the community grows and leaders emerge that reinforce this pattern so that everyone profits from being better at what they do.

Dec 9 11

Working With Checkboxes and Other Controls in Word Using OpenXml

by Phil

Microsoft’s adoption of the OpenXML format for its Office documents such as Word, PowerPoint and Excel has meant that developers can provide considerably improved experiences over the web and on desktop applications when working with Office documents. For example, it used to be the case that in order to serve up a Word document from a web server, you needed to have Microsoft Office installed on that server. While this might still occasionally be the case, in most instances you can now reference the OpenXml SDK and create a Word 2007 / 2010 document on the fly.

While the OpenXml framework offers a huge improvement in interoperability, it also adds a reasonable degree of complexity if you’re trying to work with attributes of a document that would generally be considered outside of the “routine” tasks of working with text and layout. My specific problem last week was trying to get references to checkbox controls I’d placed in the document and read or manipulate their states. The trick, it turns out, is knowing your checkboxes from your checkboxes. Not all checkboxes are created equal.

Understanding Control Genealogy

The Microsoft Office suite has evolved from a set of applications that employed proprietary binary file formats, windows-centric embedded controls and a litany of other ideosyncrasies that served their purpose at the time, but are little more than legacy relics today. Still – they need to be supported and we are therefore left with quite a diverse range of possible controls that might be found in a given document or spreadsheet.

What this means is that when working with embedded controls – particularly those used in forms such as checkboxes, dropdown lists and the like – we have to be aware of what kind of control is actually being used and thereby how to access it. In working with Word, we’re faced with three different incarnations of the checkbox: the “legacy” control, the ActiveX control and the most recent “Content Checkbox”.

Let’s look at how to get to each.

Legacy Checkboxes

Legacy Word Form Controls

Legacy Controls

The following code shows how to iterate across legacy checkbox controls. The Parent node of the checkbox holds information such as the name of the actual checkbox instance. To get identifying information we need to traverse up the XML tree and retrieve the FormFieldName element.

using (WordprocessingDocument doc = WordprocessingDocument.Open("c:\\checkbox.docx", true))
{
  foreach (CheckBox cb in doc.MainDocumentPart.Document.Body.Descendants())
  {
    Console.Out.WriteLine(cb.LocalName);

    FormFieldName cbName = cb.Parent.ChildElements.First();
    Console.Out.WriteLine(cbName.Val);

    DefaultCheckBoxFormFieldState defaultState = cb.GetFirstChild();
    Checked state = cb.GetFirstChild();

    Console.Out.WriteLine(defaultState.Val.ToString());

    if (state.Val == null) // In case checkbox is checked the val attribute is null
    {
      Console.Out.WriteLine("CHECKED");
    }
    else
    {
      Console.Out.WriteLine(state.Val.ToString());
    }
  }
}

ActiveX Checkboxes

ActiveX controls present a more problematic situation. The OpenXML spec is not concerned with ActiveX controls, which are Windows-specific program building blocks and OpenXML is focused on interoperability across operating systems or platforms.

The code to find an ActiveX control is a lot more verbose and requires us to “sniff around” to check the class type of any controls found in the document. We can find ActiveX controls using the Control element type:


foreach (Control ctrl in doc.MainDocumentPart.Document.Body.Descendants())
{
   Console.Out.WriteLine(ctrl.Id);
   Console.Out.WriteLine(ctrl.Name);
   Console.Out.WriteLine(ctrl.ShapeId);
}

As I’ve already mentioned, we can’t differentiate between ActiveX controls within the SDK framework itself, meaning we need to check the class ID of any control we find. The class ID for a checkbox is {8BD21D40-EC42-11CE-9E0D-00AA006002F3}.


OpenXmlPart part = doc.MainDocumentPart.GetPartById(ctrl.Id);
OpenXmlReader reader = OpenXmlReader.Create(part.GetStream());
reader.Read();
OpenXmlElement el = reader.LoadCurrentElement();
if(el.GetAttribute("classid", el.NamespaceUri).Value == "{8BD21D40-EC42-11CE-9E0D-00AA006002F3}")
{
  Console.WriteLine("Checkbox found.");
}
reader.Close();

It’s a cumbersome and counter-intuitve approach and would ideally require the developer to replace any of these controls where possible.

Native Content Controls

Default Word Form Controls

Native Controls

As previously mentioned, Office 2010 introduces a native set of controls that are more manageable when working with the OpenXML SDK. These controls are differentiated using the SdtContentCheckBox type.


using (WordprocessingDocument doc = WordprocessingDocument.Open(filename, true))
{
   MainDocumentPart mp = doc.MainDocumentPart;

   foreach(SdtContentCheckBox cb in mp.Document.Body.Descendants())
   {
     if(cb.Checked.Val == "1");
     {
       Console.Out.WriteLine("CHECKED");
     }
   }
}

Form controls are somewhat tricky to work with but hopefully these examples go some way toward clearing up their quirks and traps.

Oct 11 11

Importing Data Into SQL Azure

by Phil

Windows Azure logo

I was working to get an updated set of data from my local development machine copied up to a Windows Azure database today and spent a lot of time trying to figure out why I couldn’t simply copy the data using the SQL Server Import and Export Wizard. Getting a connection between the two databases would always succeed, but as soon as I had picked my source and destination databases, clicking Next would always produce an error:

The stored procedure required to complete this operation could not be found on the server

Sql import and export wizard error

Not overly informative

While Microsoft may have good intentions of eventually supporting data import / export using the more familiar OLEDb provider, as it stands right now, selecting OLEDb will not work as it is not currently supported by SQL Azure. Looking over the MSDN articles is of little help, with suggestions that the wizard is used and links to various unhelpful resources. That said, there is a very good article on the TechNet Social pages.

It turns out that you are still able to use the wizard, but in order to get the data imported, you should use the .Net Framework Data Provider for SqlServer as the Data Source. Populate the values from your connection string into the corresponding fields in the dialog then carry on as normal.

.Net framework data provider

The right provider makes all the difference

Hope this helps!

Aug 18 11

Design Never Stops

by Phil

When was the last time you wrote a new class?

I left my management position at a highly toxic work environment in June this year to start at a highly reputable software shop as a senior support developer. Support is an entirely different discipline to project development: its primary point of involvement is after a piece of software has shipped. Often it involves working with technologies that aren’t necessarily on the bleeding edge and – more often than not – the most common task is one of maintenance and bug-fixing rather than enhancement and evolution.

Typically when developers are in a maintenance role – or even seconded to a maintenance project – they don’t find themselves adding new classes very often at all. They’ll add new methods to existing classes or refactor old methods because the design phase “is over”. I’ve worked in various types of support development roles over the years and it’s an unfortunate fact of life that in most cases once a product has shipped, the design phase is considered to have been completed way back in the early stages of the application’s life cycle. The client has spent the budget they had alloted to the solution, there are other solutions that need to be built or there just isn’t time to continually revisit and refresh the application.

So the maintenance phase becomes less about keeping the solution up-to-date and supportable and more about functional preservation and ad-hoc enhancement. Michael Feathers (author of “Working Effectively with Legacy Code“) asserts that design is a continuous process: never completed. You should always be creating new functions and new classes.

I’m supporting (among other things) a VB6 applicationright now that uses a bit of Classic ASP thrown in for good measure. I am actually enjoying this because I can identify lots of areas where the design can be evolved and improved. The hard part in these situations is convincing the client that it’s a worthwhile investment. More often than not the general response is “Well, it’s working just fine as it is right now so I don’t really see any need to fix something that isn’t broken”.

As an application grows, evolves or ages the fear of making changes and especially deploying those changes into a production environment grows with it. This situation is exacerbated by not having any unit tests but the crux of the problem really comes down to two areas: cost and fear of change.

Fear of change is a valid concern in some respects. If you haven’t updated your application for some time or it’s a business-critical system, you’d be justifiably concerned about introducing any unnecessary risk to your systems. However the flip-side to this argument is that you would also – as a responsible business – want to ensure that the stability, availability and continuity of your applications was as assured as possible and a good way to ensuring this is to release regularly and often. The metaphor of a medical or dental check-up comes to mind where prevention is the best cure. Leaving problems untreated until they’re real hazards is both costly and risky when regular maintenance and selective modernisation would have likely avoided these situations altogether.

Investing the time and relatively low amounts of money in the ongoing design (i.e. paying back your software mortgage) is good business practice and better value for money than leaving your application neglected and unloved until age and technology advancements mean more serious problems arise.

May 25 11

Ninject and Entity Framework Code First: Easy repository patterns

by Phil

When I was still learning the basics of Dependency Injection (DI), Repositories and Object Relational Mappers (ORM), it was a pretty long struggle to get my head around when much of my professional career had revolved around web forms and .Net 2.

It was only through playing around with code in my spare time that things gradually fell into place. I’m not a gifted developer savant like Scott Hanselman, Steve Sanderson or Rob Conery. Stuff doesn’t just “make sense” as soon as I read it. I need to walk through slowly and see stuff working first before trying to understand how it works.

So having gone through many failed or flawed implementations of the repository pattern and almost-but-not-quite getting DI to work correctly (let alone trying to get my head around Inversion of Control), I believe I’ve finally hit upon a clean, simple repository structure that allows good separation of concerns, highly-testable code and makes coding up my MVC3 sites an absolute doddle. It should be noted that I’ve only been able to reach this point by reading, observing examples and borrowing heavily from other cases out there in the world.

Over the next few posts, I’m going to discuss what I’ve found is working great for me, what’s actually happening and why I’ve done it that way. I’ll also go down a few side-routes to discuss other useful tools or tricks that I’ve found increase my productivity or standard of work.

I’m going to use the same example project throughout: an ASP.Net MVC3 application for support ticket management. This project uses the new Razor syntax, a bit of jQuery and some other fun odds & ends.

Getting Started

The first thing I’m obviously going to want to do is create my MVC3 web application. I’ve started up Visual Studio 2010 and create a new ASP.Net MVC 3 Web Application. I’ll select the Internet Application template using Razor as the View Engine. I don’t have any NUnit test projects installed yet, so I’ll leave the “Create a unit test project” box unchecked and manually add this later.

I’m going to want to do some housekeeping later to organise my classes into logical locations, adding new projects to hold them. For starters though, I’ll need to create projects to manage my data transactions and to define my application models.

I created two new projects &ndash Data and Models – to fulfill these tasks, and also created a Tests project for my unit tests (obviously).

I’m pretty keen to get some actual database work kicked off at this point, but first I need to define my entity models. Let’s have a look at what I’ll need to create a basic, workable support ticketing system:

  • Ticket – this object will contain the core information about incidents and change requests
  • Assignment – this object will keep track of who the current owner of a given ticket is
  • Customer – this object contains the details for companies and contacts within those companies
  • Solution – this object contains information about how a ticket was resolved and its root cause

There are a few supplementary classes that I’ll need to support each of these classes, such as call categories, priorities, states, etc. Check the source code example that I’ve included at the end of this post to see the full data model.

In order to keep everything in its logical place, I’m going to move the automatically-generated AccountModels from the actual web application project to my new Models project. That means I need to add a couple of references to the project:

  • System.Web
  • System.Web.ApplicationServices
  • System.Configuration

I can then delete my entire MVC web app Models folder. It might feel a little weird not seeing your Models in the same project but don’t panic – our namespaces are still all the same, we’re just separating things out to be a bit more organised. Our web app will never even know the difference.

My fundamental data entity is the Ticket. Here’s the code for it:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.ComponentModel.DataAnnotations;

namespace Opendesk.Models
{
    public partial class Ticket
    {
        public int TicketId { get; set; }
        public string ReferenceId { get; set; }
        public DateTime Logged { get; set; }
        public string LoggedBy { get; set; }
        [MaxLength]
        public string Description { get; set; }
        public int ContactId { get; set; }

        public virtual Category Category { get; set; }
        public virtual Priority Priority { get; set; }
        public virtual Status Status { get; set; }
        public virtual Source Source { get; set; }
        public virtual CallType Type { get; set; }

        public virtual Assignment Assignment { get; set; }

        public virtual ICollection Journals { get; set; }
        public virtual ICollection Attachments { get; set; }
    }

    public partial class Ticket
    {
        public Ticket()
        {
            this.Logged = DateTime.Today;
        }
    }
}

As I mention above, have a look at the source code at the end of this post to see the full set of models.

Creating the Repository

The repository object is where all the excitement happens. This project is the key component for our Entity Framework data handling and where we define the behaviours for our code-first approach to data modelling and database management.

There are three key parts to this project:

  • Our base repository
  • The database mappings
  • The data context (connection) handling

The base repository is only responsible for the basic CRUD operations. I want to define an interface that sets out a contract for Add, Delete, Update methods as well as methods for getting a single instance and all instances. Here’s my code:

using System.Collections.Generic;

namespace Opendesk.Data.Interfaces
{
    /// <summary>
    /// Defines the methods that are required for a Repository
    /// </summary>
    /// <typeparam name="T">Any type of class available in the <see cref="OpendeskDb.Models"/> namespace.</see>
    public interface IRepository<T> where T : class
    {
        void Add(T Entity);
        void Update(T Entity);
        void Delete(T Entity);
        T GetById(int Id);
        IEnumerable<T> All();
    }
}

While I’m on the subject of interfaces, I’ll create one that defines the methods for getting the actual entity framework Db Context. I’m going to call my DbContext instance “OpendeskDb”. Here’s the code:

using System.Data.Entity;

namespace Opendesk.Data.Interfaces
{
    ///

    /// Defines the methods that are required for a DatabaseFactory instance.
    /// 

    public interface IDatabaseFactory
    {
        ///

        /// Returns a concrete instance of the OpendeskDb data context
        /// 

        /// DbContext (OpendeskDb)
        OpendeskDb Get();
    }
}

I need a reference to the Entity Framework 4 assembly here (and will do in other projects as well), so now seems like a good time to grab it. I use Nuget to download / update my external code dependencies like ELMaH, Entity Framework and the like. If you aren’t using Nuget, may I suggest you consider saving yourself some time by reading Scott Hanselman’s NuGet Action Plan. You can get more information on the latest (at time of writing) release of the Entity Framework on Scott Gu’s blog.

So far, so good. Next, I need to set up mappings for my data models. This is where things get a little more detailed. This class is responsible for querying and working the actual entity data in object form. It returns all entities as type DbSet<T> collections or instances. Here’s the (lengthy) code:

using System.Data.Entity;
using Opendesk.Models;
using System.Collections.Generic;

namespace Opendesk.Data
{
    ///

    /// Provides facilities for querying and working with entity data as objects.
    /// 

    /// 
    /// Note that a section of code has been commented out in this class that allows
    /// dynamic dropping and recreation of the data model (i.e. the database) but does
    /// not currently seed the newly-created database with new data.
    /// 
    public class OpendeskDb : DbContext
    {
        private IDbSet tickets;

        ///

        /// Returns a DbSet of s, this allows CRUD
        /// operations to be performed for the given entity in the context.
        /// 

        public IDbSet Tickets
        {
            get { return tickets ?? (tickets = DbSet()); }
        }

        public IDbSet Assignments { get; set; }
        public IDbSet Solutions { get; set; }
        public IDbSet Journals { get; set; }
        public IDbSet Customers { get; set; }
        public IDbSet Contacts { get; set; }
        public IDbSet CallTypes { get; set; }
        public IDbSet Sources { get; set; }
        public IDbSet Attachments { get; set; }
        public IDbSet Priorities { get; set; }
        public IDbSet Statuses { get; set; }
        public IDbSet Categories { get; set; }

        ///

        /// Returns a DbSet for the specified type, this allows CRUD operations to be performed for
        /// the given entity in the context.
        /// 

        public virtual IDbSet DbSet() where T : class
        {
            return Set();
        }

        ///

        /// Constructs a new context instance using the given string as the name or connection string
        /// for the database to which a connection will be made. For more information on how this is
        /// used to create a connection, see the remarks section for .
        /// 

        ///

        public OpendeskDb(string connectionString)
            : base(connectionString)
        {
            tickets = Tickets;
        }

        ///

        /// Saves all changes made in this context to the underlying database.
        /// 

        public virtual void Commit()
        {
            base.SaveChanges();
        }

        //protected override void OnModelCreating(DbModelBuilder modelBuilder)
        //{
        //    base.OnModelCreating(modelBuilder);
        //    Database.SetInitializer(new Initialiser());
        //}

    }

That’s quite a bit to take in, but I’ve also added XML comments to try and describe what’s going on. Notice also that the OnModelCreating method has been commented out at this stage. We’ll add this back in a later post.

Ok, so our core framework is sorted out, but we still don’t have any actual classes that implement the interfaces we’ve created. First, we need the DatabaseFactory class:

using System;
using System.Configuration;
using Opendesk.Data.Interfaces;

namespace Opendesk.Data
{
    ///

    /// The DatabaseFactory class is the proxy manager for all connections and transactions with the database.
    /// 

    /// 
    /// This class is responsible for the actual connection to the database. It holds the reference to the actual
    /// database connection string from Web.Config.
    /// 
    public class DatabaseFactory : Disposable, IDatabaseFactory
    {
        private OpendeskDb db;

        ///

        /// Returns the active database object context instance or creates a new instance if one doesn't exist
        /// already.
        /// 

        /// A OpendeskDb object (which inherits from DbContext).
        public OpendeskDb Get()
        {
            return db ?? (db = new OpendeskDb(ConfigurationManager.ConnectionStrings["OpendeskDb"].ConnectionString));
        }

        protected override void DisposeCore()
        {
            if (db != null)
                db.Dispose();
        }
    }

    ///

    /// The Disposable class is a managed disposable resource that can be explicitly called within other classes.
    /// 

    public class Disposable : IDisposable
    {
        private bool isDisposed;

        ~Disposable()
        {
            Dispose(false);
        }

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }
        private void Dispose(bool disposing)
        {
            if (!isDisposed && disposing)
            {
                DisposeCore();
            }

            isDisposed = true;
        }

        protected virtual void DisposeCore()
        {
        }
    }

}

Next we need to create the base repository class that our other classes will inherit from:

using System.Collections.Generic;
using System.Data.Entity;
using System.Linq;
using Opendesk.Data.Interfaces;

namespace Opendesk.Data
{
    ///

    /// Encapsulates the available database interactions on behalf of a Repository type.
    /// 

    /// Any type of class available in the Opendesk.Models namespace.
    /// 
    /// 
    /// The RepositoryBase type exposes the following members.
    ///

    /// RepositoryBase
    /// Add
    /// Delete
    /// GetById
    /// All
    /// Update
    /// 
    /// 
    public abstract class RepositoryBase : IRepository where T : class
    {
        private OpendeskDb db;
        private readonly IDbSet dbset;

        ///

        /// Holds a reference to the DatabaseFactory class used to manage connections to the database.
        /// 

        protected IDatabaseFactory DatabaseFactory { get; private set; }
        ///

        /// Contains a reference to the  instance used by the repository.
        /// 

        protected OpendeskDb OpendeskDb { get { return db ?? (db = DatabaseFactory.Get()); } }

        ///

        /// Initialises a new instance of the RepositoryBase class.
        /// 

        ///
A valid DatabaseFactory  object.
        public RepositoryBase(IDatabaseFactory DbFactory)
        {
            DatabaseFactory = DbFactory;
            dbset = OpendeskDb.Set();
        }

        ///

        /// Adds a new entity instance to the database on behalf of the parent type.
        /// 

        ///
Any valid database object
        public virtual void Add(T Entity)
        {
            dbset.Add(Entity);
            db.Commit();
        }

        ///

        /// Deletes an existing instance of an entity from the database on behalf of the parent type.
        /// 

        ///
Any valid database object
        public virtual void Delete(T Entity)
        {
            dbset.Remove(Entity);
            db.Commit();
        }

        ///

        /// Returns a specific instance of an entity from the database on behalf of the parent type.
        /// 

        ///
The integer value of the entity's primary key
        /// A database object (of type T)
        public virtual T GetById(int Id)
        {
            return dbset.Find(Id);
        }

        ///

        /// Returns an IEnumerable collection of all objects found in the database of type T
        /// 

        /// A collection of type IEnumerable
        public virtual IEnumerable All()
        {
            return dbset.ToList();
        }

        ///

        /// Updates an existing entity instance in the database on behalf of the parent type.
        /// 

        ///
Any valid database object
        public void Update(T Entity)
        {
            dbset.Attach(Entity);
            db.Entry(Entity).State = System.Data.EntityState.Modified;
            db.Commit();
        }
    }
}

That’s quite a lot of code, so let’s take stock of what we’ve got so far:

  • We defined interfaces and classes that describe our Entity Framework database management (IDatabaseFactory, DatabaseFactory and OpendeskDb); and
  • We defined interfaces that describe our core database interactions (IRepository and RepositoryBase)

The last main item to tick of our Data project list is the actual Ticket Repository. This class is responsible for retrieving specific subsets of data over and above the basic “get all”, “update”, “delete” and “save” functions. Again, we’ll want to define an interface so that we can unit test our concrete class and set up our Dependency Injection:

using System.Collections.Generic;
using Opendesk.Models;

namespace Opendesk.Data.Interfaces
{
    ///

    /// Defines the methods that are required for a TicketRepository instance.
    /// 

    public interface ITicketRepository : IRepository
    {
        IEnumerable SearchDescription(string Description);
        IEnumerable FilterByStatus(Status Status);
        IEnumerable FilterBySource(Source Source);
        IEnumerable FilterByCategory(Category Category);
        IEnumerable FilterByType(CallType Type);
        IEnumerable GetAttachmentsForTicket(int TicketId);

        IEnumerable GetTicketSources();
        IEnumerable GetTicketCategories();
        IEnumerable GetTicketTypes();
        IEnumerable GetTicketStatuses();
        IEnumerable GetTicketPriorities();
    }
}

We’ve defined quite a few methods here that will return ticket data according to various criteria as well as defining some methods that will primarily be needed to populate drop-down lists on our views. So let’s draw up our actual class:

using System.Collections.Generic;
using System.Linq;
using Opendesk.Data.Interfaces;
using Opendesk.Models;

namespace Opendesk.Data
{
    public class TicketRepository : RepositoryBase, ITicketRepository
    {
        public TicketRepository(IDatabaseFactory DbFactory)
            : base(DbFactory)
        { }

        public IEnumerable SearchDescription(string Description)
        {
            return OpendeskDb.Tickets.Where(t => t.Description.Contains(Description));
        }

        public IEnumerable FilterByStatus(Status Status)
        {
            return OpendeskDb.Tickets.Where(t => t.Status == Status);
        }

        public IEnumerable FilterBySource(Source Source)
        {
            return OpendeskDb.Tickets.Where(t => t.Source == Source);
        }

        public IEnumerable FilterByCategory(Category Category)
        {
            return OpendeskDb.Tickets.Where(t => t.Category == Category);
        }

        public IEnumerable FilterByType(CallType Type)
        {
            return OpendeskDb.Tickets.Where(t => t.Type == Type);
        }

        public IEnumerable GetAttachmentsForTicket(int TicketId)
        {
            return base.GetById(TicketId).Attachments.ToList();
        }

        public IEnumerable GetTicketSources()
        {
            return OpendeskDb.Sources;
        }

        public IEnumerable GetTicketCategories()
        {
            return OpendeskDb.Categories;
        }

        public IEnumerable GetTicketTypes()
        {
            return OpendeskDb.CallTypes;
        }

        public IEnumerable GetTicketStatuses()
        {
            return OpendeskDb.Statuses;
        }

        public IEnumerable GetTicketPriorities()
        {
            return OpendeskDb.Priorities;
        }
    }
}

So that wraps up our data access work. All that remains to wire everything up is to take advantage of Ninject to manage our Dependency Injection and Inversion of Control.

Using Ninject for Dependency Injection

We can now return to our web project and set up our DI wiring within our Global.asax file.

When you start a garden-variety MVC application, the global.asax file looks (more or less) like this:

    public class MvcApplication : System.Web.HttpApplication
    {
        public static void RegisterRoutes(RouteCollection routes)
        {
            routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

            routes.MapRoute(
                "Default",                                              // Route name
                "{controller}/{action}/{id}",                           // URL with parameters
                new { controller = "Home", action = "Index", id = "" }  // Parameter defaults
            );

        }

        protected void Application_Start()
        {
            RegisterRoutes(RouteTable.Routes);
        }
    }

Notice that the class MvcApplication inherits System.Web.HttpApplication. Ninject has its own HttpApplication that is controlled by a native type, Ninject.IKernel.

We’ll use Nuget again to pull down Ninject and modify our Global.asax to take leverage its capabilities.

Ninject with nuget

Using Nuget to add Ninject

Next, we’ll modify our MvcApplication class to inherit from NinjectHttpApplication:

using System;
using System.Web.Mvc;
using System.Web.Routing;
using Ninject;
using Ninject.Modules;
using Ninject.Web.Mvc;
using Opendesk.Data;
using Opendesk.Data.Interfaces;

namespace Opendesk
{
    public class MvcApplication : NinjectHttpApplication
    {

        public MvcApplication()
        {

        }

        ///

        /// Appends any global filters to all Controller classes in the current MVC application context.
        /// 

        ///
The GlobalFilterCollection defined in the application GlobalFilters object.
        public static void RegisterGlobalFilters(GlobalFilterCollection filters)
        {
            filters.Add(new HandleErrorAttribute());
        }

        public static void RegisterRoutes(RouteCollection routes)
        {
            routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

            routes.MapRoute(
                "Default", // Route name
                "{controller}/{action}/{id}", // URL with parameters
                new { controller = "Home", action = "Index", id = UrlParameter.Optional } // Parameter defaults
            );

            routes.MapRoute(
                "Ticket",
                "{controller}/{id}/{action}",
                new { controller = "Ticket", action = "Index", id = UrlParameter.Optional }
                );

        }

        protected override void OnApplicationStarted()
        {
            base.OnApplicationStarted();

            ModelMetadataProviders.Current = new DataAnnotationsModelMetadataProvider();

            AreaRegistration.RegisterAllAreas();
            RegisterGlobalFilters(GlobalFilters.Filters);
            RegisterRoutes(RouteTable.Routes);

            //ViewEngines.Engines.Clear();
            //ViewEngines.Engines.Add(new MobileCapableWebFormViewEngine());

            GlobalFilters.Filters.Add(new HandleErrorAttribute());
        }

        protected override IKernel CreateKernel()
        {
            var modules = new INinjectModule[]
            {
                new ServiceModule()
            };

            return new StandardKernel(modules);
        }
    }

    internal class ServiceModule : NinjectModule
    {
        public override void Load()
        {
            Bind().To();
            Bind().To();
            Bind().To();
        }
    }

}

There are a couple of things to note here. First is that instead of the default method, Application_Start(), we now override OnApplicationStarted(). We also override Ninject’s CreateKernel factory method and define a new NinjectModule that dictates what our dependency bindings should be (you can see these in the internal class, ServiceModule).

That’s pretty straight-forward. Let’s see how we change our code to uncouple our controller from any strongly-typed data management dependencies.

Whereas previously we might have defined a concrete instance of a DbContext or data repository, that would have meant we were committed to having a direct line with our database that is always on. This severely limits our options for unit testing but also (perhaps more crucially) means our controller classes (or other classes) end up doing far more work than they should. So rather than marry up our controller classes to concrete dependencies, our controller constructor looks like this:

    public class TicketsController : Controller
    {
        private readonly ITicketRepository repository;

        public TicketsController(ITicketRepository Repository)
        {
            repository = Repository;
        }

All this is doing is telling our TicketsController class that it needs to use a class that inherits an ITicketRepository. Ninject will take care of that for us.

In Summary

So what have we accomplished in all of this? Well, we’ve got a good, flexible, very loosely-coupled repository that uses Entity Framework Code First to supply us with our database, entity models and data handling. We’ve used Nuget to download and install our required third-party components; specifically Entity Framework and Ninject. We’ve then applied Ninject to our web application to create a highly-testable, flexible application.

In my next post, I’ll look at creating unit tests, creating some views using the MVC3 Razor syntax and I’ll also look at using Doxygen for on-the-go application documentation.

The source code for this post can be downloaded here: Opendesk source code

Apr 17 11

Error-free Key Performance Indicators

by Phil
In pursuit of perfection

There has been a certain amount of “empassioned negotiation” at The Office™ regarding what exactly constitutes “error-free” deliverables and what the implications of the words “error-free” are on key performance indicators in an employment context.

It is entirely reasonable that a company should expect you – whether you are employed as a full-time developer or as a contractor – to write code that is robust, clean, best-practice and easy to maintain. It would ordinarily be a matter of personal pride that a developer would produce code that exemplifies the best standards in software development. But what happens when, in the process of committing these expectations to paper, your employer includes the phrase “error-free” as part of your performance outputs or criteria? What are the implications?

“Error-free” is a highly subjective term and one that is open to a lot of interpretation by either party. While the employer might expect that the software you deliver should work flawlessly in the production environment, the developer might – very naturally – approach the condition from a much more technical and literal frame of view.

Let’s look at the two opposing points of view and discuss the issues surrounding each. First – your employer.

The employer’s expectation around error-free is that the code should, naturally enough, be fairly watertight. It should be of a standard that, once deployed into a production environment, it runs seamlessly and flawlessly. Any unexpected situations should be handled elegantly and the user experience should be completely fluid.

When the company refers to “error-free”, its actual expectations are primarily about deliverables: is the availability and continuity of the system upheld by the code you’ve released; does the code meet company standards for maintainability and general best-practice; and of course is it delivered with minimal impact to deadlines and cost?

Assuming these criteria are met, in most ordinary cases, you will be deemed to have fulfilled your role. So then, what of the developer’s interpretation?

The developer by nature will approach the problem from quite a different mindset. To someone who deals with logical absolutes and literal interpretations, error-free means exactly that: eliminate all possible errors from your code. The developer will go to great lengths to ensure that every possible eventuality – no matter how unlikely – is considered and handled. This degree of exception handling would add a significant amount of time and effort to any development project. Meeting a client’s delivery dates and functional requirements becomes that much harder and more stressful as the developer tries to meet the conflicting expectations of client and employer by providing a completely fault-tolerant and specification-perfect deliverable on time and on budget. A goal highly unlikely to be achieved.

So how much of a problem is this in reality? How literally should the developer realistically interpret the phrase “error-free”? Let’s assume that performance reviews contain some clause that reflects an expectation of error-free. Now assume that a new web application is deployed to a production environment. All goes well until the database server unexpectedly fails for some unrelated reason. The web application cannot make a connection and the site fails; redirecting the user to the Error page. Now the error is not with the code, but none-the-less, an error has definitely occurred. While any reasonable employer should agree that the fault does not necessarily lie with the developer, the developer has still failed to provide an error-free deliverable.

Another example might be if bad data is imported into the database that is depended on by the web application. The bad data then causes some pages to fail catastrophically and reports to display incorrect information. Again – who is to blame here?

Let’s look at another possibility. Assume that the web application runs perfectly all the time. The user is never aware of any faults and, despite perhaps being a little sluggish in some aspects, the overall experience is very favourable. However, upon opening the solution up and viewing the code, a new developer to the team discovers classes full of declarations such as this:

        public DataTable getDataTable(string product, string period)
        {
            string myDate = period;
            ...

The code makes no use of any standard object-oriented principles – there are no interfaces, dependencies are tightly-coupled and methods doing the same exact task are repeated in several different places. Try / catch blocks wrap around large swathes of code and do nothing with the exception:

            try
            {
                // Carry out a whole bunch of operations
            }
            catch (Exception e)
            { }

Now how error-free is our application? Granted there are no visible errors from the user’s perspective, but imagine the veritable minefield of problems our developer faces. So if the code works, is it really error-free? Who should be the judge? If the client is satisfied that everything is fine and the employer is blissfully unaware of the litany of bombshells then would the developer be judged to have fulfilled the obligations expected of their role? In reality, errors have been growing organically in the code base for who knows how long but never brought to the attention of non-technical management who can see and understand only the functional behaviour and visual design and how these meet the specifications.

When all is said and done, the inclusion of an expectation of “error-free” code is either idealistic folly on the part of the employer and in effect an unattainable and unrealistic goal to set, or it is a millstone for the developer’s neck that brings little-to-no value for the client given the time and effort invested in order to achieve it. No code base is ever truly error-free. It is impossible to accommodate every outcome and foresee every event and expecting the developer to do this is unreasonable; pointless at best.

Jan 5 11

Simple Console Debugging

by Phil

One of the most common tasks I carry out when building a new web application is to check my JavaScript performance and debugging in the browser console.  The console window is one of the most indispensible tools in the developer’s kit and it’s very commonplace to see smatterings of “console.log(…);” throughout the app’s code.

What you may not know is that there are a lot of additional functions available than the omnipresent console.log command.

Inspecting Javascript Variables

The most convenient use of firebug for me is of seeing the JS variable values as I develop a complex client-side application. As I discussed above, you don’t want to see alert boxes again and again, IE developer tools, do that for you in two ways.

You can use console to see any output messages. To differentiate between these messages, different console.log APIs are available:

  • console.log
  • console.error
  • console.warn
  • console.info
  • console.assert

My preferred command in Firebug used to be console.debug but I left using that long ago because IE Developer Tools support was buggy and usually broke my code. Secondly firebug formats logged objects nicely so that they are written not as static text, but as interactive hyperlinks that can be clicked to inspect the object in Firebug’s HTML, CSS, Script, or DOM tabs.

I have now moved toward using a more common set of debugging commands:

var myvar = 0;
console.log(myvar);
console.error(myvar);
console.warn(myvar);
console.info(myvar);

These commands are great in that the provide additional formatting to add visual cues to your debugging code as needed.

Browser Handling

Depending on the type of browser you’re using (oh, who am I kidding?  If you’re using old versions of IE), you may get JavaScript exceptions being thrown, saying that console.log is undefined.  Browsers that don’t have a native set of debugging tools will throw this error because, obviously, they don’t have a JavaScript console.  You can get around this problem with the following code at the beginning of any globally-used script files.

if (typeof console == "undefined") {
  window.console = {
    log: function () {
      // do nothing
    }
  };
  console.warn = console.debug = console.log;
} 

Ben Alman has a nice little debugging script that can be included as a simple wrapper for console logging.

Dec 13 10

Excel Software Estimate Template

by Phil

You know, it’s actually quite a tricky job to find a decent, simple software estimate template that I can use to present my estimates to business stakeholders.

In the absence of finding anything free, simple-t0-use and well-structured, I’ve opted to author my own Excel 2010 template that breaks a software project into five main sections:

  • Testing
  • Pages (or forms if you’re a winforms dev)
  • User Experience and CSS
  • Code
  • Documentation

Of course, it’s possible for you to add further pages or chop and change things to your own tastes.  So here I give you version 1 of my software development estimate template.

Software Estimate

This will work on Excel 2007 and 2010 and should also work on Excel 2003 if you have the Office Compatibility Pack installed.  Let me know if it works for you or if you think there are glaring omissions or oversights.

Sep 30 10

Missing Icons for Web Deployment on Win2k8

by Phil

Scott Gu recently posted a really helpful article on how to take advantage of the Web Deploy option in Visual Studio, including how to configure Windows Server 2008 and Windows Web Server 2008.

In it, he outlined how to configure Management Service Delegation on IIS7, but what if, like me, that icon isn’t even available?  Gone.  Missing.  Disappeared.

The solution is actually quite straight-forward: the most likely cause is that the features required are not installed when using the Web Platform Installer.  Download the Deployment Package directly – rather than via the Web Platform Installer – (or get the x86 or x64 versions from here) and choose the option to change your installation.

Installing additional features for web deploymentYou should see that your IIS7 Deployment Handler and the child node Management Service Delegation are both currently uninstalled.  Install these options, reload the IIS manager and you should see the icon appear as expected.