Friday, 21 December 2012

MSDN and MS Fail - Developing Store Apps? You're on your own

One of the reasons I often say to people that the Microsoft stack is the best to work on is the quality of the documentation.  For most APIs you get decent documentation as to what the method/class does, what kind of parameters to pass and, most crucially, why the operation will throw an exception (and what to do about it).  Across most of the BCL, too, you get lots of helpful exceptions when things go wrong - so you can address any issues and try again.

For this post I'm going to be working primarily with a random pair of examples (and it's by no means the best, especially throughout the Windows Store MSDN you'll see stuff like I'm about to show all over the place):

From the BCL MSDN: The AesManaged class (from the core BCL) and the documentation for it's CreateEncryptor(byte[], byte[]) method.

With those two opened in tabs, let's now open the Windows Store MSDN topics DataProtectionProvider and the DataProtectionProvider.ProtectStreamAsync(IInputStream, IOutputStream) method.

Font Size and Page Width

The first thing to note between the two different types of documentation is that whilst both are fixed width, the BCL pages are slightly wider.  'Why is that a problem?'  You ask - just look at the size of that text on the Windows Store pages!  The following image shows what the class name code block looks like in my browser (first) and what it would look like if it used the same font size as the BCL documentation:


It's about 55px difference in width there, for less than 50 characters.  So consider that for the code examples this means a lot more broken lines and wrapping: (screenshots from the DataProtectionProvider topic):



That second one is almost laughable - actually (and I'll be getting to this in a minute).

Now of course MSDN also does have a general design rule that code samples shouldn't wrap.  The fact that these do means someone in the QA team there either should be given a stern talking to - or the sack.

But of course this gripe doesn't just stand for code samples - the text in general is all larger.  The difference is that the BCL stylesheet uses 12px for the font size, and the new one uses 0.87em.  Okay so using ems is better; but you should try to get it right.  Sure I can change my font size to 'smaller' in IE and get the desired effect - but I just shouldn't have to.  At least start with an em value that is equivalent, for most people, to what the original MSDN uses - I believe 0.75em is a good one for most cases.

The overall effect of the larger text, in my opinion is to make it all look unprofessional and makes me feel like I'm working with a toy framework.

The page size has been reduced to 985px vs 1220px.  So on my 1680x1050 desktop (I'm sure a minimum for most developers) I end up with a preponderance of whitespace in the gutters of my browser.  Okay so it stops people with 1024px screens having to pan - that's great.  But for people with bigger screens it just looks crap.  This is what the min-width CSS rule is for, people!  We obviously don't want our lines too long (and re-pagination of code samples would be a step too far and too-long lines of text get hard to read), so a liquid layout that grows to a max of 1220px would seem to make the most sense here.

On to the content - the code samples

Why oh why oh why are the code samples for the DataProtectionProvider, which are written in C#, not highlighted, and why oh why does it say in the language tag tab on the top: 'None'.  I assume the two are related, but either way it's immensely difficult to read those walls of code.  In fact, because they're not coloured they're actually just walls of fixed-width text, which is horrible.  It's also especially difficult when we have seriously ugly code wrapping as shown in the screenshots above.

Clearly, whoever wrote the samples was expecting the available width to be the same as it is for the 'proper' MSDN - but nobody in the design department told him or her that it isn't.  Left-hand, right-hand…

Also - where are the samples for other developers?  Now in some other classes and APIs you will find multiple-language examples, but you'll also find a lot like this, where only one language is presented.  Not acceptable.

In nearly all of the .Net BCL documentation you'll get at least C# and VB; and sometimes JScript.

The runtime experience - and the actual documentation

This is where the real problems start.

First - and we're looking at the ProtectStreamAsync method documentation here - a reference is made back to which constructor you should use.  That constructor takes a magic string.  Now try and find a list of all the magic strings that are acceptable for that constructor.  Go on - I dare you - you simply won't.

In fact, if you run the Win RT Crypto API sample with either of the WEBCREDENTIALS versions - which do, admittedly appear to be placeholder strings for you to fill in - the sample crashes!

And this is where we get to what is actually my biggest gripe - and this gripe is not just with the documentation, but the whole WinRT .Net layer too:  Exceptions.

Knowing what exceptions are to be expected if something goes wrong is a central tenet to .Net framework programming.  This method's documentation contains absolutely nothing about the exceptions that will be raised if something goes wrong.  The implication being that nothing can go wrong if no exceptions are listed.  Not so!  As my previous paragraph states - the sample crashes with this exception:


To find out what that actually means I had to google 'dataprotectionprovider hresult 80090034' (but notice google changes the dataprotectionprovider name to 'data protection provider' - because if you actually search for the original phrase you get no results).  The top result here is COM Error Codes which lists 0x80090034 as 'NTE_ENCRYPTION_FAILURE: Encryption Failed'.

Marvellous - thanks for that.

So what should change?

Well the COM error code list is absolutely fine - I have no actual gripe with that; if I'm doing native development then I do need to know what all these error codes are.

But I'm not doing native development, I'm doing .Net development, which uses exceptions to communicate failure.  So that exception should be raised with some meaningful information about what's actually gone wrong (based on the HRESULT that was returned) and/or - MSDN should contain a good list of the exceptions that can occur and why they will occur.  In this case, there's clearly something wrong with the magic string I sent to the DataProtectionProvider constructor.  But since I've got no way of knowing what the acceptable values actually are; and I've got no way of knowing what's actually wrong with the string I did send - the API is basically useless to me.  It really might as well not exist at all.


In fact this whole exceptions thing is something that's pervasive throughout the 'new' MSDN, and I think something that points to a core decision that's been made somewhere.

In Windows Store/RT development, exceptions apparently never happen and if they do, then you should swallow them - don't worry about what they actually are.  I came across a code sample on a topic the other day (I'm sorry but I can't remember what it was) which used a catch(Exception){ } block to swallow every exception and return null from the example method.

There wasn't even a paragraph saying something like 'this is not good practise' and linking off to a helpful topic about handling exceptions effectively (like this very topic taken from MSDN itself!).

I mean: wow.

So what's this framework and it's documentation trying to achieve?

…Because I really have found it to be less than effective in many many scenarios so I can't imagine it's to be helpful.

As it stands, in my opinion, it's to make Windows Store development accessible to the beginner programmer, that's what.  The aim is to get as many people blundering about in the new .Net framework; hiding all the complicated stuff like what could go wrong and why, so that more store apps are developed.

Okay so that's a reasonable goal, but at the same time it's alienating people like me who want to know why stuff goes wrong and what I can do to either fix it or avoid it.  I really wanted to use that DataProtectionProvider, for example, so I could use out-of-the-box encryption tied to the user's roaming identity (I'm assuming that the "LOCAL=User" magic string therefore isn't applicable, but I wouldn't bloody well know would I!?) to store some sensitive data that I also wanted to roam.  But I can't - because I literally have no way of knowing how to do it.

Oh and there's another target audience implicit within this - Java and Objective-C developers from Android and iOS-land.  They might be just starting to learn .Net and C#/VB - so let's not clutter up our documentation with loads of technical details that might frighten them off (I'm not implying that such devs are non-technical by the way - I'm suggesting that the MSDN team is condescending to them) and pretend that all our APIs are really friendly and will always work.

They'll really thank you when shizzle does happen but there's no fizzling clue as to what's gone wrong.  Really, they will.  If I, a professional .Net developer with many years experience across most of the .Net envronment stacks (service, desktop, command line, web forms, mvc) am getting really frustrated with this then I'm sure they're going to feel absolutely abandoned.

And finally - another example - the Geolocator

My last rant - and another indicator of both a documentation fail but also some shortcomings in WinRT itself - is one of the very very core APIs that you'll use in Windows Store Development: the GeoLocator class and it's GetGeopositionAsync overload.  (Note here, for some reason, all the code examples are now magically in JavaScript and not available in anything else!).

Now - for some reason I've yet to fathom.  Geolocation will not work on my desktop in the office.  Any app that tries to use my location will never get it.  Even IE10 will not dish out a location (and yet IE9 running on Windows 7 would happily get one resolved by IP address - figure that one out).  Now, I'm thinking this likely because I'm in an enterprise environment with a corporate proxy/firewall - but really the reason it doesn't work is irrelevant to this discussion.  As a developer I should be able to handle all scenarios effectively and easily - especially when the API and it's documentation implies that all the issues are already dealt with for me.

So, anyway, JobServe Windows Store App I'm developing right now will of course take advantage of geo-location if it's available - so I wrote this very simple piece of code, in an async method:

var loc = new Geolocator();
var t =
    await loc.GetGeopositionAsync(TimeSpan.FromDays(7), TimeSpan.FromSeconds(10));

That means - 'get a location that's at most 7 days old, and return within 10 seconds if you can't'.

On my machine, with whatever geo location issues it has, this call - never returns - not after 10 seconds, not after 5 minutes.  I left it running for an entire lunchtime once - still nothing.

The timeout value expressed here implies that you can control just how long to wait for a location update to be received.  In fact that only seems to work if the Geolocator can be initialised (on my machine it never moves past the 'Initialising' state). 

But surely the documentation will tell me about this - and how to handle it.  No - it doesn't.  It mentions about 7 second timeouts when the object is in 'Connected Standby' - but that doesn't even seem to relate to anything else on the Geolocator object - it's certainly not one of the enum values in the PositionStatus enum.

So what can I do - our app will be used by people in business environments to search for jobs in their lunch breaks (contractors especially) - I'll have no choice but to do a manual wait for the ten seconds and move on manually if the task hasn't completed.  The task can't be killed or terminated, though, because it's an IAsyncOperation and doesn't support CancellationTokens - so I'll just have to leak the task until the app is restarted.

All in all - that's not good - and to be honest I'm embarrassed to have to deploy code to people's machines that will do such a thing.

But I have no choice, because for the first time in a long time, Microsoft have dropped quite a few balls with WinRT and it's .Net interface - and all the time they remain rolling around the floor I'm finding the experience of developing for it a bit like swimming through treacle.

I should say, though, the WPF element is fantastic - it's a really mature environment now - and plugging into that has been a joy.  Documentation, though - again - a bit light (I keep having to go to the full framework's documentation to get decent XAML examples for some of the stuff that isn't new to RT).

Tuesday, 27 November 2012

Shameless plug - Use the new JobServe Web API to search for jobs your way

As my signature states - I work for JobServe in the UK.  Over the past few months I have been working on a new REST API (using the ASP.Net Web API from pre-beta to RTM) and it is now in official public beta.
Now of course, this isn't just so your good selves can start running your own job searches using whatever web-enabled client takes your fancy, but that is one of the cool benefits that has come out of it - and that's why we're calling it a public beta.
At the time of writing, you can use the API to run almost any job search that you can on the website.  As you would expect, you can get the results in XML or JSON (also GET requests with search parameters in the query-string are supported as well).  Note that JSONP is not currently supported - but it's slated for a future release.

Sounds great, how do I get in?

In order to get cracking with this - you need to request an API token.  This is not an automated process but we should be able to get you set up in a day or two at most.  When filling out this form it's important to be honest about what you intend to do with the API.  That way, we'll know what kind of request patterns to look for and are less likely to revoke access at a later date.

…And then what?

Once you've got your API token - just follow the documentation that I've put together on the JobServe Web API mini-site to get started.  You can contact the support team by email from the site if you have a problem - but please don't contact us looking for code on how to make HTTP requests; that's outside our remit!
If you're environment is .Net, and you don't mind sending/receiving XML, we are making available DataContractSerializer-compatible classes that are generated live from the latest API Types schema.  These are available from the Source Code download page.  These code files (C# and VB.Net available) give you compatible types for all the types used by the API.  Note that these classes aren't, however, natively suitable for JSON - some might work with Json.Net, for example, but they're not intended to.
The most likely usage of this API is to extract the Permalink for a desirable job.  This will take you straight to that job on the JobServe website, from which point you can then apply in the browser as normal.  The sharp-eyed among you will notice in the API types list that clearly we have got objects that can manage the application process outside the browser - but at the moment I'm sorry to say that we have no plans to make that bit public - this is chiefly for the security of our users instead of bloody-mindedness.
So, get yourself signed up for an API token and start writing!  Will you setup an azure service to notify you of jobs?  Perhaps you'll write yourself a personalised search app?  If you want to share any code with us to help others (we might put it on the aforementioned source code page), then please do.

Can I use SSL?

If you're concerned about security - SSL endpoints are available for all operations - just use instead of

Any code samples on the way?

I will try and put together a code example for accessing the service in C# 5 with the 'modern' HttpClient using async/await.  If you do have problems making HTTP requests - then I can also heartily recommend StackOverflow (I am known to hang around there a bit, so I might even see your question) as a great source of help.  Provided, that is, you try your best solve your problem first!

Happy coding!  Oh, and happy job hunting!

Wednesday, 19 September 2012

Adding ‘Deny’ functionality to AuthorizeAttribute in Asp.Net Web API

For the web service project I’m working on at the moment I need to be able to treat authorization differently based on the hostname of the URL that requests are made through.

To state more clearly – these web services will have a ‘sandbox’ mode in addition to the real mode, and the mode a request will operate under is determined as part of the controller-selection phase early in the Web API request lifecycle.  So, say that my web services will be hosted on; the sandbox will simply be

Please note – a discussion of how this is implemented is entirely outside the scope of this article; but I’ll just say that I’ve developed an in-house multi-tenancy layer for both MVC 4 and Web API that allows us to define ‘brands’ and, under those, you can then redefine content, controllers, and even the DI container that is used.

These services are going to require caller-level authentication for most operations via SCRAM Authentication (RFC 5802), and as such most controllers or actions will be decorated with the AuthorizeAttribute:

[Authorize(Roles = APICallerIdentity.Authenticated_Role)]

The value being passed to the Roles member there is just a constant I’m using to keep things consistent.

Now here’s the thing – in Sandbox mode you will be able to access many of the operations as a Guest caller (i.e. without needing to undertake authentication) but in the live mode you will not.  You can still run through the SCRAM Authentication process, of course, and once you do you will have an access token that can be used to authenticate future requests so that you are no longer treated as a guest.

To set this up, a filter that runs earlier in the pipeline (registered via the global HttpConfiguration.Filters collection) – which is responsible for authenticating the request – asks the current mode if guest access is enabled.  If it is, and no credentials are found in the request, then the guest user is set to the current thread’s identity with both the Guest_Role and Authenticated_Role roles set.  Thus, all the methods that require authentication normally can still work in sandbox mode.

I then started work on an operation yesterday that I only ever want to be accessible to non-guest authenticated callers – regardless of whether we’re in sandbox or live mode – where was AuthorizeAttribute now!?

The simple fact is – it can’t do that, but it’s ridiculously simple to subclass it and make it so it can – here’s the full listing of AuthorizeExAttribute:

/// <summary>
/// Extends core AuthorizeAttribute to support denying users and roles.
/// An error occurs during authorization if a user or role is found in
/// both allow and deny rules.
/// </summary>
public class AuthorizeExAttribute : AuthorizeAttribute
    private static readonly string[] _emptyArray = new string[0];

    private static string[] SplitString(string value)
        if (value == null)
            return _emptyArray;

        return (from s in value.Split(",".ToCharArray(),
                        select s.Trim()).ToArray();

    //initialised in constructor
    private readonly Lazy<bool> _isValid;

    private string _denyUsers;
    private string[] _denyUsersSplit = _emptyArray;

    /// <summary>
    /// Gets or sets the users that are to be denied access.
    /// Note: if any of these are also present in the base Users property,
    /// then an error occurs during authorization.
    /// </summary>
    public string DenyUsers
            return _denyUsers ?? string.Empty;
            _denyUsers = value;
            _denyUsersSplit = SplitString(value);

    private string _denyRoles;
    private string[] _denyRolesSplit = _emptyArray;
    /// <summary>
    /// Gets or sets the roles that are to be denied access.
    /// Note: if any of these are also present in the base Users property,
    /// then an error occurs during authorization.
    /// However, if a user is allowed access, but has a role that is denied
    /// access, the deny rule wins.
    /// </summary>
    public string DenyRoles
            return _denyRoles ?? string.Empty;
            _denyRoles = value;
            _denyRolesSplit = SplitString(value);

    /// <summary>
    /// Initializes a new instance of the <see cref="AuthorizeExAttribute"/>
    /// class.
    /// </summary>
    public AuthorizeExAttribute()
        _isValid = new Lazy<bool>(() =>
            //have to re-split the base Users and Roles (if this were
            //implemented within the AuthorizeAttribute this would not
            //be necessary)
            var usersSplit = SplitString(Users);
            var rolesSplit = SplitString(Roles);

            return !_denyUsersSplit.Any(u => usersSplit.Contains(u)) &&
                        !_denyRolesSplit.Any(r => rolesSplit.Contains(r));

    protected override bool IsAuthorized(
            System.Web.Http.Controllers.HttpActionContext actionContext)
        if (!_isValid.Value)
            throw new InvalidOperationException(
                "One or more users or roles appear in both deny and allow rules");

        var baseResult = base.IsAuthorized(actionContext);
        if (!baseResult)
            return false;
        //since it returned true we know there is an authenticated user.
        var user = Thread.CurrentPrincipal;
        if (_denyUsersSplit.Length > 0 && _denyUsersSplit.Contains(
                    user.Identity.Name, StringComparer.OrdinalIgnoreCase))
            return false;

        if (_denyRolesSplit.Length > 0 && _denyRolesSplit.Any(user.IsInRole))
            return false;

        return true;

Please note that the coding style here is intended to mirror that of the core AuthorizeAttribute class – you can see what I mean by looking at the current version on codeplex – because, as the comments mention, this code could easily be merged into that (I know I can submit this myself – but I already have one pull request on the go – and still haven’t gotten around to putting the tests in that are needed to get that one included!).

Notice that an error occurs if a user or role is present in both the allow and deny lists.  Notice also that a ‘deny role’ rule takes precedence over an allow user or role rule.  So if ‘Jimmy’ is allowed, but has the denied role ‘BannedUsers’, then Jimmy will have to work on getting himself un-banned…

Similarly if Jimmy has the ‘AllowedUsers’ role, but still has the ‘BannedUsers’ role, then it’s back to trying make friends with us for Jimmy.

With this in place – all I have to do now to allow authenticated users but deny authenticated guest users is to change my use of AuthorizeAttribute to:

[AuthorizeEx(Roles = APICallerIdentity.Authenticated_Role,

And now authorization is denied for the action or controller on which it is applied if the caller is authenticated, but as a guest.

Happy coding!