Ruminations of J.net idle rants and ramblings of a code monkey

.NET Dojo: Silverlight!

.NET Stuff | Community | Events
We’ve got the next .NET Dojos scheduled - Todd Anglin will be presenting Silverlight! There will be a dojo in Austin as well as Houston. Here are the details: Overview: Microsoft officially released Silverlight 2.0 in October 2008 as a platform for .NET developers to build rich internet applications (RIAs) that run cross-browser and cross-platform. Silverlight 2.0 introduces a whole new way of developing .NET applications. It is blurring the lines between what it means to develop for the web and the desktop, enabling .NET developers to rethink how they build and distribute applications. Topics covered include: Silverlight tools, Silverlight controls, Silverlight APIs, data access, and some security. What you will learn: Todd Anglin will guide you through a combination of lecture and hands-on labs where you will learn everything you need to know to get started with Silverlight 2.0. Specifically, you’ll learn how to work with Silverlight in Visual Studio and Expression Blend, how to use Silverlight controls, how to interact with the HTML DOM, how to access and manipulate data, how to use isolated storage, and secure your Silverlight applications. We’ll go from Silverlight 101 to 301 in about 4 hours and you’ll leave with all the essential tools you need to start building real applications with Silverlight today. Prerequisites: To fully participate in this workshop, attendees will need a laptop with the following: ·        Visual Studio 2008 Professional or higher (trial is okay) with Service Pack 1 installed ·        Expression Blend 2.0 or higher (trial is okay) with Service Pack 1 installed ·        Silverlight Tools for Visual Studio 2008 SP1 installed (free) ·        Deep Zoom Composer installed (free) ·        Silverlight Toolkit December 2008 or higher (available on CodePlex) Times, dates and registration links: Austin (Microsoft Office): March 9, 2009 1:00 – 5:00 PM. Register now. Houston (Microsoft Office): March 13, 2009 1:00 PM – 5:00 PM. Register now.

Austin's First .Net Dojo! (Windows Communication Foundation)

.NET Stuff | Community
Ever since I posted about .Net Dojo, I’ve had quite a few requests to have it elsewhere. Well, after seeing Pablo’s Day of TDD and how awesome that was, I got this little notion that it might be cool to do .Net Dojo in Austin. And … also … well, this format was kinda new and I wanted to give it a trial run or two before I started doing more. So, with that said, we’ve got the very next .Net Dojo scheduled for Austin … our first one out there! It’s going to be the same as the last Houston event and, judging from the feedback that I got on the Houston Dojo, it’s gonna rock! Here are the details: When: November 3, 2008  1:00 PM – 5:00 PM Where: Microsoft Austin Office Overview: Windows Communication Foundation (WCF) is Microsoft’s strategic technology for building distributed, connected systems. Initially released with .NET 3.0, it has been significantly enhanced in .NET 3.5 to support the various web programming models including SOAP, POX, WS-*, REST and RSS/Atom syndications.  Topics covered include: SOA basics, WCF architecture, WCF bindings, hosting WCF services, WCF behaviors, WCF security and some design best practices. What you will learn: Stephen Fulcher will guide you through a combination of lecture and hands-on labs where we will learn how to implement the various web programming models with WCF. Specifically, we will cover SOA basics, WCF architecture, WCF bindings, hosting WCF services, WCF behaviors, WCF security and some design best practices. Prerequisites: To fully participate in this workshop, attendees will need an open mind and a laptop with: -  Visual Studio 2008 Professional or higher (Trial is OK) with Service Pack 1 -  Windows PowerShell 1.0 (optional, but recommended) -  Microsoft SQL Server 2005 Standard or Express -  A CD-ROM drive to load additional components and lab files. Now that you have the details, I know that you want to go here and register now. Hope to see y’all there!

Linq Performance - Part I

.NET Stuff | Linq | Performance
Well, it’s been a while since I did my initial review of some simple Linq performance tests. Since then, I’ve done a bit more testing of Linq performance and I’d like to share that. The results are enlightening, to say the least. I did this because I’ve gotten a lot of questions regarding the performance of Linq and, in particular, Linq to Sql – something that is common whenever there is a new data-oriented API. Now, let me also say that performance isn’t the only consideration … there are also considerations of functionality and ease of use, as well as the overall functionality of the API and its applicability to a wide variety of scenarios. I used the same methodology that I detailed in this previous post. Now, all of the tests were against the AdventureWorks sample database’s Person.Contact table with some 20,000 rows. Not the largest table in the world, but it’s also a good deal larger that the much-beloved Northwind database. I also decided to re-run all of the tests a second time on my home PC (rather than my laptop) as the client and one of my test servers as the database server. The specs are as follows: Client DB Server AMD Athlon 64 X2 4400+ AMD Athlon 64 X2 4200+ 4 GB RAM 2 GB RAM Vista SP1 x64 Windows Server 2008 Standard x64 Visual Studio 2008 SP1 Sql Server 2008 x64 So, with that out of the way, let’s discuss the first test. Simple Query This is a simple “SELECT * FROM Person.Contact” query … nothing special or funky. From there, as with all of the tests, I loop through the results and assign them to temporary, local variables. An overview of the tests is below: DataReaderIndex Uses a data reader and access the values using the strongly-typed GetXXX methods (i.e. GetString(int ordinal)). With this set, the ordinal is looked up using GetOrdinal before entering the loop to go over the resultset. This is my preferred method of using a DataReader. int firstName = rdr.GetOrdinal("FirstName"); int lastName = rdr.GetOrdinal("LastName"); while (rdr.Read()) { string fullName = rdr.GetString(firstName) + rdr.GetString(lastName); } rdr.Close(); DataReaderHardCodedIndex This is the same as TestDataReaderIndex with the exception that the ordinal is not looked up before entering the loop to go over the resultset but is hard-coded into the application. while (rdr.Read()) { string fullName = rdr.GetString(0) + rdr.GetString(1); } rdr.Close(); DataReaderNoIndex Again, using a reader, but not using the strongly-typed GetXXX methods. Instead, this is using the indexer property, getting the data using the column name as an object. This is how I see a lot of folks using Data Readers. while (rdr.Read()) { string fullName = (string)rdr["FirstName"] + (string)rdr["LastName"]; } rdr.Close(); LinqAnonType Uses Linq with an anonymous type var contactNames = from c in dc.Contacts select new { c.FirstName, c.LastName }; foreach (var contactName in contactNames) { string fullName = contactName.FirstName + contactName.LastName; } LinqClass_Field Again, uses Linq but this time it’s using a custom type. In this class the values are stored in public fields, rather than variables. IQueryable<AdvWorksName> contactNames = from c in dc.Contacts select new AdvWorksName() {FirstName= c.FirstName, LastName= c.LastName }; foreach (var contactName in contactNames) { string fullName = contactName.FirstName + contactName.LastName; } DataSet This final test uses an untyped dataset. We won’t be doing a variation with a strongly-typed dataset for the select because they are significantly slower than untyped datasets. Also, the remoting format for the dataset is set to binary, which will help improve the performance for the dataset, especially as we get more records. DataSet ds = new DataSet(); ds.RemotingFormat = SerializationFormat.Binary; SqlDataAdapter adp = new SqlDataAdapter(cmd); adp.Fill(ds); foreach (DataRow dr in ds.Tables[0].Rows) { string fullName = dr.Field<String>("FirstName") + dr.Field<String>("LastName"); } cnct.Close(); LinqClass_Prop This uses a custom Linq class with properties for the values. IQueryable<AdvWorksNameProps> contactNames = from c in dc.Persons select new AdvWorksNameProps() { FirstName = c.FirstName, LastName = c.LastName }; foreach (var contactName in contactNames) { string fullName = contactName.FirstName + contactName.LastName; } LinqClass_Ctor This uses the same Linq class as above but initializes the class by calling the constructor rather than binding to the properties. IQueryable<AdvWorksNameProps> contactNames = from c in dc.Persons select new AdvWorksNameProps(c.FirstName, c.LastName); foreach (var contactName in contactNames) { string fullName = contactName.FirstName + contactName.LastName; }                                           If you are wondering why the different “flavors” of Linq … it’s because, when I first started re-running these tests for the blog, I got some strange differences that I hadn’t seen before between (what is now) LinqAnonType and LinqClassField. On examination, I found that these things made a difference and wanted to get a more rounded picture of what we were looking at here … so I added a couple of tests. And the results …       Average LinqClass_Field 277.61 DataReaderIndex 283.43 DataReaderHardCodedIndex 291.17 LinqClass_Prop 310.76 DataSet 323.71 LinqAnonType 329.26 LinqClass_Ctor 370.20 DataReaderNoIndex 401.63 These results are actually quite different from what I saw when I ran the tests on a single machine … which is quite interesting and somewhat surprising to me. Linq still does very well when compared to DataReaders … depending on exactly how you implement the class. I didn’t expect that the version using the constructor would turn out to be the one that had the worst performance … and I’m not really sure what to make of that. I was surprised to see the DataSet do so well … it didn’t on previous tests, but in those cases, I also didn’t change the remoting format to binary; this does have a huge impact on the load performance, especially as the datasets get larger (XML gets pretty expensive when it starts getting big).                                                       I’ve got more tests, but due to the sheer length of this post, I’m going to post them separately.

ASP.NET Async Page Model

.NET Stuff | Performance | Web (and ASP.NET) Stuff
I just did a Code Clinic for the Second Life .NET User’s Group on using the ASP.NET async page model and it occurred to me that it’d be a good idea to do a little blog post about it as well. I’ve noticed that a lot of developers don’t know about this little feature and therefore don’t use it. It doesn’t help that the situations where this technique helps aren’t readily apparent with functional testing on the developer’s workstation or even on a separate test server. It only rears its head if you do load testing … something that few actually do (I won’t go there right now). So, let me get one thing straight from the get-go here: I’m not going to be talking about ASP.NET AJAX. No way, no how. I’m going to be talking about a technique that was in the original release of ASP.NET 2.0 and, of course, it’s still there. There are some big-time differences between the async model and AJAX. First, the async model has nothing at all to do with improving the client experience (at least not directly, though it will tend to). Second, the async model doesn’t have any client-side goo; it’s all server-side code. And finally, there is no magic control that you just drop on your page to make it work … it’s all code that you write in the code-behind page. I do want to make sure that this clear ‘cuz these days when folks see “async” in relation to web pages, they automatically think AJAX. AJAX is really a client-side technique, not server side. It does little to nothing to help your server actually scale … it can, in some cases, actually have a negative impact. This would happen when you make additional round trips with AJAX that you might not normally do without AJAX, placing additional load on the server. Now, I’m not saying that you shouldn’t use AJAX … it’s all goodness … but I just want to clarify that this isn’t AJAX. Now, you can potentially this this for AJAX requests that are being processed asynchronously from the client. Now that we have that out of the way, let me, for a moment, talk about what it is. First, it’s a really excellent way to help your site scale, especially when you have long-running, blocking requests somewhere in the site (and many sites do have at least a couple of these). Pages that take a few seconds or more to load may be good candidates. Processes like making web services calls (for example, to do credit card processing and order placement on an eCommerce site) are excellent candidates as well. Why is this such goodness? It has to do with the way ASP.NET and IIS do page processing. ASP.NET creates a pool of threads to actually do the processing of the pages and there is a finite number of threads that will be added to the pool. These processing threads are created as they are needed … so creating additional threads will incur some overhead and there is, of course, overhead involved with the threads themselves even after creation. Now, when a page is requested, a thread is assigned to the page from the pool and that thread is then tied to processing that page and that page alone … until the page is done executing. Requests that cannot be serviced at the time of the request are then queued for processing as a thread becomes available. So … it then (logically) follows that pages that take a long time and consume a processing thread for extended periods will affect the scalability of the site. More pages will wind up in the queue and will therefore take longer since they are waiting for a free thread to execute the page. Of course, once the execution starts, it’ll have no difference on the performance … it’s all in the waiting for a thread to actually process the page. The end result is that you cannot services as many simultaneous requests and users. The async page model fixes this. What happens is that the long running task is executed in the background. Once the task is kicked off, the thread processing the thread is then free to process additional requests. This results in a smaller queue and less time that a request waits to be serviced. This means more pages can actually be handled at the same time more efficiently … better scalability. You can see some test results of this on Fritz Onion’s blog. It’s pretty impressive. I’ve not done my own scalability testing on one of my test servers here, but I think, shortly, I will. Once I do, I’ll post the results here.                                                                                                                                                                               How do you do this? To get started is actually quite easy, simple in fact. You need to add a page directive to your page. This is required regardless of which method you use (there are two). ASP.NET will then implement IAsyncHttpHandler for you behind the scenes. It looks like this: <%@ Page Language="C#" AutoEventWireup="true" CodeFile="Default.aspx.cs" Inherits="_Default" Async="True" %> Simple enough, right? Let me just add a couple of things that you need to make sure you have in place. You will need to follow the .NET asynchronous pattern for this to work … a Begin method that returns IAsyncResult and an end method that takes this result. It’s typically easiest to do this with API’s that already have this implemented for you (you just return their IAsyncResult object). There’s a ton of them and they cover most of the situations where this technique helps. Now, to actually do this. Like I said, there’s two different ways to use this. The first is pretty easy to wireup and you can add multiple requests (I misstated this during the Code Clinic), but all of the async requests run one at a time, not in parallel. You simply call Page.AddOnPreRenderCompleteAsync and away you go. There are two overloads for this method, as follows: void AddOnPreRenderCompleteAsync(BeginEventHandler b, EndEventHandler e)                         void AddOnPreRenderCompleteAsync(BeginEventHandler b, EndEventHandler e, object state) The handlers look like the following: IAsyncResult BeginAsyncRequest(object sender, EventArgs e, AsyncCallback cb, object state) void EndAsyncRequest(IAsyncResult ar)             The state parameter can be used to pass any additional information/object/etc. that you would like to the begin and the end methods (it’s a member if the IAsyncResult interface), so that can be pretty handy. The code behind for such a page would look like the following: protected void Page_Load(object sender, EventArgs e) { LoadThread.Text = Thread.CurrentThread.ManagedThreadId.ToString(); AddOnPreRenderCompleteAsync(new BeginEventHandler(BeginGetMSDN), new EndEventHandler(EndAsyncOperation)); } public IAsyncResult BeginGetMSDN(object sender, EventArgs e, AsyncCallback cb, object state) { BeginThread.Text = Thread.CurrentThread.ManagedThreadId.ToString(); HttpWebRequest _request = (HttpWebRequest)WebRequest.Create(@"http://msdn.microsoft.com"); return _request.BeginGetResponse(cb, _request); } void EndAsyncOperation(IAsyncResult ar) { EndThread.Text = Thread.CurrentThread.ManagedThreadId.ToString(); string text; HttpWebRequest _request = (HttpWebRequest)ar.AsyncState; using (WebResponse response = _request.EndGetResponse(ar)) { using (StreamReader reader = new StreamReader(response.GetResponseStream())) { text = reader.ReadToEnd(); } } Regex regex = new Regex("href\\s*=\\s*\"([^\"]*)\"", RegexOptions.IgnoreCase); MatchCollection matches = regex.Matches(text); StringBuilder builder = new StringBuilder(1024); foreach (Match match in matches) { builder.Append(match.Groups[1]); builder.Append("<br/>"); } Output.Text = builder.ToString(); } } If you run this (on a page with the proper controls, of course), you will notice that Page_Load and BeginGetMSDN both run on the same thread while EndAsyncOperation runs on a different thread. The other method uses a class called PageAsyncTask to register an async task with the page. Now, with this one, you can actually execute multiple tasks in parallel so, in some cases, this may actually improve the performance of an individual page. You have two constructors for this class:     public PageAsyncTask( BeginEventHandler beginHandler, EndEventHandler endHandler, EndEventHandler timeoutHandler, Object state) and public PageAsyncTask( BeginEventHandler beginHandler, EndEventHandler endHandler, EndEventHandler timeoutHandler, Object state, bool executeInParallel){}   The only difference between the two is that one little argument … ExecuteInParallel. The default for this is false, so if you want your tasks to execute in parallel, you need to use the second constructor. The delegates have identical signatures to the delegates for AddOnPreRenderComplete. The new handler timeoutHandler, is called when the operations times out and has the same signature to the end handler. So … it’s actually trivial to switch between the two (I did it to the sample listing above in about a minute.) I, personally, like this method better for two reasons. One, the cleaner handling of the timeout. That’s all goodness to me. Second, the option to have them execute in parallel. The same page as above, now using PageAsyncTask looks like to following: public partial class _Default : System.Web.UI.Page { protected void Page_Load(object sender, EventArgs e) { LoadThread.Text = Thread.CurrentThread.ManagedThreadId.ToString(); PageAsyncTask t = new PageAsyncTask( BeginGetMSDN, EndAsyncOperation, AsyncOperationTimeout, false); } public IAsyncResult BeginGetMSDN(object sender, EventArgs e, AsyncCallback cb, object state) { BeginThread.Text = Thread.CurrentThread.ManagedThreadId.ToString(); HttpWebRequest _request = (HttpWebRequest)WebRequest.Create(@"http://msdn.microsoft.com"); return _request.BeginGetResponse(cb, _request); } void EndAsyncOperation(IAsyncResult ar) { EndThread.Text = Thread.CurrentThread.ManagedThreadId.ToString(); string text; HttpWebRequest _request = (HttpWebRequest)ar.AsyncState; using (WebResponse response = _request.EndGetResponse(ar)) { using (StreamReader reader = new StreamReader(response.GetResponseStream())) { text = reader.ReadToEnd(); } } Regex regex = new Regex("href\\s*=\\s*\"([^\"]*)\"", RegexOptions.IgnoreCase); MatchCollection matches = regex.Matches(text); StringBuilder builder = new StringBuilder(1024); foreach (Match match in matches) { builder.Append(match.Groups[1]); builder.Append("<br/>"); } Output.Text = builder.ToString(); } void AsyncOperationTimeout(IAsyncResult ar) { EndThread.Text = Thread.CurrentThread.ManagedThreadId.ToString(); Output.Text = "The data is not currently available. Please try again later." } } Not much difference there. We have 1 additional method for the timeout and the registration is a little different. By the way, you can pass null in for the timeout handler if you don’t care about it. I don’t recommend doing that, personally, but that’s up to you. There you have it … a quick tour through the ASP.NET asynchronous page model. It’s clean, it’s easy, it’s MUCH better than spinning up your own threads and messing with synchronization primitives (this is mucho-bad-mojo, just say NO) and it’s got some pretty significant benefits for scalability.                                                             With that, I’m outta here. Happy coding!

.NET Dojo: ASP.NET MVC

.NET Stuff | Events | Web (and ASP.NET) Stuff
First, what's this .Net Dojo stuff? Well, here it is ... it is a new series of events (I'm hoping monthly) that we will be having monthly to help developers hone and sharpen their .Net Kung Fu. It will be a combination of lecture and hands-on labs ... lecture to explain the concepts (of course) and hands-on to let you get your fingers dirty playing with the technology. Since it is hands-on, you will need to bring a laptop to do the labs and (possibly) install some additional prerequisites before the session (these will be listed, of course). They will be held at the Houston Microsoft office from 1:00 PM to 5:00 PM and cover a wide range of .Net topics. I'm also hoping to be able to get some of the technical leaders here in Houston to deliver some of the sessions ... so that you'll be learning from the best around. Ben Scheirman of Sogeti has volunteered to deliver the very first .Net Dojo on the ASP.NET MVC Framework. (Didn't I tell you that I wanted to get the best around?) Here are the details: Overview: ASP.NET MVC is one of the key new features that will be included in .NET Framework 3.5 Service Pack1. It provides a framework that enables you to easily implement the model-view-controller (MVC) design pattern in your web applications, helping you build loosely-coupled, pluggable components for application design, logic and display. While not a replacement for traditional ASP.NET WebForms, ASP.NET MVC does provide a compelling alternative for developing web applications and better facilitates test driven development (TDD) in web applications. This workshop will dig into what ASP.NET MVC is, what it does, and how it is different from WebForms. What you will learn: Through a combination of lecture and hands-on labs, attendees will learn how to create ASP.NET MVC applications using Visual Studio 2008 and how to work with the key components of an ASP.NET MVC application. Additionally, they will learn how to test their MVC components and applications using TDD principles. Prerequisites: To fully participate in this workshop, attendees will need the following: An open mind A working knowledge of ASP.NET, C#, HTML and CSS. A laptop with: Visual Studio 2008 Professional or higher (Trial is OK) ASP.NET MVC Preview 4 (available as a separate download. A CD-ROM drive to load additional components and lab files. Sign up here ... but hurry ... space is limited!Hope to see you there ...

Cool way to do ASP.NET Caching with Linq

.NET Stuff | Linq | Web (and ASP.NET) Stuff
OK, well, I think it's cool (and since the mind is its own place ...). I've been a big fan of ASP.net's cache API since I found out it way back in the 1.0 beta. It certainly solves something that was problematic in ASP "Classic" in a clean, elegant and darn easy to use way. Unfortunately, not a lot of folks seem to know about it. So I'll start with a little overview of ASP.net caching. As the name implies, it's a cache that sits server side. All of the relevant, .Net-supplied classes are in the System.Web.Caching namespace and the class representing the cache itself is System.Web.Caching.Cache. You can access it from the current HttpContext (which you'll see). The management of the cache is handled completely by ASP.net ... you just have to add objects to it and then read from it. When you add to the cache, you can set options like dependencies, expiration, priority and a delegate to call when the item is removed from the cache. Dependencies are interesting ... they will automatically invalidate (and remove) the cache item based on notification from the dependency. ASP.net 1.x had only 1 cache dependency class (System.Web.Caching.CacheDependency) that allowed you to have a dependency on a file, another cache item, and array of them or another CacheDependency. Framework 2.0 introduced System.Web.Caching.SqlCacheDependency for database dependencies and System.Web.Caching.AggregateCacheDependency for multiple, related dependencies. With the AggregateCacheDependency, if one of the dependencies changes, it item is invalidated and tossed from the cache. Framework 2.0 also (finally) "unsealed" the CacheDependency class, so you could create your own cache dependencies. With expiration, you can have an absolute expiration (specific time) or a sliding expiration (TimeSpan after last access). Priority plays into the clean-up algorithm; the Cache will remove items that haven't expired if the cache taking up too much memory/resources. Items with a lower priority are evicted first. Do yourself a favor and make sure that you keep your cache items reasonable. Your AppDomain will thank you for it. ASP.net also provides page and partial-page caching mechanisms. That, however, is out of our scope here. For the adventurous among that don't know what that is ... So ... the cache ... mmmmm ... yummy ... gooooood. It's golly-gee-gosh-darn useful for items that you need on the site, but don't change often. Those pesky drop-down lookup lists that come from the database are begging to be cached. It takes a load off the database and is a good way to help scalability - at the cost of server memory, of course. (There ain't no free lunch.) Still, I'm a big fan of appropriate caching. So ... what's the technique I mentioned that this post is title after? Well, it's actually quite simple. It allows you to have 1 single common method to add and retrieve items from the cache ... any Linq item, in fact. You don't need to know anything about the cache ... just the type that you want and the DataContext that it comes from. And yes, it's one method to rule them all, suing generics (generics are kewl!) and the Black Voodoo Majick goo. From there, you can either call it directly from a page or (my preferred method) write a one-line method that acts as a wrapper. The returned objects are detached from the DataContext before they are handed back (so the DataContext doesn't need to be kept open all) and returned as a generic list object. The cache items are keyed by the type name of the DataContext and the object/table so that it's actually possible to have the same LinqToSql object come from two different DataContexts and cache both of them. While you can load up the cache on application start up, I don't like doing that ... it really is a killer for the app start time. I like to lazy load on demand. (And I don't wanna hear any comments about the lazy.) Here's the C# code: /// <summary> /// Handles retrieving and populating Linq objects in the ASP.NET cache /// </summary> /// <typeparam name="LinqContext">The DataContext that the object will be retrieved from.</typeparam> /// <typeparam name="LinqObject">The object that will be returned to be cached as a collection.</typeparam> /// <returns>Generic list with the objects</returns> public static List<LinqObject> GetCacheItem<LinqContext, LinqObject>() where LinqObject : class where LinqContext : System.Data.Linq.DataContext, new() { //Build the cache item name. Tied to context and the object. string cacheItemName = typeof(LinqObject).ToString() + "_" + typeof(LinqContext).ToString(); //Check to see if they are in the cache. List<LinqObject> cacheItems = HttpContext.Current.Cache[cacheItemName] as List<LinqObject>; if (cacheItems == null) { //It's not in the cache -or- is the wrong type. //Create a new list. cacheItems = new List<LinqObject>(); //Create the contect in a using{} block to ensure cleanup. using (LinqContext dc = new LinqContext()) { try { //Get the table with the object from the data context. System.Data.Linq.Table<LinqObject> table = dc.GetTable<LinqObject>(); //Add to the generic list. Detaches from the data context. cacheItems.AddRange(table); //Add to the cache. No absolute expirate and a 60 minute sliding expiration HttpContext.Current.Cache.Add(cacheItemName, cacheItems, null, System.Web.Caching.Cache.NoAbsoluteExpiration, TimeSpan.FromMinutes(60), System.Web.Caching.CacheItemPriority.Normal, null); } catch (Exception ex) { //Something bad happened. throw new ApplicationException("Could not retrieve the request cache object", ex); } } } //return ... return cacheItems; } And in VB (see, I am multi-lingual!) ... ''' <summary> ''' Handles retrieving and populating Linq objects in the ASP.NET cache ''' </summary> ''' <typeparam name="LinqContext">The DataContext that the object will be retrieved from.</typeparam> ''' <typeparam name="LinqObject">The object that will be returned to be cached as a collection.</typeparam> ''' <returns>Generic list with the objects</returns> Public Shared Function GetCacheItem(Of LinqContext As {DataContext, New}, LinqObject As Class)() As List(Of LinqObject) Dim cacheItems As List(Of LinqObject) 'Build the cache item name. Tied to context and the object. Dim cacheItemName As String = GetType(LinqObject).ToString() + "_" + GetType(LinqContext).ToString() 'Check to see if they are in the cache. Dim cacheObject As Object = HttpContext.Current.Cache(cacheItemName) 'Check to make sure it's the correct type. If cacheObject.GetType() Is GetType(List(Of LinqObject)) Then cacheItems = CType(HttpContext.Current.Cache(cacheItemName), List(Of LinqObject)) End If If cacheItems Is Nothing Then 'It's not in the cache -or- is the wrong type. 'Create a new list. cacheItems = New List(Of LinqObject)() 'Create the contect in a using block to ensure cleanup. Using dc As LinqContext = New LinqContext() Try 'Get the table with the object from the data context. Dim table As Linq.Table(Of LinqObject) = dc.GetTable(Of LinqObject)() 'Add to the generic list. Detaches from the data context. cacheItems.AddRange(table) 'Add to the cache. No absolute expirate and a 60 minute sliding expiration HttpContext.Current.Cache.Add(cacheItemName, cacheItems, Nothing, _ Cache.NoAbsoluteExpiration, TimeSpan.FromMinutes(60), _ CacheItemPriority.Normal, Nothing) Catch ex As Exception 'Something bad happened. Throw New ApplicationException("Could not retrieve the request cache object", ex) End Try End Using End If 'return ... Return cacheItems End Function   The comments, I think, pretty much say it all. It is a static method (and the class is a static class) because it's not using any private fields (variables). This does help performance a little bit and, really, there is no reason to instantiate a class if it's not using any state. Also, note the generic constraints - these are actually necessary and make sure that we aren't handed something funky that won't work. These constraints are checked and enforced by the compiler. Using this to retrieve cache items is now quite trivial. The next example shows a wrapper function for an item from the AdventureWorks database. I made it a property but it could just as easily be a method. We won't get into choosing one over the other; that gets religious. public static List<StateProvince> StateProvinceList { get { return GetCacheItem<AdvWorksDataContext, StateProvince>(); } } And VB ... Public ReadOnly Property StateProvinceList() As List(Of StateProvince) Get Return GetCacheItem(Of AdvWorksDataContext, StateProvince)() End Get End Property Isn't that simple? Now, if you only have one DataContext type, you can safely code that type into the code instead of taking it as a generic. However, looking at this, you have to admit ... you can use this in any ASP.net project where you are using Linq to handle the cache. I think it's gonna go into my personal shared library of tricks. As I think you can tell, I'm feeling a little snarky. It's Friday afternoon so I have an excuse. BTW ... bonus points to whoever can send me an email naming the lit reference (and finish it!) in this entry. Umm, no it isn't Lord of the Rings.

ADO.NET: What to use when?

.NET Stuff
This comes from a young lady's question last night at the Houston .Net User's Group. She's at HCC, taking C# courses and felt that it really wasn't clear when to use DataReaders vs. DataTables vs. DataSets. So ... I'm going to put a couple of thoughts down here on that topic, in the hope that she'll be reading it. DataReader These are provider specific classes (i.e. SqlDataReader, OracleDataReader, etc.) that give you access to a read-only, forward-only, server-side cursor. It's a firehose cursor. It's not scrollable, it's not updatable and the current record location is tracked by the server, not the client. The classes will be found in the namespaces for the provider (i.e. System.Data.SqlClient) and will implement the System.Data.IDataReader interface. You get a DataReader by calling ExecuteReader on the provider's DataCommand class (i.e. SqlDataCommand). When to use DataReaders provide the most performant way to read through a set of data returned from a query, by far. But they also provide the least additional functionality. In general, you want to use then when you want to do a single pass through a resultset and then don't need it. They are typically perfect for ASP.NET applications when you are doing databinding directly to resultsets where you won't need the data again and are going to toss it out after the page renders. If you are using a SqlDataSource, you can set the DataSourceMode property to DataReader to force the data source to use a reader (the default is DataSet).  Also, if you are using an object structure that you build from query results, you should use a DataReader to populate the object hierarchy. They will not, however, work with databinding in Windows Forms application ... this inherently needs to be scrollable and that's something that a DataReader won't do.One thing to consider ... even if you are doing a single pass, you may not always want to use a DataReader. If you are doing a non-trivial amount of additional processing on each record as it is read, you may want to consider a DataSet because the DataReader will keep a connection open on the database server and consume server resources. Where is that line? Hmmm ... it depends.Make sure that you dispose of your DataReaders (the do implement IDisposable). I will typically nest DataReaders in a Using block to make sure that they get disposed properly. I also open my DataReaders using the CommandBehavior.CloseConnection option. Untyped DataTable/DataSet DataTables and DataSets are not dependant on the ADO.NET provider but are common classes. Both DataSet and DataTable are found in the System.Data namespace and a DataSet is a container for multiple DataTables and adds things like parent/child relationships to multiple DataTables (so we'll focus mainly on DataTables but also discuss where DataSets come into play). DataTables use client-side cursors (like the old ADO Classic cursor location adUseClient) ... the data is retrieved from the server and then the connection is closed, a process that is done by the provider-specific DataAdapter. The DataAdapter opens a DataReader to do the population so, as you can guess, DataTables are slower than DataReaders. They hold no locks and they don't detect updates on the server. Unlike DataReaders, DataTables are updatable and scrollable. When to use First, if you are doing databinding directly to resultsets in WindowsForms applications, you'll need to use DataTables (at least). In web applications, these are very appropriate for data that you cache ... things like lookup lists, for example. Since these resultsets change very infrequently, caching the results can really help with scalability and performance (though be careful ... you probably won't want to cache a 5000 row DataTable). For a one-time, non-scrollable read, it really doesn't make any sense at all to use a DataTable in most circumstances, though (unless, as mentioned above, you are doing a non-trivial amount of processing on the data as it is read). There is a monkey wrench to throw out here ... if you have a high-volume, data-driven site, you may well want to do some testing with DataReaders vs. DataTables as your most important limiting factor may well be server load. In some scenarios, DataTables, because of the relatively quick open/close semantics the adapter's Fill method, can actually scale better though they don't perform as well. That's something that you need to determine by understanding what your scalability requirements are and where your critical bottlenecks are. In most scenarios, however, DataReaders will be a better choice. Just to muddy it up a hair, you can also get a DataReader from a DataTable (by calling CreateDataReader()), but this isn't the same as the DataReader mentioned above. It's not a server-side cursor. It is, however, forward-only and read-only and can enumerate through the rows in a DataTable faster than looping over the Rows collection in a For Each loop. DataTables allow you to add client-side columns to tables (which can be handy at times); these client-side columns can be completely custom information that the application populates for its own nefarious purposes or it can be a column based on a calculation of other columns. Since they are updatable, they can also be useful when data needs to be updated. When you use a DataTable for this purpose, the DataAdapter does the updates using the associated DataCommands for Insert, Update and Delete operations. Since a DataTable is disconnected, it won't inherently detect if there are concurrency conflicts ... you need to do that in your insert/update/delete commands. Handling these concurrency conflicts is a topic all in itself as there is no one "right" way to do it (how you do it depends on requirements and goals).DataTables also work with the System.Data.DataView class, allowing you to sort and filter the DataTable without making a round trip to the database. This is especially useful for caching scenarios.DataTables and DataSets are also serializable (unlike DataReaders) which means that you can easily pass them across the wire from a web server to a client application or persist them to disk. Typed DataTable/DataSet These are custom DataTables/DataSets that inherit from the base classes (above) and add strongly typed properties, methods and classes to the experience. Visual Studio has a designer that enables you to create these in a visual manner. When to use Typed DataTables and DataSets provide the best design-time experience ... you'll get intellisense, compile-time type-checking for your fields, key (both primary and foreign) enforcement and methods for navigating the relationships between tables. Yes, you can add relationships and keys to tables in an untyped DataSet, but you have to write the code for it. With the typed DataTables/DataSets, it's already in there (well, as long as you define them in the designer, that is). The custom columns mentioned above can also be defined in the DataSet designer as can additional Fill/Get methods for the DataAdapter to use. From a RAD perspective, typed DataSets are the way to go ... all of the stuff that you have to wire up with the untyped versions is done for you by the designer. Like their untyped progenitors, they can be serialized but this time they have strong typing when serializing. That said, typed DataTables and DataSets are the least performant, primarily due to instantiation cost (all of the fields, relationships, etc. are created in generated code when you instantiate one of these puppies). But, realistically, in many scenarios, this hit is worth the benefit to speed application development ... it's one of those things where you need to balance your requirements and goals. Typically, I'll tend to lean towards the RAD beneifts of typed datasets and use untyped only when there is no need at all for things like updates and strong typing (they do exist). Regardless of which method that you use, I have to say that I do love the way it is virtually impossible to get yourself into an endless loop with ADO.NET (you can, but you have to work at it). With DataReaders, you loop while(rdr.Read()). For DataTables it's foreach(DataRow rw in tbl.Rows) and DataViews is foreach(DataRowView vw in view). Very nice. If you've ever done ADO "Classic", you've forgotten to call MoveNext and had to get medieval on the process (and that could be IIS) to get it to stop running circles around itself. And, if you've done ADO "Classic", you have forgotten to do that. More than once. If you say you haven't done that, you're lying.So ... I hope that helps a bit. Unfortunately, there are times when there isn't a hard line in the sand when you choose one vs the other, especially when deciding between typed and untyped datasets. That's one of the reasons why I often refer to development more as an art than as a true engineering discipline ... in my mind, engineering is more defined and cut and dry. Calling it an art doesn't mean that you don't need rigor and discipline ... remember, the best artists are so because of the rigor and discipline that they apply to their art. Of course, that could also be because, long ago, I turned away from engineering as a major (what I intended when I started college) and went into English Lit instead (yes, lit is an artform). I know that I didn't even touch in Linq here ... that can complicate this decision tree as well.

New Account Email Validation (Part II)

.NET Stuff | Security | Web (and ASP.NET) Stuff
In my previous post, I discussed the things to keep in mind with new account validation. Well, as promised, I've done a sample of one way to do this. Certainly step 1 to to do as much as possible without writing any code, following the KISS principle. Since I am using the CreateUserWizard Control, I set the DisableCreatedUser property to true and LoginCreatedUser to false. Easy enough. But that's not the whole story. We need to generate the actual validation key. There are a lot of ways that one can do this. Personally, I wanted, as much as possible, to not have any dependency on storing the validation code in the database anywhere. This, of course, ensures that, should our database be penetrated, the validation codes cannot be determined. With that, then, the validation code should come from data that is supplied by the user and then generated in a deterministic way on the server. Non-deterministic, of course, won't work too well. I started down (and really, almost completed) a path that took the UserName and Email, concatenated them, generating the bytes (using System.Security.Cryptography.Rfs2898DeriveBytes) to create a 32-byte salt from this. I again concatenated the UserName and email, then hashing it with SHA1. This certainly satisfied my conditions ... the values for this would come from the user and so the validation code didn't need to be stored. And it was certainly convoluted enough that a validation code would be highly difficult to guess, even by brute force. In the email to the user, I also included a helpful link that passed the validation code in the query string. Still, this code was some 28 characters in length. Truly, not an ideal scenario. And definitely complex. It was certainly fun to get the regular expression to validate this correct ... more because I'm just not all that good at regular expressions then anything else. If you are interested, the expression is ^\w{27}=$, just in case you were wondering. Thinking about this, I really didn't like the complexity. It seems that I fell into that trap that often ensnares developers: loving the idea of a complex solution. Yes, it's true ... sometime developers are absolutely drawn to create things complex solutions to what should be a simple problem because they can. I guess is a sort of intellectual ego coming out ... we seem to like to show off how smart we are. And all developers can be smitten by it. Developing software can be complex enough on its own ... there really is no good reason to add to that complexity when you don't need to. There are 3 key reasons that come to mind for this. 1) The code is harder to maintain. Digging through the convolutions of overly complicated code can make the brain hurt. I've done it and didn't like it at all. 2) The more complex the code, the more likely you are to have bugs or issues. There's more room for error and the fact that it's complicated and convoluted make it easier to introduce these errors and then miss them later. It also makes thorough testing harder, so many bugs may not be caught until it's too late. So, I wound up re-writing the validation code generation. How did I do it? It's actually very simple. First, I convert the user name, email address and create date into byte arrays. I then loop over all of the values, adding them together. Finally, I subtract the sum of the lengths of the user name, password and creation date and subtract from the previous value. This then becomes the validation code. Typically, it's a 4 digit number. This method has several things going for it. First, it sticks to the KISS principle. It is simple. There are very few lines of code in the procedure and these lines are pretty simple to follow. There are other values that can be used ... for example, the MembershipUser's ProviderKey ... when you are using the Sql Membership provider, this is a GUID. But not depending on it gives you less dependence on this. Second, it is generated from a combination of values supplied by the user and values that are kept in the database. There is nothing that indicates what is being used in the code generation ... it's just a field that happened to be there. This value is not as random as the previous, I know. It's a relatively small number and a bad guy could likely get it pretty quickly with a brute-force attack if they knew it was all numbers. To mitigate against this, one could keep track of attempted validations with the MembershipUser using the comments property, locking the account when there are too many attempts within a certain time period. No, I did not do this. Considering what I was going to use the for (yes, I am actually going to use it), the potential damage was pretty low and I felt that it was an acceptable risk. Overall, it's a pretty simple way to come up with a relatively good validation code. And it's also very user-friendly. Here's the code:public static string CreateValidationCode(System.Web.Security.MembershipUser user) { byte[] userNameBytes = System.Text.Encoding.UTF32.GetBytes(user.UserName); byte[] emailBytes = System.Text.Encoding.UTF32.GetBytes(user.Email); byte[] createDateBytes = System.Text.Encoding.UTF32.GetBytes(user.CreationDate.ToString()); int validationcode = 0; foreach (byte value in userNameBytes) { validationcode += value; } foreach (byte value in emailBytes) { validationcode += value; } foreach (byte value in createDateBytes) { validationcode += value; } validationcode -= (user.UserName.Length + user.Email.Length + user.CreationDate.ToString().Length); return validationcode.ToString(); } Architecturally, all of the code related to this is in a single class called MailValidation. Everything related to the validation codes is done in that class, so moving from the overly-complex method to my simpler method was easy as pie. All I had to do was change the internal implementation. Now that I think of it, there's no reason why it can't be done using a provider model so that different implementations are plug-able. Once the user is created, we generate the validation code. It is never stored on the server, but is sent to the user in an email. This email comes from the MailDefinition specified with the CreateUserWizard ... this little property points to a file that the wizard will automatically send to the new user. It will put the user name and password in there (with the proper formatting), but you'll need to trap the SendingMail event to modify it before it gets sent in order to put the URL and validation code in the email. //This event fires when the control sends an email to the new user. protected void CreateUserWizard1_SendingMail(object sender, MailMessageEventArgs e) { //Get the MembershipUser that we just created. MembershipUser newUser = Membership.GetUser(CreateUserWizard1.UserName); //Create the validation code string validationCode = MailValidation.CreateValidationCode(newUser); //And build the url for the validation page. UriBuilder builder = new UriBuilder("http", Request.Url.DnsSafeHost, Request.Url.Port, Page.ResolveUrl("ValidateLogin.aspx"), "C=" + validationCode); //Add the values to the mail message. e.Message.Body = e.Message.Body.Replace("<%validationurl%>", builder.Uri.ToString()); e.Message.Body = e.Message.Body.Replace("<%validationcode%>", validationCode); } One thing that I want to point out here ... I'm using the UriBuilder class to create the link back tot he validation page. Why don't I just take the full URL of the page and replace "CreateAccount.aspx" with the new page? Well, I would be concerned about canonicalization issue. I'm not saying that there would be any, but it's better to be safe. The UriBuilder will give us a good, clean url. The port is added in there so that it works even if it's running under the VS development web server, which puts the site on random ports. I do see a lot of developers using things like String.Replace() and parsing to get urls in these kinds of scenarios. I really wish they wouldn't. Things do get a little more complicated, however, when actually validating the code. There is a separate form, of course, that does this. Basically, it collects the data from the user, regenerated the validation key and then compares them. It also checks the user's password by calling Membership.ValidateUser. If either of these fails, the user is not validated. Seems simple, right? Well, there is a monkey wrench in here. If the MembershipUser's IsValidated property is false, ValidateUser will always fail. So we can't fully validate the user until they are validated. But ... we need the password to validate their user account. See the problem? If I just check the validation code and the password is incorrect, you shouldn't be able to validate. What I had to wind up doing was this: once the validation code was validated, I had to then set IsApproved to true. Then I'd called ValidateUser. If this failed, I'd then set it back. protected void Validate_Click(object sender, EventArgs e) { //Get the membership user. MembershipUser user = Membership.GetUser(UserName.Text); bool validatedUser = false; if (user != null) { if (MailValidation.CheckValidationCode(user, ValidationCode.Text)) { //Have to set the user to approved to validate the password user.IsApproved = true; Membership.UpdateUser(user); if (Membership.ValidateUser(UserName.Text, Password.Text)) { validatedUser = true; } } } //Set the validity for the user. SetUserValidity(user, validatedUser); } You do see, of course, where I had to Approve the user and then check. Not ideal, not what I wanted, but there was really no other way to to it. There are a couple of things, however, that I want to point out. Note that I do the actual, final work at the very end of the function. Nowhere am I called that SetUserValidity method until the end after I've explored all of code branches necessary. Again, I've seen developers embed this stuff directly in the If blocks. Ewww. And that makes it a lot harder if someone needs to alter the process later. Note that I also initialize the validatedUser variable to false. Assume the failure. Only when I know it's gone through all of the tests and is good do I set that validatedUser flag to true. It both helps keep the code simpler and ensure that if something was missed, it would fail. Well, that's it for now. You can download the code at http://code.msdn.microsoft.com/jdotnet.

New Account Email Validation (Part I)

.NET Stuff | Security | Web (and ASP.NET) Stuff
We’ve all seen it … when you sign up for a new account, your account isn’t active until you validate it from an email sent to the registered email address. This allows sites with public registration to ensure a couple of things. First, that the email provided by the user actually does exist (and they didn’t have a typo). Second, it also validates that the person signing up has access to that email address. Now, let’s be clear, it doesn’t necessarily ensure that the user signing up is the legitimate owner of the email address … there isn’t much more that we can do to actually validate that as we don’t control the email system that they use … but, in a realistic world, that’s the best we can do. Now, there was a little exchange recently on the ASP.NET forums that I had with someone asking how to do this very thing with ASP.NET’s membership system. Of course, this is perfectly possible to do. Now, I do believe in the KISS (that’s Keep It Simple Stupid) principle, so I look at this from a perspective of using as much built-in functionality as possible to accomplish it. So, for example, I’d really prefer not to have any additional database dependencies such as new tables, etc., to support this new functionality (that isn’t there out-of-the-box) as possible. First things first … when the new account is created, the account (represented by the MembershipUser class) should have the IsApproved property set to false. This will prevent any logins until such time as the flag is changed. There are two ways to do this, depending on how you the user signs up. If you are using the built-in CreateUserWizard, you can set the DisableCreatedUser property to true.You can also do it if you are calling the API directly from a custom WebForm (or other method). This is accomplished by calling the CreateUser method on the Membership class. There are two overloads that will allow you to do this; both of them take a boolean IsApproved argument. Again, if this is false, the user won’t be allowed to log in until they are approved. Of course, in extranet-type scenarios with some user self-service, this can be used to validate that the newly registered extranet user is valid via a manual review process. And in those types of cases, because of the very nature of extranets, you would want it to be a manual review process to thoroughly vet the users. Note that you’ll also want to do this if you happen to be a judge and you may have some nasty personal stuff that some people may find offensive think leads to a conflict of interest in a case that you are trying. But that’s not what we are doing here. We want this to be open and completely self-service, but to still validate that the email is valid and the user has access to it, ensuring that we can communicate with them (or spam them, depending on your viewpoint). We’ve already discussed the whole e-mail-account-security thing … nothing that we can do about that, so we’ll just move on. But how can we further ensure that we have a (relatively) secure method for doing this, even with the whole e-mail security issue? First, we need to make sure that whatever validation code we use is not easy for a bad guy to guess … this does defeat the purpose. How far you go with this will certainly depend a great deal on what the risk is from this failure … for example, if this is a site where you have dog pictures, it’s not that big of a deal. If, however, it’s an ecommerce site, you need to be a bit more cautious. Second, we also need to make sure that the validation code wasn’t intercepted en route. Keep in mind – and a number of devs seem to forget this – SMTP is not a secure protocol. Neither is POP3. They never were; they just weren’t designed for it. (This highlights one of the things that I tell developers a lot … There is no majik security pixii dust … you cannot bolt “security” on at the end of the project; it absolutely must be built in from the initial design and architecture phase.) Everything in SMTP is transmitted in the clear, as is POP3. In fact, if you’re feeling ambitious, you can pop open a telnet client and use it for SMTP and POP3. It’s not the most productive but it is enlightening. These are the two things that come to mind that are unique to this scenario. There are additional things that you need to account for … Sql Injection, XSS and the rest of the usual suspects. Now that I’ve said all of that, I will also tell you that I’m working on a sample that shows some techniques for doing this. When I’m done, I will post it here along with a discussion of what was done and what alternative options are that you can do based on your needs, requirements and risk analysis. So … keep tuned right here for more fun and .Net goodness!

Thoughts on Secure File Downloads

.NET Stuff | Security | Web (and ASP.NET) Stuff
Well, that’s kinda over-simplifying it a bit. It’s more about file downloads and protecting files from folks that shouldn’t see them and comes from some of the discussion last night at the OWASP User Group. So … I was thinking that I’d put a master file-download page for my file repository. The idea around it is that there would be an admin section where I could upload the files, a process that would also put them into the database with the relevant information (name, content type, etc.). This would be an example of one of the vulnerabilities discussed last night … insecure direct object reference. Rather than giving out filenames, etc., it would be a file identifier (OWASP #4). That way, there is no direct object reference. That file id would be handed off to a handler (ASHX) that would actually send the file to the client (just doing a redirect from the handler doesn’t solve the issue at all). But I got to thinking … I might also want to limit access to some files to specific users/logins. So now we are getting into restricting URL access (OWASP #10). If I use the same handler as mentioned above, I can’t use ASP.NET to restrict access, leaving me vulnerable. Certainly, using GUIDs makes them harder to guess, but it won’t prevent UserA, who has access to FileA, sending a link to UserB, who does not have access to FileA.  However, once UserB logged in, there would be nothing to prevent him/her from getting to the file … there is no additional protection above and beyond the indirect object reference and I’m not adequately protecting URL access. This highlights one of the discussion points last night – vulnerabilities often travel in packs. We may look at things like the OWASP Top Ten and identify individual vulnerabilities, but that looks at the issues in isolation. The reality is that you will often have a threat with multiple potential attack vectors from different vulnerabilities. Or you may have a vulnerability that is used to exploit another vulnerability (for example, a Cross-Site Scripting vulnerability that is used to exploit a Cross Site Request Forgery vulnerability and so on and so on). So … what do I do here? Well, I could just not worry about it … the damage potential and level of risk is pretty low but that really just evades the question. It’s much more fun to actually attack this head on and come up with something that mitigates the threat. One method is to have different d/l pages for each role and then protect access to those pages in the web.config file. That would work, but it’s not an ideal solution. When coming up with mitigation strategies, we should also keep usability in mind and to balance usability with our mitigation strategy. This may not be ideal to the purist, but the reality is that we do need to take things like usability and end-user experience into account. Of course, there’s also the additional maintenance that the “simple” method would entail as well – something I’m not really interested in. Our ideal scenario would have 1 download page that would then display the files available to the user based on their identity, whether that is anonymous or authenticated. So … let’s go through how to implement this in a way that mitigates (note … not eliminates but mitigates) the threats. First, the database. Here’s a diagram:                                                               We have the primary table (FileList) and then the FileListXREF table. The second has the file ids and the roles that are allowed to access the file. A file that all are allowed to access will not have any records in this table. To display this list of files for a logged in user, we need to build the Sql statement dynamically, with a where clause based on the roles for the current user. This, by the way, is one of the “excuses” that I’ve heard about using string concatenation for building Sql statements. It’s not a valid one, it just takes some more. And, because we aren’t using concatenation, we’ve also mitigated Sql injection, even though the risk of that is low since the list of roles is coming from a trusted source. Still, it’s easy and it’s better to be safe. So … here’s the code. public static DataTable GetFilesForCurrentUser() { //We'll need this later. List<SqlParameter> paramList = new List<SqlParameter>(); //Add the base Sql. //This includes the "Where" for files for anon users StringBuilder sql = new StringBuilder( "SELECT * FROM FileList " + "WHERE (FileId NOT IN " + "(SELECT FileId FROM FileRoleXREF))"); //Check the user ... IPrincipal crntUser = HttpContext.Current.User; if (crntUser.Identity.IsAuthenticated) { string[] paramNames = GetRoleParamsForUser(paramList, crntUser); //Now add to the Sql sql.Append(" OR (FileId IN (SELECT FileId FROM " + "FileRoleXREF WHERE RoleName IN ("); sql.Append(String.Join(",", paramNames)); sql.Append(")))"); } return GetDataTable(sql.ToString(), paramList); } private static string[] GetRoleParamsForUser(List<SqlParameter> paramList, IPrincipal crntUser) { //Now, add the select for the roles. string[] roleList = Roles.GetRolesForUser(crntUser.Identity.Name); //Create the parameters for the roles string[] paramNames = new string[roleList.Length]; for (int i = 0; i < roleList.Length; i++) { string role = roleList[i]; //Each role is a parameter ... string paramName = "@role" + i.ToString(); paramList.Add(new SqlParameter(paramName, role)); paramNames[i] = paramName; } return paramNames; } From there, creating the command and filling the DataTable is simple enough. I’ll leave that as an exercise for the reader. This still, however, doesn’t protect us from the failure to restrict URL access issue mentioned above. True, UserA only sees the files that he has access to and UserB only sees the files that she has access to. But that’s still not stopping UserA from sending UserB a link to a file that he can access, but she can’t. In order to prevent this, we have to add some additional checking into the ASHX file to validate access. It’d be easy enough to do it with a couple of calls to Sql, but here’s how I do it with a single call … public static bool UserHasAccess(Guid FileId) { //We'll need this later. List<SqlParameter> paramList = new List<SqlParameter>(); //Add the file id parameter paramList.Add(new SqlParameter("@fileId", FileId)); //Add the base Sql. //This includes the "Where" for files for anon users StringBuilder sql = new StringBuilder( "SELECT A.RoleEntries, B.EntriesForRole " + "FROM (SELECT COUNT(*) AS RoleEntries " + "FROM FileRoleXREF X1 " + "WHERE (FileId = @fileId)) AS A CROSS JOIN "); //Check the user ... IPrincipal crntUser = HttpContext.Current.User; if (crntUser.Identity.IsAuthenticated) { sql.Append("(SELECT Count(*) AS EntriesForRole " + "FROM FileRoleXREF AS X2 " + "WHERE (FileId = @fileId) AND " + "RoleName IN ("); string[] roleList = GetRoleParamsForUser(paramList, crntUser); sql.Append(String.Join(",", roleList)); sql.Append(")) B"); } else { sql.Append("(SELECT 0 AS EntriesForRole) B"); } DataTable check = GetDataTable(sql.ToString(), paramList); if ((int)check.Rows[0]["RoleEntries"] == 0) //Anon Access {return true;} else if ((int)check.Rows[0]["EntriesForRole"] > 0) {return true;} else {return false;} } So, this little check before having the handler stream the file to the user makes sure that someone isn’t getting access via URL to something that they shouldn’t have access to. We’ve also added code to ensure that we mitigate any Sql injection errors. Now, I’ve not gotten everything put together in a “full blown usable application”. But … I wanted to show some of the thought process around securing a relatively simple piece of functionality such as this. A bit of creativity in the process is also necessary … you have to think outside the use case, go off the “happy path” to identify attack vectors and the threats represented by the attack vectors.