Web/ASP.NET

Scraping, or Programatically Accessing, a Secure Webpage

Ryan Farley — Tue, 26 Aug 2008 06:07:01 GMT

There are many secure websites out there that provide useful information but do not have a public API to access it's data. A prime example of this is the LinkedIn website. You might love to gather some info from LinkedIn, but their promise to deliver a public API has yet to come to fruition. The problem is, the pages with all the good data are secure, requiring the user to log in before accessing these pages. Let's say we want to scrape this data from these pages programatically? We need to authenticate to access these pages. We can do that by reusing the authentication cookie from the site that we receive when we log in with a browser.

Note: I've mentioned LinkedIn as an example of a secure site to programatically access data from. It's actually a violation of LinkedIn's user agreement to scrape data from it's site. The techniques here apply to any form-based authenticated website, built on ASP.NET or anything else.

Before we move on with this, I wanted to state a few assumptions with the approach I'll be showing here:

You already have browser-based access to the secure pages (meaning you have a user account).
You're OK to use your own authentication cookie to access the secure pages without violating some site agreement. Doing this sort of thing on a site that prohibits it can get you banned from the site. Remember, you'll be using something that can link these requests back to your own user account.

When you visit a webpage that requires some sort of form-based authentication, usually there is an authentication token stored in a cookie. This certainly is the case with any ASP.NET site using Forms Authentication and is the case with LinkedIn as well as about any other similar site out there. Even if it isn't a persistent cookie, and is only active for the session, there still is a cookie. This cookie is passed back to the website in the request header each time a page is accessed. We can sniff out the data for that cookie using Firebug (the awesomely-awesome Firefox addon), or using Fiddler for IE.

Using Firebug, we can access the secure page and take a look at the cookie value in the header. For my example, I'll be using a sample ASP.NET site using forms authentication, but this all works the same for non-ASP.NET sites too.

If you're using Fiddler, just access the page using IE and then you'll see the cookie data by going to the Headers section under the Session Inspector.

For an ASP.NET site using forms authentication, the authentication token name is indicated in the "name" attribute of the forms key in the authentication section of the web.config. By default that name is ".ASPXAUTH", but you won't know what that name is, or the site might not even be an ASP.NET site. That is OK. You can usually pick out the authentication token in the cookie data, or just use the entire cookie.

Now, using that cookie, we can use the following code to access the secure webpage:

using System.Net;
using System.IO;
//...
 
 
//grab cookie authentication token from Firebug/Fiddler and add in here
string cookiedata = ".ASPXAUTH=FB8ADA49D4BFE4EF531A4539D0B74CCA6762F9CC6F62C8E...";

HttpWebRequest request = HttpWebRequest.Create("http://somesite.com/securepage.aspx") as HttpWebRequest;
//set the user agent so it looks like IE to not raise suspicion 
request.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)";
request.Method = "GET";
//set the cookie in the request header
request.Headers.Add("Cookie", cookiedata);

//get the response from the server
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
using (Stream stream = response.GetResponseStream())
{
    using (StreamReader reader = new StreamReader(stream))
    {
        string pagedata = reader.ReadToEnd();
        //now we can scrape the contents of the secure page as needed
        //since the page contents is now stored in our pagedata string
    }
}
response.Close();

One thing to point out here, since we're passing the entire contents of the cookie, I'm just adding that as a whole to the request header instead of adding each cookie element to the request.Cookie container.

If I wire that up in a form, we'll quickly see the secure page since the site will see the authentication token in the cookie sent in the page header and will not redirect us to the login page.

Not bad. We can now scrape any data we'd like from the secure page contents. Just to point out, we're not just limited to reading the data. We can also send form data back to the site as a POST on secure pages as well.

Why I Am No Longer Supporting IE6

Ryan Farley — Tue, 19 Aug 2008 01:17:09 GMT

In the recent relaunch of this site, I created a new custom skin for Subtext. Since this is just a personal site, I threw things together fairly quickly, testing along the way with Firefox3 and IE7. Once I was getting closer to complete, I took a look at my new site in all the usual browsers, including IE6. Ugh, it looked terrible. I gave it some thought and made some decisions about supporting IE6. I'm not going to support it. Not on this site and not on others that I have a say in. I'm not talking about leaving my site unusable for IE6 users - they just won't get as good of an experience. I'm not alone with this decision. Read on to see why I am no longer supporting IE6.

As I mentioned above, I'm not talking about leaving the site inaccessible or unusable for IE6 users. I completely understand that there are many corporate machines that are held back by IT restrictions & policies where IE6 is the only choice. I'm not talking about making my sites inaccessible to these users, or even those who just don't upgrade due to their own ignorance. However, I'm just going to choose to no longer dumb-down my sites to support them. They'll be able to access my sites. They just won't have as good of an experience.

I know that this sort of decision cannot be made about all sites. It's a smart idea for major public sites to be able to fall back to IE6 support if needed. Most of my sites are focused on the developer, either in the form of a blog or a community site. Knowing that this is my audience I feel completely justified in not supporting IE6 and focusing only on more current browsers. When redesigning my new site, I intentionally used PNG images with transparencies among other things that I knew wouldn't look cool for IE6 users. I'm OK with that. I don't think that is me being "elitist" or anything, again the audience for this site is developers - although there is still a disturbing percentage of the traffic that is using IE6.

Letting IE6 Users Know What They Are Missing
OK. As I mentioned, IE6 users can still read everything on this site. Things are way too spaced apart and many dividers are not where they are supposed to be, but things are still quite readable. I don't want these users thinking that this is how my site is supposed to look. Come on, that would be embarrassing. Instead of fixing it however, I've decided to add a little item to the header area of the page to let them know that "I know the site looks like crap, it's because of your browser".

Users who come to this site using IE6 will see the above and can come read more on this post. I'm completely on-board with Mad Kristensen's post creating better experiences for your visitors, I'll just be focusing on creating a better experience for the users with more modern browsers.

What's Wrong With IE6?
This is a development blog, so I shouldn't have to explain this to anyone here, but off top of my head (this is a small part of a very large list):

Lack of support for current standards for HTML markup, CSS, etc
Support for non-standard features not compatible with other browsers
No PNG transparency support
Released in 2001, we've completely moved beyond everything about IE6. This is a browser that PC World rated one of the worst tech products of all time.

How Long Will Developers Have to Continue Supporting IE6?
At some point, we'll all have to be OK with putting and end to adding support for IE6 in our sites. 37Signals dropped support for IE6 in their products last Friday, Aug 15th. I've decided that for my sites, the time is now as well.

If you'd like to end supporting IE6 in your sites as well, all you have to do is focus on developing for newer browsers. Let the site suck for IE6 users IMO. To let them know that "you know it sucks and they should upgrade", SaveTheDevelopers.org has a nice little Javascript you can add to your site to drop down a little message on the top of your pages that encourages the user to upgrade their browser to IE7 or install Firefox, Opera, Safari, etc.

Say No To IE 6!

Our current campaign focuses on assisting users in upgrading their Internet Explorer 6 web browser. This campaign will result in former IE 6 users having a more enjoyable experience on the web while (hopefully) creating a less stressful and complicated environment for web developers by hastening the retirement of an outdated browser.

OK. Enough for me. I'm sold. No more support for IE6 from me. IE6 users, sorry. You can still use my sites, they just won't look as nice. As a consolation prize, you will at least get to see my cool "Eek IE6!" message.

More on Device Filtering With ASP.NET Server Control Properties

Ryan Farley — Thu, 14 Aug 2008 19:13:57 GMT

I posted yesterday about setting ASP.NET Browser control properties differently for different browsers by using device filtering syntax for setting the properties. I've received some questions via e-mail about that post so I wanted to follow up on some additional things I've found on this topic.

See this first: Set Browser Specific ASP.NET Server Control Properties and Taking an ASP.NET Site Offline

1) This can only be done declaratively. If you need to do this in the code-behind you'll need to use Request.Browser. Example:

switch (Request.Browser.Browser.ToLower())
{
    case "ie":
        if (Request.Browser.MajorVersion == 7)
            labelText.Text = "Hello IE7!";
        else 
            labelText.Text = "Hello IE (time to upgrade!)";
        break;
    case "firefox":
        labelText.Text = "Hello Firefox";
        break;
    default:
        labelText.Text = "Hello other browser";
        break;
}

2) There is no designer support. Don't expect to see this stuff in the intellisense. It won't be there, but it will work.

3) Not just for simple properties. This device filtering technique worked with every property I tried it with. In fact, it even worked in other unexpected ways as well, such as with templated controls. Take a look at the following example:

<asp:Menu runat="server" id="myMenu">
    <ie:Items>
       <asp:MenuItem Text="IE Item" />
    </ie:Items>
    
    <mozilla:Items>
       <asp:MenuItem Text="Firefox Item" />
    </mozilla:Items>

    <Items>
        <asp:MenuItem Text="Other Item" />
    </Items>
</asp:Menu>

Now that is pretty cool.

4) How it works. This didn't take too long to work out. A little bit of digging quickly showed how this all comes together because it works just how you would probably guess. The biggest question is "how do you know what device values you can use for filtering?". In the .NET Framework directory you'll find some config files showing how to behave and render controls for different types of browsers.

C:\Windows\Microsoft.NET\Framework\v2.0.50727\CONFIG\Browsers

In that folder you'll see several config files named things like "ie.browser", "mozilla.browser", "safari.browser", "palm.browser", etc. In each of these files there are several "browser" elements, each with an ID. These ID values are what can be used to declaratively filter server control properties for the specific devices. Mind you, it's not just the names of the files, but the browser ID values contained inside. This means that you can get very specific about the targeted browser for a property value. Instead of just specifying "ie", you could use any of the following "ie", "ie5", "ie5to9", "ieaol", "ie4" and many many more. Also, you can edit these, or add an App_Browsers folder to your propject and place a new file named "something.browser" in it to define your specific browser capabilities.

5) It works with directives too! This was the one that blew me away. You can use it on directives as well! That means, control directives (to modify properties, attributes, templates, etc), page directives (to modify CodePage, MasterPageFile, Culture, Theme, etc), and master page directives (to modify CodeFile, Viewstate, etc). W-w-what? The following will set a separate master page for IE and Firefox and a third for all other browsers:

<%@ Page Language="C#" 
    ie:MasterPageFile="~/MasterPageIE.master" 
    mozilla:MasterPageFile="~/MasterPageFirefox.master" 
    MasterPageFile="~/MasterPageGeneric.master" 
%>

Freaky, I know.

6) Why use this? There are a lot of uses for device filtering with server control properties. Consider this. You have a simple page with a welcome label. It's a simple page so you don't really need a separate mobile version of the page. You could shorten that text easily for anyone on a PocketPC IE or Palm browser to account for the limited screen real estate.

7) You can disallow the use of device filtering in your custom controls if you want. If you build your own control and you don't want the consuming developer to be able to use device filtering with it's properties, you can mark the class with the Filterable(false) attribute and it will ignore the device specific properties that are set for the control.

Well, that is about all I know. I'm still just trying to cope with the fact that this has been there since ASP.NET 2.0 and I am just finding out about it now. Apparently, I'm not the only one since it was difficult to track down details on this.

Set Browser Specific ASP.NET Server Control Properties and Taking an ASP.NET Site Offline

Ryan Farley — Thu, 14 Aug 2008 07:00:18 GMT

Isn't it great when you work with a tool day after day and you thought you knew everything there was to know about it? Then find out something that has been there for a long time that you somehow missed? Here's two things that have been in ASP.NET since version 2.0 that I somehow missed until just recently.

...as if there are only 2? Hehe. I am sure there are many more, but I was actually surprised when I came across these items. They're simple enough that I could not believe that I missed them before. I'm sure that I'm not the only one who missed these items. The fact that I hadn't heard about them indicates that they've not been blogged about heavily or that obvious in the docs (if at all?).

Setting Server Control Properties Based on Target Browser
This one was surprising. I came across this one on John Katsiotis' Web Dev & Stuff. The fact that I've written code so many times to accomplish this, when it's apparently been built in since version 2.0 just kills me. The properties of server controls will let you specify a browser filter to apply the property value for. That is, you can set a different value for a property for IE than for Firefox - and for anything else. I didn't try all properties, but I did try many that would matter, such as Text and OnClientClick etc, and they do work. Take a look at the following markup:

<asp:Label runat="server" ID="labelText" 
    ie:Text="This is IE text" 
    mozilla:Text="This is Firefox text" 
    Text="This is general text" 
/>  

<br />

<asp:Button runat="server" ID="buttonText" 
    ie:Text="IE Button" 
    ie:OnClientClick="javascript:alert('Hello IE!');" 
    mozilla:Text="FF Button" 
    mozilla:OnClientClick="javascript:alert('Hello Firefox!');" 
    Text="General Button" 
    OnClientClick="javascript:alert('Hello everyone else!');" 
/>

Notice the "ie:" and "mozilla:" prefixes on the Text and OnClientClick properties. The results? Take a look. The following was the result in Firefox:

In Internet Explorer:

In Opera (or anything other than IE or Firefox since I've defined properties for those)

Wow. Pretty cool. Now, I am not saying that this will apply in all cases. It doesn't seem to differentiate between IE7 and IE6, so there are some big limitations. However, this is quite a cool trick and one that will without question come in handy.

Update: I posted a follow up that shows a lot more of the capabilities of using these device filters to target specific browsers in ASP.NET.

See More on Device Filtering With ASP.NET Server Control Properties

Taking an ASP.NET Site Offline with a Message
This was another one that I was literally beside myself that I didn't know this one before. A fellow business partner of mine named James Sutton mentioned this one to me. This one's been around since ASP.NET 2.0 as well. If you have an ASP.NET web application site, and you place a text file named "app_offline.htm" in the root of the site, all requests to that website will redirect to that app_offline.htm file. Basically, if you need to take an entire ASP.NET site offline, you can place some nice message in that file. Then, any new requests to a URL, any URL, in that website will redirect to that file allowing you to do maintenance to the site, upgrades, or whatever. It is not really a redirect though. ASP.NET essentially shuts down the site, unloads it from the server, and stops processing any requests to that site. That is, until you delete the app_offline.htm file - then things will continue as normal and your ASP.NET site will load up and start serving requests again.

A super-cool side effect of this is that any files that are locked by the site, such as a database or other resources, are freed since the application domain has been unloaded from the server. This allows you to remove the locks from those files and replace them, without the need to do a full IISRESET, taking down other sites on the server. One thing to keep in mind with this file however, make sure you out enough content in it so it is larger than 512 bytes or IE will consider it a 404 and will display the 404 instead of the contents of your app_offline.htm file.

I love coming across new tricks like this, but it does make me wonder, why didn't I know about these before?

Adding Calendar Items to Outlook or Other Calendar App from a Webpage via iCalendar

Ryan Farley — Fri, 08 Aug 2008 07:42:43 GMT

If you have a website that maintains a list of events for users, it is a great idea to allow users to selectively add those events to their own calendar. Using automation from a website with something like Outlook is a bad idea. It would be blocked by the browser's security and your users might use something else for their calendar. Fortunately, many main-stream (most?) calendar applications, such as Outlook, Windows Calendar on Vista, and a whole lot more, support the iCalendar specification. This makes adding items from your applications a breeze.

The iCalendar Specification
The iCalendar spec (see RFC 2445) was created to extend an earlier created vCalendar spec. This spec basically provides a universally accepted way to represent a calendar item in a file, or really, as text. An iCalendar can represent some single generic calendar item, or an Event, To-Do, Journal item, and can even include free/busy schedule information. We are going to focus here on including only a single event in our iCalendar data, although it can also represent multiple items all within the same iCalendar. When expressed as a file, the iCalendar data will typically have an "ics" extension, but really it is just text. It also has a MIME type of "text/calendar"

Scenario
For our scenario, we'll assume we have a website with events on it. For any of these events, we want to allow the user to be able to selectively add an event to their own local calendar. On the display of an event on our website, we'll provide a link that the user can click to add the event to their calendar. Here's a sample of a website that I've used this on to give you an idea:

The user clicks that link and we get some jazz hands and it is added to the user's calendar. Let's take a look at how it all works.

The iCalendar Class
As I mentioned before, the iCalendar data is just text. The thing about it is, it is ugly text. I don't like ugly text. I want to write it once and then forget it ever existed. So, I created a class that does all the ugly text for me to use whenever I need. If you're really wanting to see more of this ugly text then you and the rest of your zombie friends can go nicely over to the RFC and read all you want.

Here's my class:

using System;
using System.Text;
using System.Collections.Generic;

namespace RF.Events
{
    public enum PriorityLevel : int
    {
        Normal = 5,
        High = 1,
        Low = 9
    }

    public class Event
    {
        private const string _DATEFORMAT = "yyyyMMdd\\THHmmss\\Z";

        public Event() { }

        public DateTime StartTime;
        public DateTime EndTime;
        public string Location = string.Empty;
        public string Title = string.Empty;
        public string Description = string.Empty;
        public bool UseAlarm = true;
        public PriorityLevel Priority = PriorityLevel.Normal;
        public string Category = string.Empty;

        public override string ToString()
        {
            return Output;
        }

        public string Output
        {
            get
            {
                StringBuilder sb = new StringBuilder();

                sb.Append("BEGIN:VEVENT\n\n");
                sb.Append("DTSTART:");
                sb.Append(StartTime.ToUniversalTime().ToString(_DATEFORMAT));
                sb.Append("\nDTEND:");
                sb.Append(EndTime.ToUniversalTime().ToString(_DATEFORMAT));

                sb.Append("\nLOCATION:");
                sb.Append(Location);

                sb.Append("\nCATEGORIES:");
                sb.Append(Category);

                sb.Append("\nTRANSP:OPAQUE\n");
                sb.Append("SEQUENCE:0\n");
                sb.AppendFormat("UID:RFCALITEM{0}\n", DateTime.Now.Ticks);
                sb.Append("DTSTAMP:");
                sb.Append(StartTime.ToUniversalTime().ToString(_DATEFORMAT));

                sb.Append("\nDESCRIPTION:");
                sb.Append(Description);
                sb.Append("\nSUMMARY:");
                sb.Append(Title);
                sb.Append("\n\nPRIORITY:");
                sb.Append((int)Priority);
                sb.Append("\nCLASS:PUBLIC\n");

                if (UseAlarm)
                {
                    sb.Append("BEGIN:VALARM\n");
                    sb.Append("TRIGGER:PT15M\n");
                    sb.Append("ACTION:DISPLAY\n");
                    sb.Append("DESCRIPTION:Reminder\n");
                    sb.Append("PRIORITY:5\n");
                    sb.Append("END:VALARM\n");
                }
                sb.Append("END:VEVENT\n");

                return sb.ToString();
            }
        }
    }

    public class iCalendar
    {
        public iCalendar()
        {
            this.Events = new List<Event>();
        }

        public List<Event> Events;

        public override string ToString()
        {
            return this.Output;
        }

        public string Output
        {
            get 
            { 
                StringBuilder sb = new StringBuilder();

                sb.Append("BEGIN:VCALENDAR\n");
                sb.Append("PRODID:-//RyanFarley.com//iCalendar Sample MIMEDIR//EN\n");
                sb.Append("VERSION:2.0\n");
                sb.Append("METHOD:PUBLISH\n");

                foreach (Event ev in Events)
                    sb.Append(ev.Output);

                sb.Append("END:VCALENDAR");

                return sb.ToString();
            }
        }
    }
}

So, first question. Why the separation of the Event item from the rest of the iCalendar item? We'll get to that. For now, here's how we'll use that class. Let's add it to a handler to send back the iCalendar data in the response. We'll call this handler from the link we provide to the user (Typically we would do something like pass an "event ID" of some kind to it and grab the associated data from a database or something. We'll ignore all that stuff in this example to keep it simple).

Here's our handler code:

<%@ WebHandler Language="C#" Class="CalendarItem" %>

using System;
using System.Web;
using System.Text;
using RF.Events;

public class CalendarItem : IHttpHandler 
{
    public void ProcessRequest(HttpContext context) 
    {
        // Set up the Event item first. This would typically come 
        // from a database or some object in the application
        Event ev = new Event();
        ev.Title = "Some Event";
        ev.Description = "This is a test event. Yeye.\\nKTHXBYE";
        ev.Location = "At ryanfarley.com";
        ev.UseAlarm = true;
        ev.StartTime = DateTime.Parse("11/1/2008 8:00:00 AM");
        ev.EndTime = DateTime.Parse("11/1/2008 10:00:00 AM");
        ev.Priority = PriorityLevel.Normal;
        ev.Category = "Test Category";
        
        // Now add the event to an iCalendar
        iCalendar ical = new iCalendar();
        ical.Events.Add(ev);

        // Set the content-type of the response and file name
        // so Windows knows how to handle the response.
        context.Response.ContentType = "text/calendar";
        context.Response.AddHeader("content-disposition", "inline;filename=CalendarEvent.ics");

        // Write the bytes of the iCalendar item output to the resonse stream
        context.Response.BinaryWrite(new System.Text.ASCIIEncoding().GetBytes(ical.Output));
        context.Response.End();
    }
 
    public bool IsReusable 
    {
        get { return false; }
    }
}

If we add that as a hyperlink on a webpage, like this:

The user can click it (this is where we get the jazz hands) and they are prompted with this:

Of course, here they click OK and they are presented with some sort of dialog from whatever app is associated with ics files. On my machine that is Outlook, so I get this:

Cool. The user clicks the Save & Close and it now appears on their calendar. We're not just limited to do this from a webpage either. We could just as easily save the iCalendar.Output to a file and Process.Start it to open it for the user if we're in a Windows app. Or attach it to an e-mail to a user, similar to services like WebEx, GotoMeeting, and other services that send you an e-mail with an iCalendar file attached.

Adding Multiple Events to Create an iCalendar Feed
Now, back to that separation of the Event from the iCalendar class. Why is that? The iCalendar spec indicates that an iCalendar can have one, or multiple events in it. If you've ever added an iCalendar feed to Outlook from your Google Calendar or TripIt, etc, this feed is not an RSS feed or something like that. It is a big long iCalendar file containing multiple events in it. We can create the same with the following code:

        // Create the iCalendar
        iCalendar ical = new iCalendar();
        
        // Add the first event and add it to the iCalendar
        Event ev = new Event();
        ev.Title = "Some Event #1";
        ev.Description = "This is a test event #1. Yeye.\\nKTHXBYE";
        ev.Location = "At ryanfarley.com";
        ev.UseAlarm = true;
        ev.StartTime = DateTime.Parse("11/1/2008 8:00:00 AM");
        ev.EndTime = DateTime.Parse("11/1/2008 10:00:00 AM");
        ev.Priority = PriorityLevel.Normal;
        ev.Category = "Test Category";        
        ical.Events.Add(ev);

        // Add the second event and add it to the iCalendar
        ev = new Event();
        ev.Title = "Some Event #2";
        ev.Description = "This is a test event #2. Yeye.\\nKTHXBYE";
        ev.Location = "At ryanfarley.com";
        ev.UseAlarm = true;
        ev.StartTime = DateTime.Parse("11/5/2008 8:00:00 AM");
        ev.EndTime = DateTime.Parse("11/5/2008 10:00:00 AM");
        ev.Priority = PriorityLevel.High;
        ev.Category = "Test Category";
        ical.Events.Add(ev);
        
        // ...you get the idea
        
        // Now out iCalendar contains multiple events.
        // We can output it all to the response stream.

        // Set the content-type of the response and file name
        // so Windows knows how to handle the response.
        context.Response.ContentType = "text/calendar";
        context.Response.AddHeader("content-disposition", "inline;filename=CalendarEvent.ics");

        // Write the bytes of the iCalendar item output to the resonse stream
        context.Response.BinaryWrite(new System.Text.ASCIIEncoding().GetBytes(ical.Output));
        context.Response.End();

What happens now when the user clicks our link? They'll get prompted just as before, but when they click OK to add it, it will add the iCalendar feed as a second calendar in their calendar application. In Outlook it will show as a separate calendar which I can view side-by-side with my existing one or overlay on top of my existing one. In this case we'd also want to include the appropriate 304 response status in case the calendar item hasn't changed so we keep things nice.

So that is it. Not bad work and we've got a reusable class we can use for single item, or complete calendar feeds. Have fun.

Download the iCalendar class

Must Have .NET Developer Podcasts

Ryan Farley — Wed, 30 Jul 2008 21:56:16 GMT

In my last post I mentioned some of the podcasts that I've been listening to that have inspired me as a .NET developer. I thought it would be a good idea to start a list of all the podcasts that have taken over my Zune lately and some that I am planning on checking out.

My Current Favorites

HerdingCode

HerdingCode is just a great discussion between Jon Gallaway, K. Scott Allen, Kevin Dente, and Scott Koon on .NET development topics. I suppose the thing that I really like about it is that it's just down to earth, like you're just listening in on a conversation that they would be having anyway.

Hanselminutes

There's likely few that don't know who Scott Hanselman is. He manages to carry the same passion and energy he shows on his blog to the Hanselminutes show. His co-host is Carl Franklin (see DotNetRocks below) who is always equally as great.

DotNetRocks

DotNetRocks, from Carl Franklin, has been around for a long time. Carl keeps interesting topics on the show that range from development related, to technology. The show isn't supposed to be a show about .NET, but is a show for .NET developers. Carl's co-host is Richard Campbell, who always seems to know what he is talking about.

Deep Fried Bytes

Deep Fried Bytes is a show from Keith Elder and Chris "Woody" Woodruff. This is more of a technology show, but with a strong slant for .NET developers. Very laid back and they've had some great guests.

ALT.NET Podcast

The Alt.NET podcast is a podcast from the ALT.NET movement that focuses on improving ourselves as developers, challenging assumptions, and looks at things available to help us in our pursuit of excellence as software developers. This is one I've just added to my list, so I'm still pretty much getting started with it. So far it's been great (but it is a bit less casual than some of the others I've listed).

Others I Plan on Checking Out

PolymorphicPodcast

ASP.NET Podcast

StackOverflow

Are there any other great ones I'm missing from the list?

SQL Queries to Analyze SharePoint Usage

Ryan Farley — Tue, 30 May 2006 18:45:00 GMT

I came across this list on CodeProject of SQL Queries to analyze SharePoint details and usage. This is an awesome list. Granted it does bypass the SBS object model so isn't the recommended route, but still a great list of queries you could use to make reporting on your SharePoint sites a breeze.

Useful SQL Queries to Analyze and Monitor SharePoint Portal Solutions Usage

Top 30 Popular Posts

Ryan Farley — Thu, 25 May 2006 18:41:00 GMT

I started this blog in August of 2003, almost 3 years ago. I've made 176 posts in those 3 years. I don't post too often to my blog because I'm not all that big on posting stories about my kids, wife, dog, etc - although those do come in every now and then. Anyway, even when I have lulls where I am not posting as much, my traffic seems to stay pretty consistent. I'm actually amazed at how much traffic I get, especially when I consider how often I get around to posting (big thanks to all the visitors over the years).

Some of my posts have apparently been useful :-), so I thought I would post the top 30 most popular posts I've made over the years (based on total number of unique views to the posts).

1. Set Focus to an ASP.NET Control
Setting focus to controls in your ASP.NET application is a part of giving your end users the feel that they have come to expect. Making your web applications act more like Windows applications is a key to success (IMO). While setting focus to con...
Total views: 371,568 - Posted on: 21-Dec-2004

2. Using the Web Browser Control in your C# Applications
It can be a powerful thing to display dynamic HTML in your C# applications. It can give your applications a modern look and feel and can make displaying data in non-standard ways easy with some simple markup. We have the web browser ActiveX control that wr...
Total views: 232,037 - Posted on: 23-Dec-2004

3. Writing to Your .NET Application's Config File
There's likely been times that you might have thought that it would make things convenient to write back to your .NET application's config file. The framework provides simple methods for reading from the config file, but gives you nothing for writing value...
Total views: 197,345 - Posted on: 13-Jul-2004

4. Tips for SQL Server Identity Columns
Tips on allowing inserts to identity columns and also for reseeding the identity value for a table....
Total views: 117,941 - Posted on: 19-Dec-2004

5. Multiple Monitors rule
I love multiple monitors. I can't even imagine trying to work without multiple monitors and don't think I could ever go back to a single monitor....
Total views: 96,533 - Posted on: 7-Jan-2004

6. Unable to Start Debugging on the Web Server
I hate that, and it seems that every time I (or a co-worker) gets the error “Unable to Start Debugging on the Web Server” on a machine when attempting to debug an ASP.NET project, I have to scramble to remember what to look at. Here's a few things that has...
Total views: 91,037 - Posted on: 23-Aug-2005

7. Stop Hijacking my Browser!
Often in applications you have the need to launch a browser window to a specified URL. I recently evaluated various RSS readers where links to blog posts could be launched in an external browser window. What I found was that most of these applications woul...
Total views: 90,029 - Posted on: 16-May-2004

8. Determining the Control that Caused a PostBack
Many times you might need to perform some action on an ASP.NET postback based on the control that caused the postback to occur. Some scenarios for this might include a form with many regions, each having it's own CustomValidator and the ability to perform...
Total views: 69,089 - Posted on: 11-Mar-2005

9. Solving problems through programming...and why the Skype API sucks
I love to take the approach of solving computing problems through programming. Sometimes it backfires and I over-complicate the problem (I have been known to prematurely generalize from time to time). But usually I bask in the greatness of being a programm...
Total views: 60,149 - Posted on: 14-Jan-2005

10. Enabling XP Themes in your .NET Applications
When you build Windows applications in .NET, by default your application will not have support for XP Themes, or Visual Styles. It is an easy enough task to do, and I think it goes a long way in giving your application a complete and professional look & fe...
Total views: 49,397 - Posted on: 5-May-2004

11. On the Subject of Dates in T-SQL
While we're on the subject of dates in T-SQL, I never liked getting the month and year for a date and sticking an '01' in the middle (then casting it all back to a datetime) to get the first day of the month for a given date value. Then you do the same to...
Total views: 42,317 - Posted on: 15-Feb-2005

12. Determining if a Date is a Weekday in T-SQL
I was reminded of a SQL function to determine if a date was a weekday or a weekend I wrote a while back when I saw the requirements of a project a colleague was working on. You'll see this requirement fairly often in many business applications. A company m...
Total views: 40,265 - Posted on: 14-Feb-2005

13. Awesome web.config Changes in ASP.NET 2.0
There are some really cool changes coming in ASP.NET's web.config files that I am really excited about. I'll just point out a few that I've used (I hate going back to 1.1 because I can't use them). The web.config file in ASP.NET 2.0 allows you to set a lot...
Total views: 40,145 - Posted on: 8-Jun-2004

14. Setting the Value of a TextBox with TextMode=Password
When the TextMode property of an ASP.NET TextBox is set to Password the value set in the Text property will not display at runtime. This can be a pain, however it is actually by design to prevent the unmasked password from being displayed in the HTML sourc...
Total views: 36,984 - Posted on: 18-Dec-2004

15. T-SQL Olympics
OK, there is not an Olympics for T-SQL - but there should be. A friend of mine and I were talking the other day about “gold-medal” T-SQL we've come accross. The one that wins the gold for me is the code I found a long time ago on SQL Server Central from Ma...
Total views: 36,617 - Posted on: 25-Oct-2004

16. T-SQL: SET vs SELECT when assigning variables
Do you know what the difference is between using SET and SELECT when assigning varaibles in T-SQL? Well, there is a difference. I came accross a great article by Narayana Vyas Kondreddi from the UK that describes the difference between the two....
Total views: 36,204 - Posted on: 1-Mar-2004

17. Interacting with the Web Browser Control
In my last post, I outlined some ways to make the Web Browser control more useful in your C# applications, to include things such as printing and setting the text or html of the browser dynamically. That is all good, but in a typical application it does li...
Total views: 35,885 - Posted on: 27-Dec-2004

18. Disabling the Windows Close action
There are times that you'll see a Window that has a close button in the titlebar, but it is disabled. This is often found in applications where the dialog/window changes it's status past a stoppable point so the Windows close action is removed so the user...
Total views: 35,796 - Posted on: 12-Apr-2004

19. Creating a IWin32Window from a Win32 Handle
There are times when you are integrating your .NET applications with other existing applications that you cannot modify and is possibly even non-.NET application. This can often result in problems integrating your .NET application's windows with the other...
Total views: 35,465 - Posted on: 23-Mar-2004

20. In Search of the Perfect RSS Reader
For the last year or two, I've been switching from reader to reader. While each one had features that I loved, they all seemed to fall short on one thing or another and I was never really 100% pleased with any of them. I went from SharpReader, to RSSBandit...
Total views: 33,384 - Posted on: 14-May-2004

21. Flattening Out Data with One of the Coolest SQL Tricks Ever
Not sure why, but I seem to be on a T-SQL kick lately - so here's another T-SQL post. One of my favorite T-SQL hacks ever is one that can flatten out data by taking a value from multiple rows and concatenating the values into a single string....
Total views: 33,281 - Posted on: 17-Feb-2005

22. Dynamically Loading Master Pages in ASP.NET 2.0
One of the cool new things introduced in ASP.NET 2.0 is Master Pages. Master Pages give you the ability to define a master page layout and look that is used throughout a site to give a consistent look & feel to all pages. Any updates or changes to the look...
Total views: 33,156 - Posted on: 16-Jun-2004

23. Communication between applications via Windows Messages
At times I'll build a suite of related, but separate applications. Even though each application is a separate executable, I like to be able to integrate the applications so they can work together. Sending messages between your applications is a great way t...
Total views: 31,548 - Posted on: 10-May-2004

24. Rendering Size (and other things) Correctly in FireFox
One thing that I just can't stand, is when a web page I build looks different in FireFox than how it looks in IE. Well, who doesn't?! The thing that really sucks is that there are things build in to how ASP.NET works that will cause this to happen. So unle...
Total views: 31,325 - Posted on: 6-Sep-2005

25. Returning Objects from Web Services
When I work with web services I want things to work the same way as if I were working with a local layer that returns objects, not data. I don't want my code outside of the service to even see the data, just the objects that represent the data. Who doesn't...
Total views: 27,545 - Posted on: 26-May-2004

26. Disabling Auto-Complete on ASP.NET Forms
Popular browsers, such as Internet Explorer and Firefox support something called Auto-Complete. You've seen this many times. You go to a online form and as you start to type in fields you get a drop-down showing values you've typed in that field before. Th...
Total views: 26,844 - Posted on: 23-Feb-2005

27. Intersection of Date Ranges
A friend of mine called me yesterday about a scheduling application he is working on. His question was so simple, or so it seemed, but it really drove me nuts. Basically he just wanted to find out if two date ranges intersected at all. Simple enough. It wa...
Total views: 24,593 - Posted on: 19-Aug-2004

28. Tricking out the Desktop
Something I have always been a sucker for is tricking out my desktop. I love any kind of tweak or gadget that enhances my pc and the whole user-experience thing. It's one of the many reasons I am looking forward to Vista. I've been a fan of applications th...
Total views: 24,497 - Posted on: 11-Jan-2006

29. Retrieving database independent schema information
Something that I think is often overlooked in the .NET Framework is the cool stuff you can get at using OleDbSchemaGuid to retrieve database schema information. I just thought of this again earlier today when I was reviewing a C# database application where...
Total views: 21,989 - Posted on: 3-Apr-2004

30. Creating Tracking Images for ASP.NET
Often with web applications you want to track traffic statistics to get a general idea of the number of visitors viewing a resource. Whether it be a web page, an RSS feed, an e-mail or whatever, you might want an inconspicuous way to determine that it has...
Total views: 21,725 - Posted on: 10-Jun-2004

Some of these are not as relevant now with .NET 2.0, so I guess I better get back on the bandwagon ;-)

Adding Meta Tags to the Head in ASP.NET 2.0

Ryan Farley — Sat, 25 Mar 2006 19:05:00 GMT

After my last post on adding items to the page head in ASP.NET 2.0, Karthik Nataraaj asked if there was a way to add meta tags as well. You are in luck Karthik. The HtmlMeta class is provided for just that. You can easily create a HtmlMeta object and add it to the Controls collection in the HtmlHead class exposed via Page.Header. Here's a few samples:

// Render: <meta name="keywords" content="Some words listed here" />
HtmlMeta meta = new HtmlMeta();
meta.Name = "keywords";
meta.Content = "Some words listed here";
this.Header.Controls.Add(meta);

// Render: <meta name="robots" content="noindex" />
meta = new HtmlMeta();
meta.Name = "robots";
meta.Content = "noindex";
this.Header.Controls.Add(meta);

// Render: <meta name="date" content="2006-03-25" scheme="YYYY-MM-DD" />
meta = new HtmlMeta();
meta.Name = "date";
meta.Content = DateTime.Now.ToString("yyyy-MM-dd");
meta.Scheme = "YYYY-MM-DD";
this.Header.Controls.Add(meta);

Easy Header Access in ASP.NET 2.0

Ryan Farley — Fri, 24 Mar 2006 20:28:00 GMT

I am easy to please when it comes to small and simple things that make my life as a developer easier. For example, I came accross something I had not noticed before in ASP.NET while reading a post from Dave Burke. The HtmlHead class exposed by the Page class as Page.Header. I love this. It makes it so easy to get to, and manipulate the header attributes for a page. A simple act of changing the page's title, style, etc before was a pain. Now it's just setting a few properties.

To change a page's title:

this.Header.Title = "This is the new page title.";

To add a style attribute for the page:

Style style = new Style();
style.ForeColor = System.Drawing.Color.Navy;
style.BackColor = System.Drawing.Color.LightGray;

// Add the style to the header for the body of the page
this.Header.StyleSheet.CreateStyleRule(style, null, "body");

To add a stylesheet to :

HtmlLink link = new HtmlLink();
link.Attributes.Add("type", "text/css");
link.Attributes.Add("rel", "stylesheet");
link.Attributes.Add("href", "~/newstyle.css");
this.Header.Controls.Add(link);

Simple and elegant. I guess I can finally get rid of my helper classes to do all of that and keep things simpler. Sad that I had not noticed that was there in 2.0 earlier.