Omar Shahine's WebLog : dasBlog

dasBlog Security Advisories

omars — Mon, 07 Mar 2005 05:07:00 GMT

Dominick Baier has posted two security bullitens for dasBlog:

Security Advisory: Log File Path Predictability in dasBlog Community Edition

This issue may or may not affect you. On my host, this does not occur as they lock down /logs from anon users.

Security Advisory : New XSS Vulnerability in dasBlog Community Edition

This issue was fixed in the latest version of dasBlog.

Special thanks to Dominick for his work in this area.

dasBlog 1.7.5016.2 released

omars — Tue, 22 Feb 2005 12:08:00 GMT

We've released dasBlog 1.7.5016.2. This is a release to address almost every bug that was reported since releasing 1.7.5016.1. There is an important Cross Site Security Vulnerability fix in this release and it's recommended that everyone upgrade.

If you are running 1.7 already, the upgrade is a good old xcopy. It's important to note that there is a potential breaking change. If your homeTemplate.blogtemplate file contains the macro <%titleList%> you need to rename it to <%titleList()%>.

Enjoy! Thanks to all the folks that helped make this release happen (including everyone that tool the time to report bugs/problems).

dasBlog 1.7 questions and answers

omars — Sat, 22 Jan 2005 12:08:00 GMT

First of all, the community reaction to dasBlog Community Edition 1.7 has been great! I’m delighted that people have been able to move with little trouble. However, a number of folks have asked some questions and I’d like to answer some of those.

Question: What happened to the Attachment feature, and what is this new Enclosure feature?

Answer: in previous versions of dasBlog there was a way to Attach files etc to a post. However, all this did was move the file from your desktop to your /binary folder and adds a link to the content of an entry. I never thought this was very useful. Furthermore, dasBlog never utilised the enclosure feature of RSS. I decided it was high time we actually make this feature useful, and support RSS enclosure. However, RSS enclosure only allows for a single enclosure per RSS item. My goal was to also allow the old functionality, and support a true n number of attachments but only expose a single one as the RSS enclosure, but I never finished this work. Expect it in the next version.

Question: What happened to the image upload feature?

Answer: Since FreeTextBox has a nice image upload feature via the Gallery I assumed this was good enough for folks and I removed the duplicate functionality in upload image. Well, a number of folks have expressed that they miss the old feature. Expect this to come back in the .2 version of 1.7, and in 1.8 expect a better image upload feature that allows you to store all the images/attachments per Entry in a guid based folder in the /binary folder. I’ve wanted this for a while as I have tons of stuff in my /binary folder that’s orphaned.

Question: The directions for uploading are a bit confusing, can you clear this up?

Answer: I reworked them a bit as I got plenty of feedback that it was confusing. I’ve placed them in the Release Notes.

Question: How do these new anti-spam feature work, and how do I turn them on?

Answer: dasBlog 1.7 has 4 major anti-spam features. They are all disabled by default, and you need to enable them in your Configuration. When enabled there is an additional option to send an HTTP 404 Status Code to blocked referrers, dropping the connection and not wasting any server resources on the referral.

1. CAPTCHA for Comments

This feature will force users to prove that they are human via a proof. The proof is that you mist enter a 6 character series of digits and numbers in the CommentView page before the Comment will be accepted. The aim of this is to prevent automation of comment entry.

2. MovableType Blacklist Support

dasBlog includes support for the MovableType Blacklist by Jay Allen. This will prevent referrals and comments from any domain that is listed in the blacklist.txt file. When enabled, dasBlog will attempt to download this file in the Application Start method, or when enabled from the Configuration page. At most the file is downloaded once per day, and will also be downloaded whenever your ISP sets your ASP.NET worker process to recycle.

3. Content based Blacklist

dasBlog will also filter out any domain from referrals or comments based on keywords that you can enter in the Configuration Page. You should place the entire domain, or any subset of that domain separated by a semi-colon. For example, “foo;bar;foo.com;bar.com”.

4. IP Blacklist

The IP Blacklist must be enabled from the Web.config section (httpModules) since it operates as an HttpModule. The list of the IP addresses to block is located in the blockedips.config file in your SiteConfig folder.

Question: Will you add support for rel=nofollow?

Answer: Yes, this is checked in and will be in .2. It will only be used for Referrals, not for Trackbacks or Pingbacks. If it becomes a problem for Trackbacks and Pingbacks I will add support for this as well. dasBlog does not render html tags from comments so there is nothing to do there.

Question: I’ve found a bug or have a feature request, where do I send them?

Answer: Please enter them into SourceForge

Question: When will bug x be fixed?

Answer: In about a month we will fix the highest priority bugs in a Service Release. This will be the .2 version of 1.7.

Question: I have some code I would like to contribute, but I don’t want or know how to join the project.

Answer: Just zip the code and send it to me. You can get the code from CVS Anonymously.

dasBlog 1.7 released

omars — Wed, 19 Jan 2005 12:34:00 GMT

Well, I'm a bit late to the party... but as Scott announced, dasBlog 1.7 is finished. There are numerous and substantial improvements in this version. Some of them were done months ago, and the most substantial came in the last 2 months as Scott and I feverishly whipped dasBlog into a fast asp.net app. We hope we've also given folks the tools to help combat referral and comment spam.

Anyhoo, here is the 411:

dasBlog is no longer hosted on GotDotNet, and has new home at SourceForge.net. Using CVS as source control actually makes it a joy to check in code to dasBlog.
New Features
Release Notes
YOU MUST UPGRADE your content folder. This is *not* and XCOPY deployment. There were two major bugs fixed in the dasBlog runtime that require that the dasBlog files be rebuilt. Unfortunately there was no avoiding this. Read the instructions in the DasBlogUpgrader download.
For most other details see the wiki

Anyway, I learned a lot about asp.net, programming and dasBlog working on this release. It was really fun and I look forward to working on 1.8 :-).

Some of my favorite 1.7 features

All the search bot referrals to your site are "pretty printed" in your logs
CAPTCHA for entering comments
MovableType Blacklist and Content based Blacklist for visitors to the site
<%ReferralListFiltered()%>Macro that will shrink the number of referrals displayed in the Permalink and expose the rest using javascript and css.
Performance
Lots of smart Caching
Trackbacks/Pingbacks work reliably.

update: if you happen to use dasBlog to do any CrossPosting (post to another blog from dasBlog) DO NOT CrossPost in 1.7. I just found a bug that causes dupes. If you want a fix and can't wait for .2 (which will of course fix these types of things) leave a comment and I will send you an updated patch (one dll).

Referral Spam and Movable Type Blacklist

omars — Mon, 10 Jan 2005 17:26:00 GMT

Well, just in time for a wave of referral spam that is hitting my blog (mostly from http://www.ownsthis.com) I spent part of today writing a class that can consume the Movable Type Blacklist. The class will allow you to download this file from the server periodically (no more than once a day). I have written it such that anyone can integrate this into their .Net blogging package, or any other .Net program. I just checked this into the dasBlog 1.7 tree. The nice thing about this is that the Blacklist is maintained in real time, and you won't have to rely just on content filtering (the stuff that Scott did) but you'll get a pretty long and decent blacklist of bad sites. So far, in the past few hours I've gotten 100% of the referral spam and no false positives...

We are a few days away from releasing the final version of dasBlog 1.7. A very small number of folks have been running the bits over the weekend and as a result we've fixed a few bugs. A couple more days and we'll post the bits to SourceForge.

When that happens I'll post the MovableTypeBlacklist class. I've also considered writing an HttpModule to send these guys 404s, but didn't really think that was appropriate. The list is basically loaded into a long string, delimited by "|" and passed into a Regex to match a url. Interestingly enough, when I tried to Compile the Regex, my little console app balooned to 150 MB and it never quite finished running. Using a static Regex with the long static string I was able to execute matches in 0 - 10 milliseconds.

Here is a dump of the class:

6p.org.uk : True
Executed in : 20 milliseconds

microsoft.com : False
Executed in : 0 milliseconds

shahine.com : False
Executed in : 0 milliseconds

flatbedshipping.com : True
Executed in : 0 milliseconds

apply-to-green-card.org : True
Executed in : 0 milliseconds

ownsthis.com : True
Executed in : 10 milliseconds

RSS xhtml:body

omars — Sat, 08 Jan 2005 16:20:00 GMT

A while ago there was some hubub about support for xhtml:body in the RSS item tag. Since all RSS aggregators must support the description tag, which contains all the escaped html content of the post, having another tag in the RSS feed that duplicates all this content into a tag that only some RSS readers understand seems silly. My RSS feed is 162k today, and if I stop including the body tag it goes down to 82k. Given that I'm doing about 400 MB in bandwidth transfer at my hosting provider and a large chunk of that is RSS, I'm thinking if folks want the body tag from dasBlog they can use the Atom feed.

Dying Thread on Trackbacks, Referrals and Pingbacks

omars — Thu, 06 Jan 2005 12:11:00 GMT

Part 2 of 2

Bug 2: TrackingHandler Thread Dies

Another problem that Scott Hanselman informed me of was that he would frequently stop receiving Trackbacks, Pingbacks and Referrals on his posts. Furthermore, it was intermittent. This was troubling since losing a Trackback means it's lost forever. Well we went hunting in the code, and thanks to some UnitTest of a theory I had found the answer.

Basically the situation is this. Scott gets a lot of traffic. More than I do. There is a thread in dasBlog that sits around waiting for Trackbacks and the like. You use it by calling trackingQueue.Enqueue(tracking) and then trackingQueueEvent.Set(). So basically dasBlog can sit there and queue a bunch of trackings, and when it's ready the thread runs to execute them. The code looks like this:

private void TrackingHandler( )
{
    while ( true )
    {
        Tracking tracking;

        trackingQueueEvent.WaitOne();
        while ( true )
        {
            lock( trackingQueue.SyncRoot )
            {
                tracking = trackingQueue.Dequeue() as Tracking;
            }
            if ( tracking != null )
            {
                try
                {
                    InternalAddTracking( tracking );
                }
                catch (Exception e)
                {
                    ErrorTrace.Trace(TraceLevel.Error,e);
                }
            }

            if ( trackingQueue.Count == 0 )
            {
                break;
            }
        }
    }
}

The objects below are created like so:

trackingQueue = new Queue();
trackingQueueEvent = new AutoResetEvent(false);
trackingHandlerThread = new Thread(new ThreadStart(this.TrackingHandler));
trackingHandlerThread.IsBackground = true;
trackingHandlerThread.Start();

So, can you figure out what is wrong? Well I created a unit test that called this 100 times. What I quickly found out was that even though the code was calling break when the trackingQueue.Count was equal to zero the trackingQueueEvent.WaitOne() call wasn't blocking the while loop from continuing. This caused trackingQueue.Dequeue() to throw an unhanded exception (which should have been in a try catch anyway).

Not knowing a whole lot about this kind of threading I looked at a couple of docs and found the answer. Before calling break I added trackingQueueEvent.Reset(). Problem fixed (I hope).

DateTime.ToString("yyyy-MM-dd")

omars — Thu, 06 Jan 2005 11:29:00 GMT

Recently I found the answer to two very hard questions about bugs in dasBlog. They were kinda tricky to figure out, but also really interesting (bug 2 will be in a follow up post)

Bug 1: DateTime.ToString()

One of the classes in dasBlog that stores information like Comments, Trackbacks and Pingbacks determines it's filename like so:

public string FileName
{
   get
   {
      return DateUtc.ToString("yyyy-MM-dd") + ".dayextra.xml";
   }
}

Well a few weeks ago Scott Hanselman emailed me with some files with names like 1425-05-05.dayextra. Can you figure out why this is?

Well as I found out, DateTime always assumes Gregorian Calendar, so DateTime.ToString() will output a filename with a Culture.Invariant filename. Well what happened to Scott is that he got comment spammed by some one who's Windows Region was set to a Culture that uses the Hijri Calendar. So, 1425-05-05 is the equivalent of 2004-06-22.

The reason this happened was because this person happened to be the first person to leave a comment, and thus the file was created using the CultureInfo from their machine!

The fix was to do the following:

public string FileName
{
    get
    {
        // Use Invariant Culture, not host culture (or user override),
        // for date formats.
        IFormatProvider mmddFormat = new CultureInfo(String.Empty, false);

        // Ignore local DateFormatInfo (could say CCYY-DD-MM),
        // always use CCYY-MM-DD.
         return DateUtc.ToString("yyyy-MM-dd", mmddFormat) + ".dayfeedback.xml";
   }
}

DateTime stuff is very tricky and dasBlog has 4 different kinds of DateTime so it can get confusing.

DateTime for Server (timezone the server is in)
DateTime for Author (timezone the author is creating the post)
UTC, which is how all data is saved
Browser DateTime (where the reader is reading the post)

In all these cases dasBlog must know how to convert back and forth between all these formats and preserving the window of time that equals a single Day (24 hours of time, in the server, author, or UTC timezone).

Enclosure support in dasBlog

omars — Mon, 03 Jan 2005 13:02:00 GMT

Well, thanks to some encouragement from Jeff Sandquist, I just recently added Enclosure support to dasBlog. It was a bit tricky because RSS 2.0 only allows for a single enclosure per item. I didn't want to get burned by the spec changing at some point, so dasBlog actually support any number of attachments, but only one enclosure. Right now there is no UI for adding anything except for one. Eventually I'll add things like Photos, or whatever.

The nice thing about this is that when you add an attachment they are automatically added in your content/binaries/[entryId]/ folder. So now, when you delete an entry, the folder goes away as well and you don't have any straglers lying around.

If this worked, you should be able to download registry keys that will prevent office documents from opening in IE. Click the enclosure icon below (if you are reading this in an RSS reader it may not display the enclosure link).

Just one of many things coming to a dasBlog near you. Seriosly, Scott just finished setting up our SourceFourge site, and once we give a beta out to a few folks to make sure nuthin is broken, we'll have a public beta release. The Runtime has changed a lot, so we want to be carefull. However, I think Scott fixed a ton of threading bugs in the process (dasBlog wasn't very threadsafe, and this caused lots of wierd stuff).

dasBlog 1.7 progress

omars — Fri, 24 Dec 2004 11:38:00 GMT

Well I plan on spending the next few days getting together a beta of dasBlog 1.7. I've been slowly doing some work on dasBlog the past few months (emphasis on slowly). To be honest I was pretty close to closing up shop on dasBlog and eventually writing something new from scratch, mostly because I felt there was to much to fix and a bunch of things were broken that I didn't feel like dealing with. However, thanks to some inspiration from Scott Hansselman and Clemens, I'm much more excited about continuing to work on the code. In the coming weeks I will get around to moving dasBlog from GotDotNet to SourceForge or some other place. GDN is just way to frustrating to use, and I'm the only person left contributing because of the hassles and issues with using Source Control over there. Dare did the same thing with RSS Bandit and a year later he seems happier for it.

Having said that, version 1.7 has some unbelievable performance improvements. Scott Hansselman did some amazing work under the hood and increased Runtime and RSS performance a great deal. Furthermore he added a Referral Spam Blacklist, a CAPTCHA solution for comments, and an IP address Blocker. I just installed the bits on my site and it's amazing how much faster things are. My ISP should be much happier.

There is another important change in 1.7. Due to a very unfortunate bug that has existed since 1.3 or so, the dayentry.xml and dayextra.xml files were created based on UTC for dayentry and local server time for dayextra. This meant that when dasBlog went to go write a referral or trackback to the dayextra file for a lot of folks that file was the wrong file. In my case since my server is in GMT -5, any attempted writes to this file between midnight GMT and 5 am were lost. I came up with a fairly clever way of fixing it, but this does mean that for users who first install 1.7 there is a one time hit where dasBlog goes and loads all the old files, creates new ones with UTC file names, and then backs up all the old ones. This can take a while if you have a lot of entries and dayextra files.

Anyway, I plan on finishing up some final things and posting a beta to GDN before the new year. If you want something sooner to play with leave me a comment and I'll send you mail when it's ready.

dasBlog feature: Search Result Referrers

omars — Mon, 01 Nov 2004 03:21:00 GMT

I have to admit, one of the more fascinating aspects of having a blog are the ways that people find it. I'm constantly amazed at how people get here. Having a good amount of Google Juice helps put some of my posts pretty high in google's results.

Well because dasBlog just places all referrals together, it's really hard to get a good picture of what search queries people are entering. Since I started doing a little dasBlog development again, I added this new statistics table. This will be in version 1.7.

dasBlog support for Newsgator Ratings

omars — Mon, 01 Nov 2004 02:20:00 GMT

I uploaded a patch for dasBlog 1.6 that allows you to utilize the NewsGator ratings system. This patch adds support for a new PermalinkUrl Macro. You can use this macro by adding:

<%permalinkUrl%> to any of your templates. To add the rating system as I've done on my blog simply edit your itemTemplate to include the following text:

Rate this post: <script src="http://www.newsgator.com/ngs/ratings.aspx?rurl=<%permalinkUrl%>"></script> (data provided from <a href="http://www.newsgator.com">NewsGator Online</a>)

You can download this patch here. Download DasBlog.Web.Core 1.6.4121.2.

dasBlog Security Update

omars — Mon, 06 Sep 2004 03:40:00 GMT

If you are running dasBlog, make sure you install the security fix available for all released versions of dasBlog.

http://www.dasblog.net/documentation/PermaLink.aspx?guid=92ad6eb7-ee40-4c89-9f1e-a07c83859e63

dasBlog 1.6.1 hotfix

omars — Wed, 26 May 2004 14:01:00 GMT

Scott Hanselman and Clemens ran into a nasty dasBlog 1.6 bug. Basically it was possible for your entry cache to get corrupted (not your entries). Thanks to Erv Walter for quickly investigating and fixing the problem.

I have uploaded a hotfix for folks that are running 1.6. It's a single DLL that you drop into your /bin folder. I have also upgraded all the MSI installers and .zip downloads on the workspace. You can download all the releases here.