Scott Hanselman

We're calling it gold, it is DasBlog 1.8.5223.0.

What are you doing this weekend? Why not install DasBlog 1.8? (as always back up your data and READ THE README.RTF!)

Update: Since few folks read readmes, here's what you need to know:

• NEW INSTALL
• If you're a new install, get the Web-Files.zip, unzip it, and assuming you've got IIS and ASP.NET on your system run CreateDasBlogVdir.vbs and you should be set. Be sure to log in admin/admin and look at ALL the configuration options, changing them from the defaults. Also edit sitesecurity.config with the user(s) and passwords you want. Read the Readme anyway.
• UPGRADES
• Unzip Web-Files.zip locally.
• If you're running DasBlog locally, you have the \content folder local. If you're remote, bring your \content folder down to your local machine.
• Upgrader
• Referrer spam. Open DaBlogUpgrader.exe.config and add any words or domainnames that you want stripped from your dasBlog comments and referrer XML files. Your content and posts will never be touched.
• Run DasBlogUpgrader from a command line like this: DasBlogUpgrader "c:\whatever directory\dasblog\whateverweb\content"
• NOTE: You can run the upgrader even if you've already upgraded. It's a multiple use thing.
• It will as you all sorts of questions. For the most part, answering yes to all is safe.
• One of the options just strips all referrals (not trackbacks) out of your XML file. If you're running a site with any reasonable amount of traffic you'll see a nice performance boost if you do this. Omar and I recommend that you just give up on referrals. They are a hassle, they grow to quickly and provide little more than a bloated site.
• Uploading
• After running the upgrader, we recommend that you use a merge tool or your eyes to merge the differences between your web.config and the one that ships with Web-Files.zip. BeyondCompare is a nice diff tool. (Or read the readme as we tell you which ones in there!)
• Upload all the ASCX, ASMX and ASPX files to your site and the entire contents of the newtelligence.DasBlog.Web\bin folder and all subfolders to your site, overwriting ALL remote files.
• Upload the new \DatePicker folder and the \ftb folder and all subfolders
• Upload all of \themes (if you like) and all of \smilies and \images.
• Upload your new merged web.config over your old one
• Upload your new merged site.config over your old one
• If you like, add PingServices.xml, NavigatorLinks.xml, blockedips.config and blacklist.txt.
• While you're there...
• While you're connected to your remote system, consider cleaning out your \logs folder of .log files and .zip files. If your blog has been up a while, you've likely got schmutz up there, so why not clean up?
• Troubleshooting
• 93% of problems with DasBlog are web.config files being wrong or wrong directory permissions. That means, bad XML, wrong assembly namespaces, missing sections, missing HTTP Handlers, or incorrect ACLs, etc. Always compare your web.config with the one that shipped and you'll get back on track. Read the readme.
• 1% of problems with DasBlog are having old versions of assemblies mixed in with new assemblies. Know your versions.
• 3% of problems with DasBlog are Clemens' fault for writing the first version back in the day while hopped up on German Beer and Ciggies.
• 2% of problems with DasBlog are my fault, usually from lack of testing.
• 1% of problems are entirely Omar's fault, usually from lack of sleep.

Also, to the one guy who donated $50 to the project, THANKS! That was awesome. We'll use the money to get a pro license to FTB or to have a theme professionally designed.

New Features of Note

• Anti-Spam Features
  • Automatic Referral and Trackback blacklist update
  • CAPTCHA for non-admin users (Font warping has also been increased in this version)
  • Logging and display of Comment IP addresses and resolved Hostnames for Admins
  • DasBlogUpgrader can strip spam from existing content folders
  • Support for rel="nofollow"
  • Ability to delete referrals and trackbacks directly from the Admin UI
• Security Features
  • HttpOnly cookies
  • Admin access auditing
  • SMTP Authentication for outgoing mail
• Syndication Features
  • Improved RSS Comments support for SharpReader and RSS Bandit
  • Upgraded Atom support from 0.3 to Valid Atom 1.0. ATOM Syndication permalink changes but 301 is issued.
  • RSS 2.0 and Atom 1.0 validates via FeedValidator.
  • Ability to mark entries as "syndicated" or not. Entries can appear on the site but not in RSS/Atom.
  • New between RC1 and Gold: Plugable, configurable pinging of Blog Search engines like pubsub and technorati.
• Performance Features
• Search Highlighting is optional now
• Referrals are logged but not stored in XML by default. Configurable.
  (This has huge performance benefits for high traffic sites.)
• DasBlog Upgrader can optionally remove all referrals.
  (Again with high traffic sites some folks had 5meg XML files full of referrals)
• Theme templates are now cached in memory.
• Installation Features
• New VBS for IIS permissions and VDir creation
• Support for running under ASP.NET 2.0
• Support for running on Win2k 2003 without changing permissions when impersonation is enabled
• Content Features
• Ability to pre- and post-date entries
• Permalinks based on Title and Date optional: 2005/06/06/title.aspx
• Latest build of Free Text Box including ability to upgrade FTB without upgrading DasBlog.
• Text Editor (FTB) supports IE7
• Text Editor (FTB) supports FireFox
• Blog Statistics macro
• Mail-To-Weblog continues to improves. Works with Thunderbird.
• Extensibility Features
  • Custom Macro Plugin model without recompiling DasBlog (see the source for the example custom macro)
• Theme Features
• DasBlog now ships with 16 themes and a Theme Combo to change between them.
• New theme.manifest file makes themes and image assets more portable.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By

UPDATE: Ok, THAT was wildly successful. So successful that now my site is slow because of it! :)

I've swapped out the IFRAME and now you can just see my guestmap here. This picture is a static shot of the map this morning at 9am.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By

Ah, yes crazy bugs, they are my life. Here's today's saga. We did this from 9:30am until lunch, so we were able to figure it out in about two and half hours.

One of our systems retrieves Check Images (pictures of cleared checks). The Checks move through the system as Base64'ed strings and are eventually the separate front and back checks are displayed in the user's browser as a single image using a dynamic compositing technique I mentioned a while back.

However, it seemed that when we took the decoded from BASE64 schmutz and did basically this to convert the GIF to a JPEG:

using(MemoryStream m = new MemoryStream(bytes))

{

    using (System.Drawing.Image image = System.Drawing.Image.FromStream(m))

    {

        Response.ContentType = "image/jpeg";

        image.Save(Response.OutputStream,ImageFormat.Jpeg);    

    }

}

We'd get an error from the bowels of System.Drawing that there was an "invalid parameter." Reflectoring showed that Image.FromStream is managed spackle over a GDI+ method.

[DllImport("gdiplus.dll", CharSet=CharSet.Unicode, ExactSpelling=true)]
internal static extern int GdipLoadImageFromStreamICM(UnsafeNativeMethods.IStream stream, out IntPtr image);

I tried loading it into a number of picture viewers, most of which said nope. Surprisingly, IE didn't have a problem with it. This is odd to me because I thought the GDI+ security fixes would apply to IE, but not so.  

To review - I've got a weird GIF that shows up in IE, but that .NET and GDI+ refuse to recognize. I could look for other image libraries that would "clean" the GIFs but that's reaching. The mainframe/host system that generates and holds these GIF isn't likely to change, and even if it did it wouldn't be fast enough for this implementation.

We could just pass it all the way through the system unmolested as the GIF that it is. This would WORK but only until browsers like IE became more security aware and started slapping down invalid GIFs like this one.

So, these GIFs are invalid. But how? As with all things for me, I begin with Notepad2. I opened a bad example check image into Notepad2:

First I notice that it's a GIF87a. Noteworthy only like an old piece of gray paper from Kindergarten is noteworthy. Then we (Patrick and I - at this point I've drafted him) notice that the alphabet and numbers appears a hundred bytes in. We figured that's the color table as they are triplets and this is a grayscale gif of 128 colors. But, without getting all 0xHex-y this early on, what else can I do to determine if this is a valid GIF or not? Well, I got it to display in IE before. I'll copy it (now a bitmap) to the clipboard and save it as a GIF. It'll likely save as a GIF89 because, hey, it's like 2 better, right?

Here's the same graphic saved again. Ya, it looks totally different, so you assume my copy/paste was an invalid thing to do (in the scientific method sense). Well, hang in there, it gets worse. It's clearly a GIF89a and it clearly has a different color table. Otherwise, nothing here jumps out when comparing them with our eyes.

At this point, it's time to bite the bullet and decode the GIF header. We figure a GIF can be corrupt in two ways, either the header is bogus or the image data is. We'll do the easy one first. Time to pull out the June 15th, 1987 GIF spec from Compuserve.

Working structure by structure we produced this little nugget of uselessness:

    using (FileStream f = File.Open(@"C:\Documents and Settings\shanselm\Desktop\bad.gif",FileMode.Open))

    //using (FileStream f = File.Open(@"C:\Documents and Settings\shanselm\Desktop\good.gif",FileMode.Open))

    {

        using(BinaryReader reader = new BinaryReader(f))

        {

            string sigversion = new string(reader.ReadChars(6));

            if (sigversion.StartsWith("GIF"))

            {

                ushort width = reader.ReadUInt16();

                ushort height = reader.ReadUInt16();

                byte someshit = reader.ReadByte();

                int colortable = someshit & 0x7;

                byte bgcolor = reader.ReadByte();

                byte apsectratio = reader.ReadByte();

                int logicallength = (int)Math.Pow(2,colortable+1);

                int colortablelength = (int)(3 * logicallength);

                //Color table, yuck. RGB is a struct elsewhere in our file.
                // It's a value type, that's why we poke it back in at the bottom of the loop.

                RGB[] RGBs = new RGB[logicallength];

                for (int i = 0; i < logicallength; i++)

                {

                    RGB rgb = RGBs[i];

                    rgb.R = reader.ReadByte();

                    rgb.G = reader.ReadByte();

                    rgb.B = reader.ReadByte();

                    RGBs[i] = rgb;

                }

                //Image Descriptor

                byte imageseparator = reader.ReadByte();

                uint leftpos = reader.ReadUInt16();

                uint toppos = reader.ReadUInt16();

                uint widthagain = reader.ReadUInt16();

                uint heightagain = reader.ReadUInt16();

                byte localcolortableflags = reader.ReadByte();

                int localcolortablepresent = localcolortableflags & 0x80;

                int interlace = localcolortableflags & 0x40;

                int sortbit = localcolortableflags & 0x20;

                int localcolortable = localcolortableflags & 0x07;

                //We figured if the header was bad we'd mess with it in this process somewhere

                // then if we fixed it in the byte[], we'd fall through to the code below
                // that previously hadn't worked. If Image.FromStream did work, we'd have fixed the bug
                // Of course, we got all the way here and there wasn't anything wrong with the GIF header!            
            }

        }

        using (MemoryStream m = new MemoryStream(bytes))

        {

            using (Image image = Image.FromStream(m))

            {

                image.Save("foo.jpg",ImageFormat.Jpeg);

            }                    

        }   

Well, crap. We made it all this way and there didn't appear to be ANYTHING (per spec) wrong with the GIF header. We checked everything out in the Watch Window line by line. Nothing.

Ok, back to differences. How about checking them out in Beyond Compare?

Zoom in on that baby. Look real close. Notice in the upper left corner, there's not many differences. Remember that the old GIF87 is on the left, and the new one that I made via COPY/PASTE is on the right. The basic image data is the same, cool. So, really the only differences are the header, a byte or two in the middle, and what? What's that at the VERY BOTTOM RIGHT CORNER? A semicolon? In the valid image? WTF is that?

Hm...back to the spec. Since we've just decoded the header, perhaps there's a footer/trailer/terminator.

June 15, 1987

(c) CompuServe Incorporated, 1987

 GIF TERMINATOR

In order to provide a synchronization for the termination of a GIF

   image  file,  a  GIF  decoder  will process the end of GIF mode when the

   character 0x3B hex or ';' is found after an image  has  been  processed.

   By  convention  the decoding software will pause and wait for an action

Lovely. Do we have an off-by-one? Are we dropping the last byte as we go through the system?

We go back to the system that sites just ahead of the mainframe check imager and request an image. We look at the byte array returned, and notice that the LAST BYTE IS MISSING. The images are trasmitted on a secure internal network using HTTP. The Content/Type is image/gif and the Content-Length HTTP Header, in this case, was 20814. That was exactly how many bytes were received.

So here's the question (that hasn't been answered):

Is it more likely that the host system has or is generating bogus/bad/invalid GIFs or that the Content-Length HTTP Header being returned by their unknown kind of Web Server is off by one and System.Net.HttpWebRequest is trusting what it's being told? I vote bogus GIFs, Patrick thinks bad Content-Length. Not sure if we'll ever know.

The fix, of course, was to check if the byte array representing this kind of GIF is terminated with 0x3b or not, and if not, append it. Once 0x3b was appended, System.Drawing and GDI+ had NO problem with the bytes.

Crisis averted. Chao continues.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By

FYI, we were going to release DasBlog 1.8 Gold today, but SourceForge CVS is down.  My bad for not checking the schedule. We'll release as soon as it opens up.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By

Forgetting for the moment that they are the same company, I've just officially moved off Newsgator 2.5 and onto FeedDemon.

Bottom Line: I couldn't handle the Physic Weight of the bold unread [9689] messages in my Aggregrated News folder.

It was making me ill, and ever since I got deeply GTD and reached Zero Email Bounce (ZEB) nirvana in Outlook, I just couldn't have thousands of emails sitting there.

That said, FeedDemon is great - Highly Recommended. I've got it on my work laptop, my home PC and my home laptop. And finally, something that never worked even with (gasp) Newsgator 2.5, items, feeds, and read-status is sync'ed between machines. Happy Day.

I've got three folders, Friends, Tech, and News, and hundreds of feeds. However, it's easy to read them with Newspaper view. It's easy to save stuff with the News Bin. It's easy to watch for keywords with Watches.

Most wanted Feed Demon Feature: RSS Comments Support

I'm still using iTunes for my Podcasting. Haven't quite figured out why I'd use FeedStation. It's not like I'm going to care about any surprise enclosures that may show up. Audio that I want to listen to is easily added to iTunes. We shall see.

Now playing: Kaysha - La Zouk Hooray

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By