Over the last few days, I've been cleaning up some things left over from my migration to Movable Type. A lot of it has been driven the desire to eliminate as many of the 404 errors showing up in my server logs. Some of the things I've done:
Added a robots.txt file to my web site. It's a pretty simple file - bots are disallowed from all directories except / and /archives.
Added a custom 404.shtml error page. My web host allows me to also create custom error pages for 400 (Bad request), 401 (Authorization required), 403 (Forbidden), and 500 (Internal server error), but since I haven't seen any of these errors yet in the logs, there doesn't seem to be an urgent need to create them.
Fixed and added some redirects to my .htaccess file to cut down on 404 errors that were still occurring even after they had been redirected to my new domain.
Went through all of my posts to fix internal links to posts and images so they now point at the right path. I also changed the internal links to use relative paths in the hope that any future moves will be less of a hassle in this regard.
Tweaked my category pages so they now have the same layout as my index pages (blogroll, logos, etc.).
Used CPanel's "Index Manager" to disable listing of directories on the webserver other than /, /archives, /gems, and /images.
Migrated my Radio Userland "Stories" into Movable Type. (How I did this will be described in my next post.) Added more redirects to my .htaccess file so requests to view these pages no longer return 404 errors.
One last thing that I thought I wanted to do was to import my comments from HaloScan. I'm not sure any more this would be worth the effort that would be involved.