Archive

Vista: Stop: c000021a {Fatal System Error}, The initial session process or system process terminated unexpectedly.

“Stop: c000021a {Fatal System Error}
The initial session process or system process terminated unexpectedly with a status of 0×000000000 (some more hex codes)”

Can’t repair this. Trying to pinpoint source of problem.

System Restore through Repair Environment is failing. Replacing registry files with those from RegBack hasn’t helped.

Clean install of Vista, moved contents of system32\config from broken install to clean one, also moved old Users directory, and problem still exhibited. Must be a registry/configuration issue then, not system files.

Now going to try to see if I can pin it down to either System registry or Software.

Update: It is fixed. The problem was within the SOFTWARE hive. I moved all the clean install stuff (Windows , Program Files, Users, ProgramData) to a folder called “clean”, and move all the folders out of Windows.old back into C:\ , so in effect returning the machine back to its original state before the clean install. I then replaced SOFTWARE with SOFTWARE.OLD and all is well.

Something within the Software registry hive was wrong/broken. Hope this helps someone. Not sure why the SOFTWARE hive out of RegBack was no good.

At least we know from now on that  troubleshooting “Stop: c000021a {Fatal System Error}, the initial session process or system process terminated unexpectedly.” should be done from within HLKM\Software of the registry, or just replacing the Software hive with a good backup. Earlier in the process I opened regedit from the Repair Environment’s command prompt, and loaded the Software Hive, but it looked bare. There were only Microsoft subkeys, nothing else. Either this is because of the fault, or perhaps it’s a Vista security feature? (no.. it’s not a security feature - see below):

Another update: I have taken the bad software hive, and the good working one (software.old), and loaded them up into Regedit on my XP machine to compare.

Both files are around 45mb, but the bad one is completely bare except for a couple of Microsoft subkeys. I wonder what caused this? Here’s a picture of the two hives - bad-vista and good-vista:

Here the good software hive is compared against the bad one.

Here the good software hive is compared against the bad one.

Perhaps I am barking up the wrong tree. Maybe the HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Reliability\Srt key means “system restore”, and this bare registry is normal during a system restore. Perhaps the registry is supposed to be bare until System Restore finishes after the reboot, and the bare registry wasn’t the actual cause of the stop error. Perhaps the stop error was triggered during System Restore’s finishing up. Who knows. I suppose I could create a restore point on the machine now, and see if System Restore causes the Stop error to return. I might do that.

At least for now, the solution here was to replace software with software.old.

Windows Vista in-place upgrade/repair, on a non booting system.

So, I have a customer’s laptop here, which is broken & won’t boot. I can’t get it to boot in any way shape or form. Startup Repair doesn’t work, System Restore fails, I have manually taken all registry backups from RegBack and put them in \Windows\System32\config. Still, I get the same message:

“Stop: c000021a {Fatal System Error}
The initial session process or system process terminated unexpectedly with a status of 0×000000000 (0xc00000001 0×0010034c)”

This is one of those situations where the good old repair install would fix it. This was known as an “in place upgrade”.

With Vista, an in-place upgrade can only be started from within Windows, which means the system must be bootable.

Also, the System Restore data files are not user accessible (they are VSS diffs or something, rather than just RPxx files) like they were with XP, so that’s two repair processes out the window.

In place upgrades have been a standard repair method for as long as I can remember.

I remember deleting win.com out of the Windows directory and then proceeding to re-run the OEM Win95 setup. (Or was that my trick for upgrading to Windows 95 with an OEM non upgrade disk? I can’t remember -  it’s been a while).

Does anybody remember the “clean install without reformat” technique from the Windows 95 days? You would rip out the HKLM\System\CCS\ENUM, Services and other hardware parts of the registry, then do an in-place upgrade over the top.

All the way from Windows Nothing to Windows XP SP2, in-place upgrades have been the way to “re-install over the top”.

None of this is possible now on Vista because to do an in-place upgrade on Vista requires the system to be working. How’s that for stupidity. You can only repair a working system!

I hate Vista. I wish Microsoft had not hyped it up so much, maybe then they’d let it slip on by like the Millennium Edition that it is.

So, I will install a clean copy, and then pull in the registry files and user’s data from the broken install. If that looks good I’ll go with that, if not I’ll just go clean and move data files back into place.

Outlook & Exchange via RPC/HTTP(s) / Outlook Anywhere / Outlook via Internet & NTLM password saving

All the stuff I see out there, from knowledgeable folk like Daniel Petri, seems to recommend using Basic Authentication over HTTPS for RPC/HTTP. The problem with this is that Outlook will prompt for the user’s password every time, which could be useful in some situations, but it’s a pain in other situations.

The solution to stopping the password prompt is to use NTLM. There is a lot of discussion around people playing with lmcompatibilitylevel in the registry (under HKLM\System\CCS\Control\Lsa), and people talk about it in a hit and miss sort of way, e.g. “this way worked for me, but not that way”, and then somebody else does it a little differently. The consensus comes across that there’s no one way that just works.

Well, for me there is one way that just works. I’ll point out a few gotchas that can get in the way too.

Here’s how I do it. If for some reason this is bad, please let me know!

1. I “connect using SSL”.

2. I do not “mutually authenticate”, so that second box is left blank and greyed out.

3. I always have “on slow networks connect using HTTP first”.

4. I sometimes have “on fast networks connect using HTTP first”, but I configure split-DNS so that if the user is within the office, the Exchange proxy resolves to the internal IP of the RPC proxy. I do this just to test that RPC/HTTP is working. I Ctrl-RightClick the Outlook icon in the system tray and check the connection status to see if we’re working over HTTPS.

5. I set lmcompatibilitylevel to 2.

I use self-signed certificates, so I first browse to https://server, when the certificate warning comes up I view the certificate, go to the last tab and import the certificate. I manually choose where to store the cert and I put it in trusted root certification authorities. If the client is Vista then I “run as administrator” Internet Explorer before doing this.

In SBS 2003, under IIS Manager -> websites -> default website -> RPC or RpcWithCert -> Properties -> Directory Security -> Authentication & Access, “Integrated Windows Authentication” is disabled out of the box, so NTLM doesn’t work until you tick this. This is easy to forget.

In SBS 2003, if you change the server’s IP address and subnet, e.g. from 192.168.0.x to 192.168.1.x or 10.x.x.x, you might want to check “IP address and domain name restrictions” on that same tab in IIS Manager as above. Also do the same for Microsoft-Server-ActiveSync because your smartphones won’t be working.

In SBS 2003 no ports need configuring.

In normal Server 2003 & Exchange 2003, I use “RpcNoFrontEnd” from the Petri.co.il article to configure the ports for me, after I have ticked to enable Exchange for RPC as per Daniel’s instructions.

Be Broadband (http://www.bethere.co.uk)’s Speedtouch routers won’t port-foward HTTPS when configured through the GUI. This is a pain. I’ll do a separate blog entry for how to fix that.

That’s about it. It works every time for me, for different companies with different ISPs. In most cases, the client computer is joined to the domain and the user is logging onto the computer with their domain account, hence there are no popups asking for the password when launching Outlook. If the computer is not joined to the domain, I open up User Accounts in the Control Panel and I click “Manage network passwords”, and I add something like “*.ourdomain.local” and put the password in and the username in the form of either user@domain or domain\user, and also “mail.ourdomain.com” (the outside hostname/IP) and put the credentials in there too. This works fine for XP Pro/Vista Business computers that aren’t part of the domain. For Home Edition of XP or Vista, there is no way to save the password that I know of see this article.

I’d be interested if anybody knows any reasons why the above should not be the preferred way of doing things. I know I should look into getting certificates from a globally trusted CA, as it’s a pain for OWA users with a self-signed cert.

“Publisher cannot complete the operation”, on Windows XP.

Contrary to the Microsoft Knowledgebase article, my encounter with this problem was caused by an overloaded Temp directory on the user’s computer. See How to clear the temp directory.

How to clear the temp directory.

This is such a simple thing that nobody would ever consider writing a blog entry or howto about it, right? The thing is though, I frequently see IT support technicians using Windows Explorer, clicking through into the user’s temp folder, highlighting everything and attempting to delete. They are interrupted (and the process aborted) by a message stating that a particular file was in use and could not be deleted, so they de-select that one file, and try again. Some more files are deleted but once again they are interrupted and told that another file couldn’t be deleted. They de-select that file and try again. This can go on for ages..

Here’s how to do it properly.

Go to a command prompt (Start -> Run -> cmd [enter])

At the command prompt type:

cd %tmp%

and press enter. This will change you into the current user’s temp directory.

Then type:

rd /s .

(that’s rd space slash-S space dot)

then press enter. Don’t forget the . at the end. This means “remove the current directory and all subdirectories, including all files.”

What will happen is that the contents of the temp directory and all subdirectories will be removed, but not the temp directory itself, because you are currently working in that directory (via the CD command) and therefore it can’t be deleted.

You’ll receive access denied messages for all in-use files, plus the temp directory itself. That’s fine. Those access denied messages would have been showstoppers if you were using Windows Explorer.

The above is clearing out the current user’s temp directory, which is located within that user’s profile directory (Documents and Settings\username). To clear out the system temp directory, which is located under the Windows directory, type:

cd %windir%\temp

and press enter. This will change you into the Windows temp directory.

Then just like before, type:

rd /s .

All done! You can type “exit” to close the command prompt.

Overheating Core 2 Quad CPU (Q6600)

I was ripping/encoding some DVDs with Handbrake/GTK, when the CPU temperature monitoring applet started complaining that it couldn’t read the temperature of core 1. I looked at the array of temperature readings on the top panel, and all four cores were between 97 - 100˚C. That’s not good.

dmesg showed

kernel: CPU0: Temperature above threshold, cpu clock throttled (total events = 270544)

(which explains why things started to get real slow while Handbrake was still ripping/encoding)

I didn’t believe it and suspected lm-sensors or whatever monitors the sensors was going screwy, so I took off the computer’s side panel and touched the side of the CPU heatsink. I discovered the computer was telling the truth (it was very hot!).

I had an idea where I wanted to look first…

This is the Intel supplied heatsink. I always buy retail boxed CPUs for the 3x warranty and fans that are built to last. For a short while, some time around the Athlon XP 3000+, AMD were supplying a heatsink with very fine fins which had a tendancy to block up like this. Toshiba laptops are famous for it too.

If you leave your computer running 24/7, and you don’t have some means of monitoring CPU temperature and fan speeds, I suggest you find something. I used to use something called MBM5 on Windows. Gnome has sensors-applet (see screenshot) below.

Screenshot showing sensors-applet running on the top gnome panel.

Screenshot showing sensors-applet running on the top gnome panel.

Recovering from Windows registry hive corruption, the clever way.

I like this trick. Every time I do it, I think about all those people doing repair installs (in-place upgrades).

It works pretty much every time unless the filesystem is really truly screwed, in which case you need a backup, say from the system restore directory (System Volume Information), as per this knowledgebase article (don’t bother with the recovery console though, use your USB to IDE or USB to SATA cable and fix it from your laptop.)

Here are the symptoms. You try to start up your Windows 2000/XP (Vista too?) computer and you get a message, white text on black background:

Windows could not start because the following file is missing or corrupt: \WINDOWS\SYSTEM32\CONFIG\SYSTEM

or

Windows could not start because the following file is missing or corrupt: \WINDOWS\SYSTEM32\CONFIG\SOFTWARE

Sometimes, the message is cut short, so you might see “\WINDOWS\SYSTEM32\CONFIG\SYS” or similar. Hint: If it’s really cut short, and you can’t see if it’s SOFTWARE or SYSTEM, do the following procedure on both files. Whichever one is identified as having been repaired, well that’s the one that was broken ;)

Anyway, how to fix it in 2 minutes:

Use your USB to IDE/SATA adapter cable, and connect the broken machine’s hard drive to your laptop, or your spare PC or whatever. You don’t have to use a USB to IDE/SATA adapter cable - if you’re a person at home with another PC you can stick the drive on a spare IDE or SATA channel. You just need to get that hard drive into a working Windows XP computer for a few minutes.

Windows will mount the broken computer’s hard drive as, say E: or F:. Make sure you have your computer set to show hidden files and also system files. To check this, go into My Computer -> Tools -> Folder Options, -> View Tab, and select “Show hidden files”, and make sure “Hide protected operating system files” is not ticked.

First things first, run chkdsk on that drive, after all it is most likely filesystem corruption that has caused the registry to become corrupt in the first place. In My Computer, right-click the broken computer’s drive and choose properties. Go to tools, “Check Now”, put a tick in only the first box (Automatically fix filesystem errors), and click start. Let that finish before continuing.

Here’s where the magic happens. Go to start -> run, and type regedit [enter]. This will launch the registry editor on your computer. In the registry editor, highlight HKEY_LOCAL_MACHINE, and then go to File -> Load Hive. Find the file that is “missing or corrupt” (from your error message earlier), and choose to load that. It will be in E:\(or F:\)Windows\System32\Config, and will be called just SOFTWARE or SYSTEM. Regedit will ask you to name the hive, just type “badpc” (any old garbage will do - it’s only temporary).

Regedit will say “One or more files containing the registry were corrupt and had to be recovered by use of log files. The recovery was successful.” You have just repaired the registry! Now you need to Un-load that hive, so highlight that “badpc” hive that you can now see under HKEY_LOCAL_MACHINE, and go to File -> Unload Hive.

You now just need to put that hard drive back in the broken computer, which hopefully won’t be broken any more! If you used a USB to SATA or USB to IDE cable from your laptop, make sure you use the “Safely remove hardware” icon in the system tray next to the clock to safely remove the hard drive, else you may cause filesystem corruption again. Alternatively just shut your laptop/working computer down properly and remove the hard drive once it’s shut down.

All done.

Some background:

The registry is a database. It has transaction log files which can be used to recover from corruption. It would appear that the early Windows boot process is not able to work with those log files, but regedit (and Windows itself further on in the boot process) is.

Dell’s new(ish) PowerEdge T300 server

I thought I’d take some photos and do a brief overview of the Dell PowerEdge T300 server.

I have supplied and installed a few of these to my customers, and I think it is my favourite small business machine for now.

This is a single-socket (i.e. single CPU) machine, although that single processor is a Quad Core Xeon, so it’s not such a negative point really. I have found in the past that I have gone for dual-socket servers, because they are naturally reasonably high-end rather than being more like a basic desktop, but these servers have always been left with the second CPU socket empty, and by the time you might think about adding a second CPU, those CPUs are long gone off the shelves and the price/performance of a replacement machine makes replacing the whole machine much more viable. So I am happy that this a single-socket machine - that’s perfect for the small businesses that I deal with.

For less than £600 +vat, this machine comes with a 2.5GHz Quad-Core Xeon CPU, 4gb RAM, a SAS 6iR RAID-0/1 controller card, and 2×160gb SATA hard disks configured as a RAID1.

What I do then is head off to http://www.scan.co.uk or http://www.microdirect.co.uk and buy a couple of large capacity hard drives. The machine comes cabled up ready to take up to four drives on the SAS 6i/R card. The SAS 6iR can create multiple RAID1 (or RAID0 if you’re that way inclined) volumes. So all we do is drop in a couple of 1,000gb Seagate HDDs, and create a large, cheap, mirrored storage volume.

The machine only has a single half-height 5.25″ drive bay free. There are two bays in total but the first is taken up by the DVD-ROM drive. This limits your choice of tape backup drive. The backup choices from Dell are even more limited - last time I checked they were only offering those cartridged 2.5″ SATA hard drive things. What I do is head over to Scan again, and pick up a Freecom DLT-V4 half-height 160/320gb SATA DLT drive. This completes my current favourite small business server. The Freecom drive uses SATA power and SATA data, and slots perfectly into the PowerEdge T300 like it was designed to be there in the first place. The drivers that Windows Update offers for the SATA DLT do not work properly, but Quantum’s website has working ones.

I recently did a setup like this (running SBS 2003) for a chap who works from home. In his case I used an internal Freecom 36/72gb USB DAT drive for reasons of cost and lower noise (the server was next to his bedroom). Unfortunately the PowerEdge T300 has no molex power connectors at all, only SATA power. Therefore a SATA Power -> Molex adapter is required, which is precisely the opposite of what you will have lying around. These adaptors are available for a few pounds online though. The Freecom USB DAT drive comes with a USB B to motherboard header (7/8 pin) cable. Here comes the second gotcha of the PowerEdge T300 - no internal USB headers, however, there is a USB A port on the motherboard, so you use a regular USB A->B cable but inside the server, as strange as that seems (see photo of internal USB A port on the motherboard).

The PowerEdge T300 comes with dual gigabit ethernet ports as standard, provided by a Broadcom chipset.

There are three PCI-E x8 slots, one PCI-E x4 slot, and one full length PCI-X slot which I assume would also accommodate a regular 32-bit/33MHz PCI card such as a modem, ISDN adapter or WiFi card for example. You can’t see the PCI-X slot in the pictures because it’s right below the SAS RAID card.

There are six SATA 3Gbps connectors built onto the motherboard, in case you do not opt for the SAS 6iR RAID 0/1 card or a full-on PERC 6 RAID-everything card. One of the onboard SATA ports is taken by the DVD-ROM drive.

After fitting the extra hard drives as in the pictures (four non hot-plug SATA drives total), there is one spare SATA power connector which can be used for the tape drive or whatever you fancy.

There are six DDR2 DIMM slots.

The front LCD display panel can display text of your choice, for example the company name and telephone number in case the item is stolen and ends up in a responsible person’s hands.

The server is available with the option of Hot-Plug/pullout hard disks. I have not encountered this configuration though. It also looks like the machine has the option of dual/redundant power supplies, since the rear is labelled “1″ and “2″.

As usual with Dell’s servers, everything is extremely well put together and designed. There are no little hairdrier chipset or CPU fans, just large slow-turning fans and lots of properly designed ducting panels to allow good proper airflow. This is what I like most about Dell’s servers, for this kind of money everything is spot on.

Some pictures. This is my first attempt at a blog so I apologise for rambling on and not laying things out in a clear and conscise manner. I may come back to do some editing later :)