I'm just a simple property list, I didn't expect the Spanish strinquisition! (19 Jul 2007)

My co-worker looked a little tense. Our office is in the sixth floor, my window was wide open, and somehow I became slightly nervous as he walked up to it.

"Now, you're the Lisp aficionado here, right", he said, "You've got to help me out: Strings don't work in property lists!"

Oh, great. Who knows, being regarded (undeservedly) as the local Lisp, ahem, expert may become a factor for my job security some day, so I thought I'd better keep a straight face. Besides, he was still standing close to that window, and I wanted to leave nothing but a reassuring impression on him.

On the other hand, what the heck was he talking about?

Frantically grepping my grey cells for information on property lists, I somehow recalled we sometimes use them as a poor man's hashtable, usually mapping keywords to flags. But it had been so long I used property lists myself that I even had to look up the syntax details. To avoid this embarrassment next time around, here are some notes.

A property list is associated with a symbol. This flat and unstructured list can be thought of as a sequence of indicator/value pairs, with the indicator being the "key", in hash map terms. So the list starts with an indicator, followed by a value, followed by an indicator and its value, and so on. This is how you usually set a symbol property:

  (setf (get 'some-symbol some-indicator) some-value)

And to inquire a symbol property, you just say something like (get 'some-symbol some-indicator).

some-indicator can basically be any type, and so I wasn't sure what my co-worker meant when he said that he couldn't get strings to work, until he explained the details to me: He was calling some Lisp-based API function in our product, and that function returns a property list. Unfortunately, that property list was special in that somebody had stuffed a string into it as an indicator, and so the property list looked somehow like this:

  ("foo" 42 "bar" 4711)

And indeed, if you now try to inquire the "foo" property using (get 'some-symbol "foo"), all you get is - nil.

To retrieve a property value, get walks the list and compares each indicator in the list with "foo" (in this example) - using eq. From which we can immediately conclude:

• The correct spelling of "property list" is p-e-r-f-o-r-m-a-n-c-e p-r-o-b-l-e-m, as each lookup requires traversing potentially all of the list.
• eq checks for object equality, not just value equality. Which means that things like literal (!) strings or characters cannot be indicators!

In our case, we say (get 'some-symbol "foo"), and that "foo" string literal creates a new string object. While that new object happens to have the same value as the "foo" string in the property list, it is not the same object.

Indeed, the Common Lisp HyperSpec is quite clear on that topic: "Numbers and characters are not recommended for use as indicators in portable code since get tests with eq rather than eql, and consequently the effect of using such indicators is implementation-dependent."

It all boils down to the simple fact that (eq "foo" "foo") returns nil.

Now hopefully we can fix the API which returned those inadequate property lists to my co-worker's code, but his code also needs to run in older and current installations, and so he needed a workaround of some sort.

His first idea was to get the property list and fix it up in a preprocessing step before using get or getf for lookup, i.e. something like this:

(defun fix-plist(plist old-indicator new-indicator)
  (let ((cnt 0))
    (mapcar 
      #'(lambda(item)
          (incf cnt)
          (if (and (oddp cnt) (equal item old-indicator))
              new-indicator item))
      plist)))

(setf my-symbol 42)
(setf (get 'my-symbol "indicator") "value") ;; mess up plist
(print (get 'my-symbol "indicator"))        ;; returns NIL
(print (getf (fix-plist (symbol-plist 'my-symbol) "indicator" :indicator) :indicator))

This works, kind of - but it is actually quite ugly. Sure, with this code, we should be able to safely move ahead, especially since I also closed that office window in the meantime, but still: I really hope I'm missing something here. Any other ideas out there?

---

Elementary, my dear Watson! (03 Jul 2007)

When an application crashes under Windows Vista, it will contact Microsoft's Windows Error Reporting servers; only if those servers request more data about the problem, crashdump files will be generated. Bad luck if you're a developer talking to an enraged customer who just lost his data, and you first have to go through a lengthy process of registering your application on Winqual, mapping the right application versions, and waiting until the first customer crashdumps show up on the server. Chances are that your customer may have taken his money elsewhere in the meantime while waiting for you to fix his problem.

In Think globally, dump locally, I listed the following workarounds:

• Using the ForceQueue registry entry to force WER to always produce a crashdump (at the price of losing UI interaction)
• Disable the Internet connection before the crash occurs
• (Ab-)Using the CorporateWERServer registry entry
• Deploy Microsoft Operations Manager 2007, including Agentless Exception Monitoring (which replaces the older "Corporate Error Reporting" servers)
• Copy Dr. Watson from XP to the Vista system and install it as system JIT debugger
• Install a top-level crash filter in your application using SetUnhandledExceptionFilter

In Don't dump Vista just yet..., I presented a little-known Task Manager option which creates a user dump file from any running process.

And in this installment of the seemingly never-ending series on Vista crashdumps, we'll explore yet another option. What if we had a system JIT debugger which can be easily installed on a customer system and automatically produces minidump files for any crashing application? Basically a homebrew version of good ol' Dr. Watson, stripped down to bare essentials?

The following code illustrates this approach. This skeleton application is called mydearwatson and installs as a JIT debugger. When an app crashes, it attaches to it and asks the user what kind of minidump (normal or full dump) should be generated. The resulting crashdump file goes to the current user's desktop folder, readily available for sending it off to the developer of the application.

// mydearwatson
//
// Minimal JIT debugger; attaches to a crashing process 
// and generates minidump information. Intended to be used
// as the rough skeleton for a poor man's Dr. Watson 
// replacement on Vista.
//
// Written by Claus Brod, http://www.clausbrod.de/Blog

#include <windows.h>
#include <DbgHelp.h>
#pragma comment(lib, "DbgHelp.lib")
#include <Psapi.h>
#include <shlobj.h>
#include <atlbase.h>

#include <stdio.h>
#include <string.h>

#define MSGHDR "mydearwatson\n" \
               "(C) 2007 Claus Brod, http://www.clausbrod.de/Blog\n\n"

bool uninstall(void)
{
  CRegKey key;
  if (ERROR_SUCCESS != key.Open(HKEY_LOCAL_MACHINE,
    "Software\\Microsoft\\Windows NT\\CurrentVersion\\AeDebug\\",
    KEY_READ | KEY_WRITE))
    return false;

  // check for old debugger registration
  char debuggerCommandLine[MAX_PATH+256];
  ULONG nChars = _countof(debuggerCommandLine);
  LONG ret = key.QueryStringValue("PreMydearwatsonDebugger",
    debuggerCommandLine, &nChars);
  if (ret == ERROR_SUCCESS) {
    ret = key.SetStringValue("Debugger", debuggerCommandLine);
    if (ret == ERROR_SUCCESS) {
      ret = key.DeleteValue("PreMydearwatsonDebugger");
    }
  }

  return ret == ERROR_SUCCESS;
}

bool install(char *programName)
{
  CRegKey key;
  if (ERROR_SUCCESS != key.Open(HKEY_LOCAL_MACHINE,
    "Software\\Microsoft\\Windows NT\\CurrentVersion\\AeDebug\\",
    KEY_READ | KEY_WRITE))
    return false;

  char debuggerCommandLine[MAX_PATH+256];
  ULONG nChars = _countof(debuggerCommandLine);
  if (ERROR_SUCCESS == key.QueryStringValue("Debugger", debuggerCommandLine, &nChars)) {
    _strlwr_s(debuggerCommandLine, _countof(debuggerCommandLine));
    if (!strstr(debuggerCommandLine, "mydearwatson")) {
      // save command line for previously installed debugger
      key.SetStringValue("PreMydearwatsonDebugger", debuggerCommandLine);
    }
  }

  char debuggerPath[MAX_PATH];
  strcpy_s(debuggerPath, programName);  // preset with default
  ::GetModuleFileName(GetModuleHandle(0), debuggerPath, _countof(debuggerPath));

  _snprintf_s(debuggerCommandLine, _countof(debuggerCommandLine), _TRUNCATE,
    "\"%s\" -p %%ld -e %%ld", debuggerPath);
  return ERROR_SUCCESS == key.SetStringValue("Debugger", debuggerCommandLine);
}

char *getMinidumpPath(void)
{
  static char dumpPath[MAX_PATH];
  if (dumpPath[0] == 0) {
    SHGetSpecialFolderPath(NULL, dumpPath, CSIDL_DESKTOPDIRECTORY, FALSE);
  }
  return dumpPath;
}

char *getMinidumpFilename(DWORD pid)
{
  static char minidumpFilename[MAX_PATH];
  if (!minidumpFilename[0]) {
    _snprintf_s(minidumpFilename, MAX_PATH, _TRUNCATE,
      "%s\\mydearwatson_pid%d.mdmp", getMinidumpPath(), pid);
  }
  return minidumpFilename;
}

bool dumpHelper(HANDLE hDumpFile,
                HANDLE processHandle, DWORD pid,
                HANDLE threadHandle, DWORD tid,
                EXCEPTION_RECORD *exc_record, MINIDUMP_TYPE miniDumpType)
{
  bool ret = false;

  CONTEXT threadContext;
  threadContext.ContextFlags = CONTEXT_ALL;
  if (::GetThreadContext(threadHandle, &threadContext)) {
    __try {
      MINIDUMP_EXCEPTION_INFORMATION exceptionInfo;
      exceptionInfo.ThreadId = tid;
      EXCEPTION_POINTERS exc_ptr;
      exc_ptr.ExceptionRecord = exc_record;
      exc_ptr.ContextRecord = &threadContext;
      exceptionInfo.ExceptionPointers = &exc_ptr;
      exceptionInfo.ClientPointers = FALSE;

      if (MiniDumpWriteDump(processHandle,
        pid, hDumpFile, miniDumpType, &exceptionInfo, NULL, NULL))
        ret = true;
    } __except(EXCEPTION_EXECUTE_HANDLER) { }
  }

  return ret;
}

struct HandleOnStack
{
  HANDLE m_h;
  HandleOnStack(HANDLE h) : m_h(h) { }
  ~HandleOnStack() { if (m_h && m_h != INVALID_HANDLE_VALUE) CloseHandle(m_h); }
  operator HANDLE() { return m_h; }
};

bool createMinidump(HANDLE processHandle, DWORD pid, DWORD tid,
                    EXCEPTION_RECORD *exc_record)
{
  HandleOnStack hDumpFile(CreateFile(getMinidumpFilename(pid), GENERIC_WRITE, 0, NULL,
    CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL));
  if (hDumpFile == INVALID_HANDLE_VALUE)
    return false;

  HandleOnStack
    threadHandle(OpenThread(THREAD_GET_CONTEXT | THREAD_SUSPEND_RESUME, 0, tid));
  if (!threadHandle)
    return false;

  if (-1 == SuspendThread(threadHandle))
    return false;

  MINIDUMP_TYPE miniDumpType = MiniDumpNormal;
  if (IDYES == MessageBox(NULL, MSGHDR
    "By default, minimal crashdump information is generated.\n"
    "Do you want full crashdump information instead?",
    "mydearwatson", MB_YESNO|MB_ICONQUESTION|MB_DEFBUTTON2)) {
      miniDumpType = MiniDumpWithFullMemory;
  }

  bool ret = dumpHelper(hDumpFile, processHandle, pid,
    threadHandle, tid, exc_record, miniDumpType);
  if (ret) {
    char buf[1024];
    _snprintf_s(buf, _countof(buf), _TRUNCATE, MSGHDR
      "Minidump information has been written to\n%s.\n",
      getMinidumpFilename(pid));
    MessageBox(NULL, buf, "mydearwatson", MB_OK|MB_ICONINFORMATION);
  }

  ResumeThread(threadHandle);
  return ret;
}

int debuggerLoop(DWORD pid, HANDLE eventHandle)
{
  // attach to debuggee
  if (!DebugActiveProcess(pid)) {
    fprintf(stderr, "Could not attach to process %d\n", pid);
    return 1;
  }

  HANDLE processHandle = 0;
  while (1) {
    DEBUG_EVENT de;
    if (WaitForDebugEvent(&de, INFINITE)) {
      switch(de.dwDebugEventCode)
      {
      case CREATE_PROCESS_DEBUG_EVENT:
        processHandle = de.u.CreateProcessInfo.hProcess;
        printf("Attaching to process %x...\n", processHandle);
        break;

      case EXCEPTION_DEBUG_EVENT:
        printf("Exception reported: code=%x, dwFirstChance=%d\n",
          de.u.Exception.ExceptionRecord.ExceptionCode, de.u.Exception.dwFirstChance);

        if (de.u.Exception.ExceptionRecord.ExceptionCode == EXCEPTION_BREAKPOINT) {
          SetEvent(eventHandle);
        } else {
          createMinidump(processHandle, de.dwProcessId, de.dwThreadId,
            &de.u.Exception.ExceptionRecord);
          ContinueDebugEvent(de.dwProcessId, de.dwThreadId,
            DBG_EXCEPTION_NOT_HANDLED); // required?
          DebugActiveProcessStop(pid);
          printf("Detached from process, terminating debugger...\n");
          return 0;
        }
        break;

      default:
        // printf("debug event code = %d\n", de.dwDebugEventCode);
        break;
      }

      ContinueDebugEvent(de.dwProcessId, de.dwThreadId, DBG_CONTINUE);
    }
  } // while (1)

  return 1;
}


void usage(char *programName)
{
  fprintf(stderr, "To install as JIT debugger:\n");
  fprintf(stderr, "  %s -i\n", programName);
  fprintf(stderr, "  %s\n", programName);
  fprintf(stderr, "To uninstall:\n");
  fprintf(stderr, "  %s -u\n", programName);
  fprintf(stderr, "Call as JIT debugger:\n");
  fprintf(stderr, "  %s -p pid -e eventhandle\n", programName);
}

int main(int argc, char *argv[])
{
  DWORD pid = 0;
  HANDLE eventHandle = (HANDLE)0;
  bool uninstallationMode = false;
  bool installationMode = false;

  if (argc == 1) {
    if (IDYES == MessageBox(NULL, MSGHDR
      "Do you want to install mydearwatson as the system JIT debugger?",
      "mydearwatson", MB_YESNO|MB_ICONQUESTION)) {
        installationMode = true;
    }
  }

  for (int i=1; i<argc; i++) {
    if (!_stricmp(argv[i], "-p")) {
      pid = atol(argv[i+1]);
    }
    if (!_stricmp(argv[i], "-e")) {
      eventHandle = (HANDLE)atol(argv[i+1]);
    }
    if (!_stricmp(argv[i], "-i")) {
      installationMode = true;
      uninstallationMode = false;
    }
    if (!_stricmp(argv[i], "-u")) {
      uninstallationMode = true;
      installationMode = false;
    }
  }

  if (installationMode) {
    if (!install(argv[0])) {
      fprintf(stderr, "Could not register as a JIT debugger.\n");
      return 1;
    }
    return 0;
  }

  if (uninstallationMode) {
    if (!uninstall()) {
      fprintf(stderr, "Could not uninstall.\n");
      return 1;
    }
    return 0;
  }

  if (!pid || !eventHandle) {
    usage(argv[0]);
    return 2;
  }

  return debuggerLoop(pid, eventHandle);
}

To compile and build this code, open a Visual Studio command prompt and enter

  cl mydearwatson.cpp

To install mydearwatson, run the executable in elevated mode and confirm the installation message box. Now configure Windows Error Reporting to always ask the user before submitting crash data: "Problem Reports and Solutions/Change Settings/Ask me to check if a problem occurs". Alternatively, create a registry value called Auto (REG_SZ) in HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\AeDebug and set its value to "1".

The next time an application crashes, the usual WER dialog will appear; click the "Debug" option in that dialog. Another message box will be displayed asking what kind of crashdump information should be written. Make your choice, and the crashdump file will magically appear on your desktop.

To uninstall, run mydearwatson with the -u option. mydearwatson tries to remember which JIT debugger was installed before, and will reinstall that JIT debugger. The mechanism for doing this is far from perfect, though.

If you look at the code, you'll notice that it basically implements a minimal debugger, using Win32 debugging APIs such as DebugActiveProcess or WaitForDebugEvent. I've never written a debugger before, so I'd assume there are a few subtleties and bugs hidden in this code, but it did work for me on both XP and Vista systems. Test results most welcome.

---

Previous month: Click here.