mod_ntlm versus long user names (06 Mar 2009)

In Apache's server logfiles, I found entries like the following:

[Mon Mar 02 11:37:37 2009] [error] [client 42.42.42.42] 144404120 17144 
/twiki/bin/viewauth/Some/Topic - ntlm_decode_msg failed:
   type: 3, host: "SOMEHOST", user: "", domain: "SOMEDOMAIN", error: 16

The server system runs CentOS 5 and Apache 2.2. Note how the log message claims that no user name was provided, even though the user did of course enter their name when the browser prompted for it.

The other noteworthy observation in this case was that the user name was unusually long - 17 characters, not including the domain name. However, the NTLM specs I looked up didn't suggest any name length restrictions. Then I looked up the mod_ntlm code - and found the following in the file ntlmssp.inc.c:

#define MAX_HOSTLEN 32
#define MAX_DOMLEN 32
#define MAX_USERLEN 32

Hmmm... so indeed there was a hard limit for the user name length! But then, the user's name had 17 characters, i.e. much less than 32, so shouldn't this still work?

The solution is that at least in our case, user names are transmitted in UTF-16 encoding, which means that every character is (at least) two bytes!

The lazy kind of coder that I am, I simply doubled all hardcoded limits, recompiled, and my authentication woes were over! Well, almost: Before reinstalling mod_ntlm, I also had to tweak its Makefile slightly as follows:

*** Makefile    2009/03/02 18:02:20     1.1
--- Makefile    2009/03/04 15:55:57
***************
*** 17,23 ****

  #   install the shared object file into Apache
  install: all
!       $(APXS) -i -a -n 'ntlm' mod_ntlm.so

  #   cleanup
  clean:
--- 17,23 ----

  #   install the shared object file into Apache
  install: all
!       $(APXS) -i -a -n 'ntlm' mod_ntlm.la
  #   cleanup
  clean:

Hope this is useful to someone out there!