The other day, I was testing COM clients which accessed a collection class
via a COM-style enumerator (
IEnumVARIANT). And those clients crashed
as soon as they tried to do
anything with the enumerator. Of course, the
same code had worked just fine all the time before. What changed?
In COM, a collection interface often implements a function called
GetEnumerator()
which returns the actual enumerator interface (
IEnumVARIANT), or rather,
a pointer to the interface. In my case, the signature of that function was:
HRESULT GetEnumerator(IUnknown **);
Didn't I say that
GetEnumerator is supposed to return an
IEnumVARIANT
pointer? Yup, but for reasons which I may cover here in one
of my next bonus lives, that signature was changed from
IEnumVARIANT to
IUnknown.
This, however, is merely a syntactic change - the function actually still
returned
IEnumVARIANT pointers, so this alone didn't explain the crashes.
Well, I had been
bitten before by smart pointers,
and it happened again this time! The COM client code declared a smart
pointer for the enumerator like this:
CComPtr<IEnumVARIANT> enumerator = array->GetEnumerator();
This is perfectly correct code as far as I can tell, but it causes a fatal
avalanche:
- The compiler notices that
GetEnumerator returns an IUnknown pointer.
This doesn't match the constructor of this particular smart pointer
which expects an argument of type IEnumVARIANT *.
- So the compiler looks for other matching constructors.
- It doesn't find a matching constructor in
CComPtr itself,
but CComPtr is derived from CComPtrBase which has
an undocumented constructor CComPtrBase(int).
- To match this constructor, the compiler converts the
return value of
GetEnumerator() into a bool value which
compresses the 32 or 64 bits of the pointer into a single bit!
(Ha! WinZip, can you beat that?)
- The boolean value is then passed to the
CComPtrBase(int) constructor.
- To add insult to injury, this constructor doesn't even use its argument
and instead resets the internally held interface pointer to 0.
Any subsequent attempt to access the interface through the smart pointer now crashes
because the smart pointer tries to use its internal interface pointer - which is 0.
All this happens without a single compiler or runtime warning. Now, of course it
was our own silly fault - the
GetEnumerator declaration was bogus.
But neither C++ nor ATL really helped to spot this issue.
On the contrary, the C++ type system (and its implicit
type conversions) and the design of the ATL smart pointer classes
collaborated to hide the issue away from me until it was too late.