Copyright © 2003 by Gene Michael Stover. All rights reserved. Permission to copy, store, & view this document unmodified & in its entirety is granted.
For some reason I don't fully understand, I, Gene Michael Stover, effectively a Microsoft-hater, have become interested in Microsoft COM. Some irrational part of my brain motivates me to consider implementing COM for Lisp on non-Windows operating systems. So herein are notes I have gathered about COM.
These notes might be useful to other experienced programmers who know they could understand COM fully, inside & out, in just a few hours if only they could find a source of documentation that cut through the crap, hype, & tutorials, & spewed the important parts.
I'm thinking about implementing Microsoft COM for Lisp. Out-of-proc only, so I guess it is a DCOM system. Anyway, I'm thinking of doing it. I'm not sure why; maybe just because I can't figure out why no one has implemented COM for non-Microsoft platforms. I am also aware that Microsoft is phasing-out COM. Yet I still think of writing DCOM for Lisp.
I picture something like this on the Lisp client side. Let's say there is a service on your computer that I can access with DCOM. I might need the DLL to get type information, which is somewhat silly because it is out-of-proc only, but wouldn't a Windows programmer need the DLL, too?
So I, Lisp programmer, might do this:
;; I want a client to connect to the "WizBang" COM
;; service on your.computer.com. I already have
;; the DLL (or typelib) for WizBang on my computer.
;; Tell my Lisp DCOM client library to produce a
;; client library for WizBang. Because Lisp is
;; dynamic & "self-programming", this function call
;; can create classes, messages, & methods.
;;
;; The first argument is the name I want the library
;; to have in my Lisp program (to avoid name collisions
;; with other libraries). The second argument is the
;; DLL or TLB file with type information.
(create-client-library 'WizBang "wizbang.dll")
;; CREATE-CLIENT-LIBRARY probably created functions &
;; a type for WizBang. I probably need to program my
;; own wrapper functions function calls on WizBang
;; objects aren't eye-sores, but let's pretend that I
;; don't need to do that. So let's pretend that
;; CREATE-CLIENT-LIBRARY created functions that look
;; just fine when I call them.
;; Create an instance of WizBang on Pavel's server.
;; (I think the actual COM term for this is
;; "activation", but I am not certain.) Notice that
;; I must specify the server name & maybe some other
;; connection information. So this is dynamic, with
;; little or no type-checking. I _think_ I could make
;; Lisp do some dynamic type-checking, but that doesn't
;; matter now.
;; Also, it's possible that I'd need to translate the
;; WizBang class name to a GUID or whatever. Those are
;; details that I think I can ignore for now.
(setq wb (com:cocreate "WizBang" "your.computer.com"))
;; Let's pretend that a WizBang can give me a list of
;; uh, strings or whatever.
(format t "~&The strings in the WizBang are ~A"
(wizbang:get-strings wb))
;; Though Lisp has garbage collection, I need to tell
;; COM when I have finished using the WizBang. I'll
;; call that "release" now. I know there is a COM word
;; for it; can't remember if it's "release" or something
;; else.
(com:release wb)
All the magic happens in that CREATE-CLIENT-LIBRARY function. It reads the interface description from the DLL or type lib & creates a class, interface, & stub methods. It probably should re-use the generic functions that are defined elsewhere, maybe by another CREATE-CLIENT-LIBRARY that read a superclass's type lib. So it's not exactly as simple as I've shown, but I'll betcha it can be close to this.
Initially, I'll need to write some Lisp functions that read the type information from DLLs & type libraries. So I can see what is in those libraries.
Software can be re-used at compile-time (source code re-use), link time (compiler-specific binary code re-use), or run-time (compiler-independant, language-independant binary code re-use). (There are probably other ways, including cut-&-paste re-use at the source code level.) Software components allow re-use at run-time on the binary level.
Microsoft COM is Microsoft's implementation of components. It provides binary re-use at run-time that is mostly independant of the compiler. From some experienced Windows programmers I know, I hear it is almost transparently integrated into Visual Basic; it probably is one of the two or three features which make VB popular. It is more cumbersome to use in C++, but still works well.
Microsoft says that COM is, or soon will be, deprecated. It will be replaced by the dot-Nyet Common Language Runtime. That could mean a lot of things, though, because COM means a lot of things to different people & because dot-Nyet will remain backwardly compatible with COM.
COM can link components into the client's process. These are ``in-proc'' components & are effectively shared libraries with an object-oriented interface. (That's if COM means Microsoft software components. An alternative definition is that Microsoft software components can link components into the client's process via a component implementation called COM.) It requires the component to run on the same operating system that the client is running on.
From the point of view of implementing Microsoft-style software components on a non-Windows operating system, where binary compatibility is impractical, DCOM is more interesting than in-proc COM. DCOM is the over-the-wire protocol for ``out-of-proc'', remote Microsoft component object messages.
I would think that DCOM, plus some explanation of semantics, is all that a programmer would need to implement COM on non-Windows operating systems.
The official DCOM protocol documentation is []. Though Microsoft congratulated itself in 1996 for opening that protocol to the world, & though there are references to it on Microsoft's web site, the DCOM protocol document is not on the World Wide Web, & I have looked for it in a recent (circa 2003) issue of Microsoft Developer's Network (MSDN) without success.5.1
I read somewhere online (probably in one of the DCOM documents at Microsoft), that DCOM is derived from DCE RPC. Maybe I can figure out enough of the DCOM protocol if armed with knowledge of DCE RPC.
If I could get the damned DCOM protocol document, implementing it for Lisp would be easy. As it is, I'd have to read the DCE RPC document5.2, then apply that knowledge while I use a protocol analyzer to watch DCOM messages on my network. Hell, I'd have to borrow a Windows computer from a friend, too5.3. Sadly, it's one of those projects that would be fun but is made nearly infeasible due to lack of good documentation. I'll keep looking, but I'm not hopeful.
I suppose I could write the same kind of library I want but for SOAP & other more modern, ``web service'' types of components. dot-Nyet is similar to that, or uses one of those, or is one of those, so I could get a non-Windows dot-Nyet-style components library that way, maybe. And aren't the internal protocols of dot-Nyet open? Might be worth a try. Might be useful to have, too, whereas a COM implementation for non-Windws platforms (& in Lisp, yet) would be ``kinda cool, but why not ten years ago?''.6.1
I searched Microsoft's web site for ``.net component protocol'' & some variations of that. I don't see anything that looks like the dot-Nyet equivalent of the DCOM protocol specification. If not, then Microsoft intends to keep their protocol closed. There are plenty of documents about security & some higher-level protocols, but how about the packet format for an RPC message?
Here are the documents that search revealed which might be useful. (So this is my reading list for 20 July 2003.)
Using Google to search for permutations of ``binary protocol .net component'' revealed nothing useful. Everyone wants to explain the cleverness of proxies, & some talk about the APIs of the COM, DCOM, or whatever libraries, but nobody talks about the over-the-wire protocol. What gets me is that what they talk about is the complex stuff; interpretation of a package is easy stuff (for a programmer), but they don't even gloss over it. They treat it like it's magic, too complex for people to understand.
It's again appearing to me as though this would be a good, fairly simple project if I could get the documentation for the protocol, but someone is keeping it a secret (though Microsoft did congratulate itself in 1996 for opening the DCOM protocol to the public). Because of this shitty proprietary secrecy, a more feasible project would be a web services library for Lisp. It could do the same things conceptually: obtain a service description then create client (proxy) classes & methods for it.