CRYSTAL SPACE COM NOTES

Written by Dan Ogles

Last updated: October 22, 1998

Introduction

Over the past couple of months, I have been performing an experiment, shall we say, to see how Microsoft’s Component Object Model (COM) fits in with the Crystal Space project. The overall goal is to allow Crystal Space to act as an ActiveX Automation Server for a scripting language. Using COM as the core of our scripting engine gives us several benefits. Some of these benefits will be outlined in this document.

The experiment was to make all of our rasterization code (all classes formerly derived from csGraphics2D and csGraphics3D) put into separate DLLs and accessed via COM. The advantage of this is that if a person knows the interfaces that Crystal Space exports, s/he can write a new rasterization server without Crystal Space having to know anything about it. Crystal Space loads the server dynamically, and uses it based on an assumed interface.

But all that stuff will be covered below. The key point is that now our graphics servers have an extensible architecture. No recompilation is needed on CS’s part to take advantage of new graphics layers. For instance, if an OpenGL server was written, there would be no need for Crystal Space to be recompiled.

Glossary

Client The program that loads and uses COM components from other sources. The Crystal Space executable is the client in our case.

Server The library that contains a set of components that may be used by other programs. The graphics drivers in Crystal Space are the servers.

In-proc In-process. In-proc servers are sets of components that are contained in a DLL. These components exist in the same process as the server.

Out-of-proc Out-of-process. Out-of-process components either exist in another computer process (EXE), or on another machine. Out-of-process components are currently not used in Crystal Space.

Interface This is the means by which COM components are accessed. It is a binary contract that in C++ is equivalent to a pure abstract class.

Coclass This is a singleton object that is in charge of creating COM components. Also referred to as a class factory.

GUID A Global Unique Identifier. This is a 128-bit number that is guaranteed to be unique in time and space. These numbers are generated through the COM API utility GuidGen.

IID An Interface Identifier. This is a GUID that represents an interface. All COM interfaces have an IID associated with them, usually using the IID_Interface naming convention.

CLSID This is a Class Identifier. This is a GUID that represents a specific coclass. This is used by the CoGetClassObject or CoCreateInstance functions to load a COM server.

ProgID A programmatic ID. This is a human-readable name that is directly related to a specific CLSID. A COM API method exists which allows you to retrieve a CLSID based on a progID. ProgIDs usually have the naming convention library.classname.versionnum.

DLL Also known as a shared library. This is a library that is loaded into the process space during run-time instead of linked into the executable during compile-time.

Why COM?

To be perfectly blunt, COM at first glance seems like a bad solution to anything that is supposed to be platform-independent. Up until recently, the only programs that used COM were MS Windows apps. That’s because the COM API only existed on MS Windows platforms. Now the COM API is ported to several different flavors of Unix, and I believe it exists for the Mac (otherwise Internet Explorer wouldn’t work…). But what about the platforms that we support that don’t support the COM API? We certainly don’t want to stop supporting those platforms just because they don’t choose to implement an API standard developed by Microsoft.

Well, there is a distinction between what is COM and what is the COM API. The COM architecture is just that – an architecture. It only specifies a standard that COM objects adhere to. To adhere to the standard, you only need to organize your object in a specific manner. No system calls are absolutely necessary to support COM. This is very important and should not be forgotten.

The COM API is a set of functions that are implemented by the operating system. The vast majority of these functions aren’t necessary – the bulk of the COM API exists to support out-of-process and remote activation (Distributed COM). The API functions that are needed are easy to emulate. The csCom library, written specifically for Crystal Space, does this.

So what are the advantages of COM? That’s a very good question. And here’s the answer.

De-couples the physical package of a set of objects from the EXE that uses it. This allows a great deal of flexibility because it potentially allows any EXE to use it if it knows how. COM calls this packaged set of objects a component. A component is a set of objects that are completely self-contained.

A component adheres to a binary standard. This standard defines the way objects look in memory. This is advantageous because any language that can interpret the binary standard may access that object. This allows one C++ executable to access a component that was compiled with another compiler, or even access a component that was written in a different language. This is not possible without COM due to name-mangling, as well as compiler implementers choosing proprietary representations of objects in memory.

Allows a component to be extensible, so that later versions of a component can be made, yet that component can still be used by older applications that expect the old component. This means that you can upgrade a single component without having to upgrade all the executables that use that component. This can be very important if you are dealing with something such as a class library, which may be used by hundreds of different commercial applications.

Separation of interface from implementation. This is a fundamental aspect of COM. It simply means that any COM object has two separate identities – the external interface and the internal implementation. The interface is simply a set of methods that the object says it supports. No data is directly accessible through COM – all data manipulation is done through method calls. The interface serves as a contract which lets the component user (the client) know how to deal with the component. This allows you to have an implementation that may be very complex and hard to understand, yet you can ‘shield’ it by putting an easy-to-use interface on top of it.

Not sold yet? Well it gets better. First of all, components are very lightweight. The amount of code and data needed for an object to support COM is negligible. This usually makes the compiled size of components very small. Secondly, COM is fast. The amount of overhead involved in accessing a COM object is equivalent to the amount of overhead involved in a virtual function call in C++. That amounts to a pointer de-reference or two. Note: this only applies to in-proc (DLL) activation, which is the only type of activation Crystal Space uses. Out-of-proc activation is done through the ORPC transport, which is COM API specific. The degree of overhead is much larger when accessing out-of-proc components, making it undesired for our purposes.

The Crystal Space COM Library

The csCom Library is a library designed to emulate certain standard COM API functions. csCom may be compiled in three different ways:

Direct-to-COM support (for COMpatible OS’s).

COM emulation through dynamic linking.

COM emulation through static linking.

By default, the COM library is compiled assuming that the OS supports the COM API. If the API does not exist, then the directive NO_COM_SUPPORT should be defined. When this directive is defined, csCom defaults to COM emulation through dynamic linking. The platform should support some means to load a dynamic linked library (also known as a shared library) into the current process, and retrieve a pointer to a function based on the string function name. If the target OS does not support this, then the directive CS_STATIC_LINKED should be defined. This is for OS’s that don’t support dynamic linking of any sort. All component servers are compiled as static libraries and linked at compile time with the Crystal Space executable.

The following is a list of COM API functions that csCom emulates if necessary:

csCoInitialize() – initializes the COM API for the current process.

csCoUnitialize() – shuts down the COM API for the current process.

csCoCreateInstance() – creates an instance of a component in another server.

csCoGetClassObject() – retrieves a coclass from another server.

csCLSIDFromProgID() – retrieves a CLSID based on a progID.

csProgIDFromCLSID() – retrieves a ProgID based on a CLSID.

All these methods are prefixed with cs, to make them distinct from the COM API functions if they exist on the system. In addition, the COM emulation library has two helper functions, csRegisterServer and csUnregisterServer. These help register and unregister COM servers from the component registry. They are called by (under any OS that supports the COM API) RegSvr32 or (by Crystal Space’s utility) csRegSvr.

The csCom library also contains macros to simplify the writing of components, as well as standard typedefs and macros that are used by the COM API. All of these are written to be as platform-independent as possible.

Probably the most important typedef is the HRESULT. This is a 32-bit value that is used to return success/failure codes for all COM methods. A value of S_OK (which evaluates to 0) indicates success. There are several different standard HRESULT values that the COM API uses to indicate failure. These should be used in favor of custom HRESULT values whenever possible. There is a particular way custom HRESULT values should be calculated.

There are two important standard COM macros:

STDMETHOD(x) - This is used to designate a COM method within an interface. It equates to "virtual _stdcall HRESULT x". It should be used for all COM methods.

STDMETHODIMP - This is used by methods which implement a COM method. It equates to "_stdcall HRESULT". It should be used by all component implementations.

In addition, csCom has several helper macros that make the implementation of components easier. Some of these macros are based on macros written by Don Box in Essential COM:

MAKE_CS_HRESULT(x) - This is used to create a custom HRESULT when needed.

FINAL_RELEASE(d) - This is used when a COM interface pointer is no longer needed or about to go out of scope. It calls Release() and sets the pointer to NULL.

DECLARE_IUNKNOWN() - This declares the IUnknown methods within a class. Put this macro within your class declaration to implement IUnknown.

DECLARE_INTERFACE_TABLE(class) – Stick this in your class declaration to declare an interface table that is used by the standard IUnknown implementation. If you are not using the standard IUnknown implementation, there is no need to declare this.

DECLARE_AGGREGATABLE_IUNKNOWN() – This declares necessary methods required to make a component aggregatable. Use this when it is possible that the component might be aggregated by another component. It is not necessary for components that aggregate other components themselves.

DECLARE_COMPOSITE_INTERFACE() – This declares an instance of an interface implementation that is used for composition. Stick this in your class declaration to implement an interface through COM composition.

The csCOM library also has a set of macros that make the implementation of IUnknown, specifically QueryInterface, easier to write.

IMPLEMENT_UNKNOWN(classname) – This uses the standard IUnknown implementation for your class. Declare this in your Cpp file to use it. The class must have an interface table.

IMPLEMENT_UNKNOWN_NODELETE(classname) – This creates a standard IUnknown implementation for objects that are not deleted from memory on the final Release call. The class must have an interface table.

BEGIN_INTERFACE_TABLE(classname) – This begins the interface table for your class. The interface table is a static member of the class.

IMPLEMENTS_INTERFACE(name) – Used within the interface table definition. This tells the standard IUnknown implementation that your class implements the interface through C++ inheritance. A static_cast of your ‘this’ pointer is used to retrieve the interface.

IMPLEMENTS_COMPOSITE_INTERFACE(name) – Used within the interface table definition. This tells the standard IUnknown implementation that your class implements the interface through COM composition. The composition must follow the standard naming convention, outlined later.

END_INTERFACE_TABLE() – Use this to end your interface table definition.

Direct-to-COM

When the csCom library is compiled without either NO_COM_SUPPORT or CS_STATIC_LINKED, it defaults to Direct-to-COM support. This simply forwards all calls to the COM API methods, performing ANSI-to-Unicode conversions for string parameters if necessary. Use this when the COM API exists on the target platform.

COM Emulation through Dynamic Linking

When the csCom library is compiled with NO_COM_SUPPORT, it uses the dynamic-linking version of the emulation library. The majority of this code involves the access to the Crystal Space Component Registry. The registry is within the file cryst.reg. This file forms the abstraction layer over the location of COM components. All component servers are registered into this file using the method csRegisterServer. To use this emulation mode, you must implement three functions:

// used to dynamically load a library.

extern CS_HLIBRARY csLoadLibrary( char* szLibName );

// returns the address of a procedure

PROC csGetProcAddress( CS_HLIBRARY hLib, char* szProcName );

// free all libraries loaded into this process.

void csFreeAllLibraries();

The CS_HLIBRARY type is simply a 32-bit value that is used as a handle onto a loaded library. You must keep track of all loaded libraries, so that you are able to unload them on response to csFreeAllLibraries(). The csLoadLibrary() function loads a dynamic library into the process address space based on the library name. It should call the Dll entry point DllMain. This is absolutely essential to proper run-time behavior. On success, it returns the handle to the library. On failure it returns null. The csGetProcAddress() function returns the address of a function based on it’s exported name. It returns a void pointer to the function.

These functions are platform-specific, so they should be placed within the appropriate directory. They should be compiled into the csCom library.

COM Emulation through Static Linking

When the csCom library is compiled with both NO_COM_SUPPORT and CS_STATIC_LINKED, it uses the static linking emulation mode. This is the least powerful mode and is only meant to be used by legacy operating systems that don’t support dynamic linking in any form. All COM servers are linked at compile-time with the Crystal Space executable, making the plug-in architecture useless. All activation is done through a global array, gb_aiidStaticLinkIDs, which is an array mapping CLSIDs and progIDs to coclass object instances. It is ended with an empty entry. Whenever a COM server is added to Crystal Space that may be used by operating systems that don’t support dynamic linking, an entry should be added to this array.

This emulation mode is currenlty untested, as of October 25^th, 1998.

Crystal Space COM Specifics

The Crystal Space project has been split up into several different projects. The file listings are up to date as of October 22^nd, 1998.

CSpace – The main Crystal Space statically linked library. This is linked into projects such as WalkTest or Mazed.

Files: ICamera.cxx ILightMap.cxx IPolygon.cxx ITexture.cxx IWorld.cxx csspr2d.cpp csobject.cpp csobjvec.cpp csrect.cpp csstrvec.cpp csvector.cpp polyset.cpp dynlight.cpp light.cpp lightmap.cpp cssprite.cpp thing.cpp thingtpl.cpp polygon.cpp polyplan.cpp polytext.cpp portal.cpp camera.cpp inv_cmap.cpp library.cpp msectobj.cpp physics.cpp sector.cpp stats.cpp textrans.cpp texture.cpp wirefrm.cpp world.cpp csloader.cpp command.cpp csview.cpp bsp.cpp math2d.cpp math3d.cpp polyclip.cpp csscript.cpp intscri.cpp primscri.cpp syssound.cpp timing.cpp csendian.cpp fonts.cpp system.cpp tcache.cpp tcache16.cpp csevent.cpp cseventq.cpp csinput.cpp gifimage.cpp image.cpp imageldr.cpp pcx.cpp pngimage.cpp tgaimage.cpp archive.cpp inifile.cpp memheap.cpp memory.cpp parser.cpp sparse3d.cpp token.cpp util.cpp channel.cpp cssound.cpp module.cpp wave.cpp build.cpp collide.cpp collider.cpp collp.cpp overlap.cpp rapid.cpp…and the files implementing SysSystemDriver and your platform-dependent keyboard and mouse drivers.

Driver2D – This is the COM server for the default implementation IGraphics2D interface. This is compiled as a DLL/shared library.

Files: Driver2d.cpp Fonts.cpp Graph2d.cpp IGraph2d.cxx…and the files implementing your system-dependent csGraphics2d-derived class (and any supporting files).

SoftwareRender – this is the COM server for the implementation of IGraphics3D that does not take advantage of any hardware acceleration.

Files: SoftwareRender.cpp Graph3d.cpp Scan.cpp Scan16.cpp.

Direct3DRender – this is the COM server for the implementation of IGraphics3D that performs hardware acceleration through the Windows Direct3DÔ API.

Files: D3dCache.cpp Direct3DRender.cpp g3d_d3d.cpp hicache.cpp

GlideRender – this is the COM server for the implementation of IGraphics3D and IGraphics2D that performs hardware acceleration through the 3DFx GLIDEÔ API.

Files: Undetermined.

OpenGLRender – this is the COM server for the implementation of IGraphics3D that renders through the OpenGLÔ API.

Files: Undetermined.

Currently, each graphics driver exists in it’s own DLL. In addition the associated 2d graphics driver usually resides in a different DLL from the 3d graphics driver. This allows two different 3d graphics drivers to use the same 2d graphics driver without replicating code.

The COM architecture puts an abstraction layer over the names of DLLs through the registry and CLSIDs. The Crystal Space executable never loads a COM server by name. It instead loads it by CLSID, which results a look-up in the registry for the server location. To activate a COM object, a call is made to csCoCreateInstance or csCoGetClassObject. The function csSystemDriver::InitGraphics() method is in charge of creating the 3d graphics driver based on the progID specified in cryst.cfg using csCoGetClassObject. Internally, this loads the associated DLL and calls the 3d graphics driver’s DLL function DllGetClassObject. This in turn retrieves a pointer to the 3d driver’s class object. A COM class object is a singleton COM object that is in charge of creating objects. A call to IGraphicsContextFactory::CreateInstance() actually creates the instance of the 3d driver in memory.

All DLLs must export the following functions to be true COM servers. Driver2D, SoftwareRender, Direct3DRender, GlideRender, and OpenGLRender all export the following symbols from their respective DLLs:

DllMain

DllCanUnloadNow

DllGetClassObject

DllRegisterServer

DllUnregisterServer

And these DLL functions are required for COM to operate successfully.

ModuleAddRef() – Adds a reference to the module

ModuleRelease() – Releases a reference to the module

DllMain() – Initial entry point of the DLL.

DllCanUnloadNow() – returns S_OK if the module reference count is zero, S_FALSE otherwise.

DllGetClassObject() – returns a class object based on a CLSID.

DllRegisterServer() – registers the server in the registry (using csRegisterServer).

DllUnregisterServer() – unregisters the server from the registry (using csUnregisterServer).

Making Your Platform Work – A Mini How-To

Since all 3D drivers are ported or soon-to-be ported, all that’s required to get your platform up and running is to make the 2d graphics drivers COMpatible. All 2D graphics drivers are derived from csGraphics2D, which takes care of a large portion of the COM stuff for you. The COM interface IGraphics2D is implemented through composition, so that you can retain your method signatures while still supporting COM. What you’ll need to do is add the following inside the brackets of your class declaration:

DECLARE_IUNKNOWN()

DECLARE_INTERFACE_TABLE(classname)

These macros are defined in COM.H.

DECLARE_IUNKNOWN() makes declarations for all the IUnknown methods. DECLARE_INTERFACE_TABLE() is used to declare a table that is used by the default implementation of QueryInterface().

You’ll also need to add the following to your C++ file:

BEGIN_INTERFACE_TABLE(classname)

IMPLEMENTS_COMPOSITE_INTERFACE_EX( IGraphics2D, XGraphics2D )

IMPLEMENTS_COMPOSITE_INTERFACE_EX( IGraphicsInfo, XGraphicsInfo )

END_INTERFACE_TABLE()

IMPLEMENT_UNKNOWN_NODELETE(classname)

This gives you a standard implementation of IUnknown (one that doesn’t delete when the last reference is released) and defines an interface table that is used by QueryInterface to determine what interfaces your object supports. The two entries in the interface table indicate that the class implements IGraphics2D and IGraphicsInfo through COM composition using the classes IXGraphics2D and IXGraphicsInfo.

After this is completed, you have a standard implementation for IUnknown, including a working version of QueryInterface.

The next part is the harder part. You’ll have to move your 2d graphics driver our of the Crystal Space EXE and into it’s own DLL. To do this, you’ll have to make sure that the 2d graphics driver doesn’t have any dependencies on the Crystal Space executable. In other words, you can’t access any Crystal Space classes directly from within the 2d graphics DLL. To access Crystal Space, you’ll have to use one of the interfaces used by Crystal Space:

ICamera - implemented by Camera.

ILightMap – implemented by LightMap.

IMipMapContainer – implemented by TextureMM.

IPolygon3D – implemented by Polygon3D.

IPolygonSet – implemented by PolySet.

IPolygonTexture – implemented by PolyTexture.

ISystem – implemented by csSystemDriver.

ITextureCache – implemented by TextureCache and TextureCache16.

ITextureContainer – implemented by Textures.

ITextureMap – implemented by Texture8 and Texture24.

IWorld – implemented by World.

These will provide you with most of what you need. But you might need more than this. For instance, your csSystemDriver-derived class might have specific information that your 2d graphics driver needs to operate. To expose additional functionality, you’ll have to create a system specific interface for your system driver. An example name might be ISystemLinuxInternal. This interface should be exposed by your SysSystemDriver implementation. The interface might look something like this:

interface ISystemLinuxInternal : public IUnknown

{

STDMETHOD(GetSomething)(int& retval) = 0;

STDMETHOD(MakeSomethingHappen)() = 0;

};

Notice that this is a pure abstract class. This defines the binary contract for objects that implement the ISystemLinuxInternal interface. Next, you’ll need to implement ISystemLinuxInternal within your system driver. The most straightforward way is through C++ inheritance:

class SysSystemDriver: public csSystemDriver, public ISystemLinuxInternal

{

. . .

DECLARE_IUNKNOWN()

DECLARE_INTERFACE_TABLE(SysSystemDriver)

};

// in the cpp file:

BEGIN_INTERFACE_TABLE(SysSystemDriver)

IMPLEMENTS_COMPOSITE_INTERFACE(System)

IMPLEMENTS_INTERFACE(ISystemLinuxInternal)

END_INTERFACE_TABLE()

IMPLEMENT_UNKNOWN_NODELETE(SysSystemDriver)

This makes SysSystemDriver a class that supports COM. It will support the interfaces ISystem and ISystemLinuxInternal through the IUnknown method QueryInterface(). You’ll want to make sure to provide an implementation for all methods within ISystemLinuxInternal within your system driver.

Now you’ll want to modify your 2D graphics driver to take in an ISystem* in it’s constructor instead:

SysGraphics2D::SysGraphics2D(ISystem* piSystem)

You can use this ISystem* to retrieve an ISystemLinuxInternal pointer using QueryInterface:

ISystemLinuxInternal* piLinuxSystem = NULL;

piSystem->QueryInterface(IID_ISystemLinuxInternal, void**)&piLinuxSystem);

After that you can use the piLinuxSystem interface pointer to access any system-specific variables you need. Once you are done using interface pointers, or right before they go out of scope, you must make a call to Release:

FINAL_RELEASE(piLinuxSystem)

FINAL_RELEASE(piSystem)

Notice that although the two pointers point to the same underlying object (SysSystemDriver), Release is called twice. That is because AddRef and Release are called on a per-interface basis. Here are the rules for when to call AddRef and Release:

You must call AddRef when a non-null interface pointer is copied from one memory location to another.

You must call Release prior to overwriting a memory location that contains a non-null interface pointer, or immediately before the memory location is lost.

Redundant calls to AddRef and Release can be optimized away if there is special knowledge about the relationship between two or more memory locations.

Making AddRef and Release calls when appropriate is essential to having a stable COM architecture. At some point in the future, template smart-pointers will be used to call AddRef and Release when appropriate in most circumstances.

After all dependencies are ripped out (make sure you don’t include any header files from Crystal Space either! only interface include files from Crystal Space), you’ll want to write the DLL functions. The DLL functions required are outlined above, in Crystal Space COM Specifics. A good example implementation exists in SoftwareRender.cpp, or the Windows platform’s Driver2d.cpp.

After all dependencies with Crystal Space are ripped out, you should be able to compile successfully. If you get unresolved external errors during the link process, it probably means that you still have dependencies with Crystal Space.

After the DLL is compiled, you will want to register it using the utility csRegSvr.