Fri, 21. September 2012 – 16:32

While going through a recent article on fxguide.com on the latest installement of the Resident Evil franchise, I jumped offsite following two links: one for Mr. X, a Toronto-based VFX facility and Alembic, an open computer graphics interchange framework jointly developed by Sony Pictures Imageworks and Lucasfilm Ltd., recently release in version 1.1 at the ACM SIGGRAPH Conference.

Alembic is an open computer graphics interchange framework. Alembic distills complex, animated scenes into a non-procedural, application-independent set of baked geometric results. This ‘distillation’ of scenes into baked geometry is exactly analogous to the distillation of lighting and rendering scenes into rendered image data.

Since the software’s debut last year both companies have integrated the technology into their production pipelines. ILM notably using the software for their work on the 2012 blockbuster The Avengers and Sony Pictures Imageworks on the 2012 worldwide hits Men in Black 3 and The Amazing Spider-Man in addition to the upcoming animated feature Hotel Transylvania, scheduled for release September 28, 2012.

Little surprise that given it is an open source project, I have been checking out a working copy of the source code – there always is something to learn by looking at the code from projects like this. Extactly this happened while looking at the installation instructions:

The obvious thing of course I was delighted to notice was, that Alembic is using CMake to handle configuration and build of the source code. It didn’t even require a look into the documentation to have this pointed out, as the CMakeLists.txt was plainly visible in the top-level directory of the code tree. A rather nice surprise though was to find that at a low level the HDF5 technology suite is used for the storage of the actual geometry data. To be fair: the usage of HDF5 within the film industry is nothing unheard of – e.g. the Field3d Voxel Data Storage Library at its heart is using that same technology – but still it took me a little by surprise.

On a sidenote, there are comments in the documentation of the project, which hint as possible improvements for the HDF5 library:

When we profiled Alembic we found that, for some caches, a great deal of time was being spent in HDF5 functions that access groups and attributes by name. We now let you optionally write a side car data-structure within the HDF5 file that maps the HDF5 hierarchy to HDF5 object references which results in lookups that are much faster. You pay the price of a slight disk size increase and memory footprint.

As to be expected, navigation through large volumes of date, folded in a rather complex structure remains a problem that needs constant attention; not that in any shape or form I would consider HDF5 slow or incapable of handling requests and operations of such kind, but given the ever growing need to work with bigger and bigger datasets, this is something the core technology needs to keep up with.