Last year I did a couple of interviews for The Register and other sites regarding my WildFox project which in essence had the goal to add h.264 video support to Firefox using the GStreamer backend or FFmpeg. Due to circumstances I didn’t manage to do significant work on this project until a few months ago when I finally began the real modifications to the Firefox source.
As this article isn’t about the quality of the Mozilla source code, or the lack thereof, I won’t dwell on it too long. Suffice it to say that I found a lot of instances of NIH (‘Not Invented Here’) syndrome including the networking, smart pointer and threading sections. As my goal was to add Libav (recent fork of FFmpeg) support to the media backend of Firefox I became intimately familiar with these APIs as I discovered just how much of the code would be ripped out without causing adverse effects, and that the smart pointers do not work with anything but NSISupports-derived types.
Anyway, the build system… at first glance the Mozilla build system seems to use Makefiles, that is until you notice the .in extension indicating that they’re autoconfig templates. Or autobreak as lovingly called by a large section of the internet. After much trial and error I discovered that after putting my new Libav decoder & reader into /content/media/libav and the Libav includes into /media/libav of the source tree, creating a single Makefile with EXPORTS and an individual Makefile for each library of Libav (libavformat, libavcodec, etc.), I still had to edit a host of files to make it all work:
/layout/build Makefile.in /toolkit toolkit-makefiles.sh toolkit-tiers.mk /content/media Makefile.in
After following the hints in the Mozilla documentation  I first discovered the files in /toolkit, and things finally began to work. Until I hit a few snags, like having to add a compiler flag to CXXFLAGS but there being no way to specify this in the Makefiles which didn’t get ignored for some reason. Libav is a C99 project and requires -D__STDC_CONSTANT_MACROS to be added to CXXFLAGS  to make it play nice with a C++ project. In the end I put this flag directly into the CXXFLAGS definition in the top of configure.in in the root folder. Ugly but it works.
At this point everything builds, the only thing I’m still stuck on is how to add the Libav’s LIB files to the linker flags. As usual the methods I have found do not work and even adding it to configure.in didn’t seem to do the trick.
To be quite honest I’m ready to give up on improving the Mozilla source. The changes required to bring it up to speed with proper project standards are just too daunting and severe to be handled just by me. As a fun comparison I started the WildFox-Mimic project a few days ago to investigate what it’d take to create a browser which looks and feels like Firefox, but uses Qt and the QtWebKit engine. The result is a modern, up to date browser with an HTML 5 video/audio backend which uses Qt’s Phonon which wraps around the OS’s media framework, be it DirectShow, GStreamer or something else. In other words it’s perfect. QtWebKit can also use the same NPAPI plugins Firefox uses, so Flash support is available out of the box. The Persona themes and JetPack add-ons can also be supported.
The result with WildFox-Mimic would be a browser with a codebase a fraction the size of the Firefox one, with most of the development and maintenance performed by the Qt and QtWebKit maintainers. This is the direction I think I’ll be heading towards.