2019-05-27 Adding the InspIRCd v3 ebuild

Version 3 of InspIRCd has many changes, and, as proxy-maintainer, I have the responsibility of updating the ebuild. In order to do this, I began by getting the old ebuild to work without USE flag modifications, then I began updating the USE flags, and finally I tweaked configuration file installation and sent a PR upstream.

Getting the ebuild working

Earlier, I'd created a v3 ebuild in order to test some features of mine, but it was a hack-job; I merely got the program to install without actually worrying about all the features offered and whether they'd changed. Nonetheless, getting the program to install is always a good start, so this is how I began. The first issue I ran into was expected: there is a patch to the program (InspIRCd uses a custom build process, not autotools) that updates paths in the configuration and build files to what Gentoo expects, and this patch had broken on the newer version of the tool. This isn't a particularly difficult fix, but if you're a git baby like me and don't really have a method for generating diffs outside of version control this process can be rather awkward as it isn't really feasible to just edit the patch file itself, and the source code that the patch is generated from isn't a part of the repository with the patch! Luckily I had a cloned git repo of InspIRCd, so I attempted to apply the patch and fix it, but it turns out that simply running git apply will abort if the patch fails and not modify the working tree; instead I had to add the –rej option, giving:

	git apply --rej /path/to/ebuild/files/inspircd-2.0.27-fix-path-builds.patch

Then I was able to see what had applied with git diff, and then update what hadn't applied by opening up both the original and .rej files and editing the original based on what the .rej file was trying to do. Once I had incorporated all of the changes from the .rej file, I could generate a patch with:

	git diff > /path/to/ebuild/files/inspircd-3.0.1-fix-path-builds.patch

Once the patch was in-place, the ebuild was successfully compiled and merged (in Gentoo terminology, after compilation and installation, a package is merged into the live system), but the service wouldn't actually start! After some digging I figured out that the v3 build process was setting the permission bits to 0750 for the main program and 0640 for the modules; this caused problems because the program was installed as root but the service tried to run the program as the inspircd user. Looking at the configuration script showed me that I could supply –uid and –gid options, so I added them to the src_configure() stage and it built and ran successfully, but then I started to have second thoughts: the other programs under /usr/bin are all run as root, and would I really want a rogue InspIRCd process to be able to modify its binary? No, I would have to find a different solution. Thus I decided to manually set the permission bits in the src_install() phase with the code:

	fperms 0755 /usr/bin/inspircd{,-genssl}
	fperms -R 0755 "/usr/lib64/inspircd/modules/."

Note the ebuild-specific fperms in place of chmod; the latter modifies the image by treating the path as relative to the image directory and not the root filesystem. After this, the program launched successfully, but there was a couple of minor things that I noticed during compile time. First, --with-cc was an unknown option, so I instead set the environment variable CXX when executing the configuration file. Also, --enable-openssl was reported as an unknown option; turns out it had been made into a module. Once I'd cleaned these flags up, it was time to take a serious look at the compilation options, with an emphasis on the modules.

Updating USE features of the ebuild

The quintessential question I had when updating this ebuild was, "What modules changes will affect the ebuild?". I began by reviewing the change log, but its focus was on runtime configuration, whereas mine was compile-time configuration. Next I tried running configure --help and from there configure --list-extras; this gave me a list of modules which I then compared to the existing USE flags. From this information I then built up the following list of flags and modules:

	Existing:
        - debug
        - geoip -> maxminddb (m_geo_maxmind)
        - gnutls (m_ssl_gnutls)
        - ldap (m_ldap)
        - mysql (m_mysql)
        - pcre (m_regex_pcre)
        - posix (m_regex_posix)
        - postgres (m_pgsql)
        - sqlite (m_sqlite3)
        - ssl (m_ssl_openssl)
        - tre (m_regex_tre)

        New:
        - re2 (m_regex_re2)
        - stdlib (m_regex_stdlib)
        - mbedtls (m_ssl_mbedtls)
        - sslrehashsignal (m_sslrehashsignal)

        Removed:
        - ipv6

Now that I had the list, the next question was how to test the flags. Doing a test of each possible permutation of USE flags would result in an exponential number of tests, so I instead decided to try 1) compilation with no USE flags, 2) configuration with each USE flag individually, and 3) compilation with all USE flags. Note that when testing each USE flag individually I opted to only test configuration and not compilation; this is because a full compilation for each flag would have taken very long, I'm mostly checking whether the appropriate module is enabled, and compilation will be tested well enough when all USE flags are enabled. I also decided to test the existing flags before adding the new ones.

Testing the ebuild via compilation was easy enough, I simply had to merge it, but testing only the configuration was a bit more tricky. To begin with, I'd run emerge with the -o option in order to install the dependencies of the package, ensuring I had the dependencies correct. After that I'd use the ebuild command in order to run a specific function of the ebuild, such as configuration. Thus testing the configuration stage of an ebuild was done by running ebuild /path/to/package.ebuild configure, followed by ebuild /path/to/package.ebuild clean so that subsequent runs would work. In the cases where the library I needed was already installed on my system, I was able verify that the ebuild was including the correct package with equery b /file/to/check; for example, in order to verify which ebuld libssl.so came from I ran equery b /usr/lib/libssl.so, which then returned dev-libs/openssl-1.0.2r (/usr/lib64/libssl.so.1.0.0).

Of course, things didn't quite go as smoothly as I'd hoped. By default, the configuration stage of InspIRCd expects the user to be able to interactively select which modules to enable and disable, but interactivity is not acceptable when running an ebuild; for this case InspIRCd allows the user to specify the --disable-interactive argument, but there's a trick: modules must be configured in one invocation of the configure script, then a separate invocation of the script must be used to actually configure the project. Easy enough, but, while the first invocation of the script configured the modules correctly, the second invocation would enable modules that hasn't been explicitly selected! After digging in the code I found that, indeed, modules were automatically being enabled if their dependencies were found on the system. After asking the developers about this one of them pointed to a patch which added a --disable-auto-extras option to disable module auto-selection. I added the corresponding patch and argument to the ebuild, and things began working as expected.

Many of the modules were straightforward, but a couple involved a bit more digging. The first was the geoip module, which existed in v2 of InspIRCd but not v3. After a bit of searching, I found that libgeoip was considered obsolete and that its replacement was libmaxminddb. Both appear to do some kind of geography lookup based on the user's IP address; the library defines the protocol for accessing the data, but the actual data has both a free and a licensed version. It sounded like a whole pile of fun that I didn't want to get involved in, so I simply renamed the USE flag from geoip to maxminddb and changed the dependency from dev-libs/geoip to dev-libs/libmaxminddb.

Rather baffling was the m_regex_stdlib.so module. It turns out that this module uses the regex library that comes with the C++11 standard, but that its support was noted in the configuration file as not compliant in GCC. After some searching I found this post which I interpreted to mean that the module would work correctly on all GCC versions >= 4.9, as well as a compatibility matrix which I found rather indecipherable; it wasn't apparent to me whether just the headers were built into GCC or if the implementation was also completed. After some contemplation, I decided to go ahead and add >=sys-devel/gcc-4.9.0 as a dependency to the ebuild. I figured that perhaps I'd get a bug report in the future saying that a later version is required.

The rest of the module changes were simple. The m_ldap.so module activation was simplified in InspIRCd's configuration from m_ldapauth.cpp,m_ldapoper.cpp to simply m_ldap.cpp. The m_regex_re2.so module corresponded to a regular expression library, called "RE2", provided by Google; it required adding the dev-libs/re2 package as a dependency. The m_ssl_mbedtls.so module allowed yet another SSL library, previously called "Polar SSL"; it required adding the net-libs/mbedtls dependency. The m_sslrehashsignal.so library didn't require any additional dependencies; it simply allowed rehashing the SSL configuration via the SIGUSR1 signal, whereas, previously, a logged-in IRCop would have to issue the /rehash -ssl command. There was no equivalent of the --enable-ipv6 options provided in v3, but, upon the asking the devs, I was told that this flag didn't actually do anything in v2 anyways; thus I removed the ipv6 flag from v3. Lastly came the debug USE flag; this flag was tricky because it corresponded to setting an environment variable INSPIRCD_DEBUG to a numeric value during compilation time, and the exact behaviour depended on the value specified. I could have added a number of debug flags, but that would have been weird and bloated, so I decided against it, but the question then remained which value would be most sensible. It turns out that the value 1 added -Werror to the compilation options and would thus cause compilation failures when, say, the compiler was upgraded, so it was not acceptable. The value 3 would remove all symbols from the binary and was meant explicitly for CI, so I ended up setting on the value 2, which would include debug symbols but also enable optimizations.

Now that I'd added the USE flags, I also wanted to update their descriptions as ebuild u inspircd was showing some of the flags as <unknown>. I opened up the Package Management Specification (PMS) documentation and found that I could add the flag to the /usr/portage/profiles/use.desc file for global use flags, and, supposedly, /usr/portage/profiles/use.local.desc for package-specific flags; I say "supposedly" because I was able to do so for the former, global, file but not the latter, package-specific file (the changes didn't appear to be getting picked up). After asking for clarification in IRC, I was told to edit the metadata.xml file in the package's directory; this successfully updated the flags' descriptions for me.

Finally, I took a bit of time to ensure that the documentation files were being installed as expected. InspIRCd is written such that the configuration files contain most of the needed documentation in them, but as a result the files are quite lengthy; they also include a couple tricks such as "die" statements which will terminate the program if not edited out, thus ensuring that the user actually bothered to configure their server properly rather than just running it blindly. For this reason it wasn't clear to me whether the files should be installed into /etc or only placed in /usr/share/doc; installing them into /etc would cause the user to have to manually update their configuration files with etc-update, tediously removing the "die" statements and other cruft, but not doing so meant that the user's configuration would slowly diverge from the default, which could cause confusion later-on as options are removed, renamed, and added. In the end I decided to merge the files into /etc as the user could easily enough forgo updating their files, but conscientious users could then more easily keep their configurations up to date.

Merging to Mainline

Merging went straightforward enough. I created a pull request and got some pushback against the stdlib and posix flags as their names didn't make it clear what they did, and was also told that the GCC >= 4.9 dependency was pointless because simply having it installed wouldn't mean that it was being used due to Portage's slotting mechanism, and modern versions of Gentoo require at least 6.4 anyways. So, I prefixed both of the flags with regex- and removed the GCC version dependency. While I was doing this, though, my Internet went out for 6 says, during which a new version was released, thus, when my Internet was finally restored I also bumped the version to 3.1.0, which meant that I no longer had to include the patch adding --disable-auto-extras, as it had been included in the upgrade. A little later my pull request was merged! I now await the bug reports...