macOS is an awesome platform, especially for developers. It is a consumer-friendly Unix-like operating system, which makes it pretty similar to most Linux and BSD systems. This especially helps for developing software that is intended to be ran in such an environment. While macOS still has some differences, most software is almost completely source-compatible.
Since macOS doesn’t ship with a package manager by default, many third-party projects have been startet to fill this gap, the most popular being homebrew. Because of the source-compatibility with Linux, most packages that can be found in a Linux package manager like apt, pacman or apk cna be found in homebrew.
Homebrew however isn’t without its flaws. It is not very transparent on the source of the .app files (if it is just a pre-built .app downloaded from the projects website, for example in the case of firefox or if the .app is actually being built by the package management team. The first being more of a security risk because a malicious actor could just swap out the binaries and the source code would still look okay), the permissions (brew doesn’t need sudo because it basically just changes the permission on the globally installed packages path to be edited by anybody…) but personally my biggest issue is that homebrew went all-in on LLM’s. While LLM’s have real world use-cases, the issues with those tools are omnipreent.
This is where pkgsrc comes into play. It presents itself as a “framework for managing third-party software” and after learning and using this “framework” for about three months now, it is possibly one of my favourite pieces of software.
pkgsrc is a project that originates from NetBSD, an operating system that aims to be “free, fast, secure and highly portable”. The portability is one of the largest focus in this project. The phrase “Of course it runs NetBSD” is a commonly heard sentence about the project. The operating system currently has releases for 53 different processor architectures and it even runs on a toaster!
NetBSD also has a pretty solid stance on code that was generated by LLM’s: The project prohibits code that was generated by tools such as Copilot or ChatGPT without prior written approval by the core team. This has not happend to this date.
I’m pretty sure that this exeption was made because of security vulnerabilities that are found with the use of LLM’s and affect real world scenarios. LLM’s can be useful for such cases and while I do not support anybody that uses such tools, I see why this is useful and should not be prevented. The NetBSD team also encourages the contributers strongly to understand the code that they’ve written. I believe that this only firther implies that LLM usage will be an exeption rather than the norm.
pkgsrc reflex both previously mentioned philosophies. It is available on various different operating systems such as macOS, Linux and of course NetBSD. It also prohibits the use of LLM’s due to the project being under the NetBSD umbrella.
The framework can be used in two ways: Building everything from source, like for example portage on gentoo, or by using pre-compiled binary packages like the more commonly used apt-get, pacman and apk. Though the NetBSD project doens’t provide pre-compiled binaries for the macOS platform itself, MNX Cloud serves those and pkgsrc directly recommends them.
To get started with using the binary packages, head over to the official installation guide. Because this guide is a subject that might change in the future, it is best if I do not explicitly show copy-paste commands in here, however I can explain the general concept of the installer.
First, a tarball is being downloaded and validated that it hasn’t been tampered with, then the tarball will be extracted into /. This will put some binary files into some path and make those available to the user by adding them to the PATH variable. At the time of writing this, the path is /opt/pkg. Lastly it reloads the environment to make the binaries available.
pkgsrc now can be controlled with the high-level cli tool pkgin. This provides an easy interface to the underlying architecture of pkgsrc to install, update and remove packages on the system. The “underlying architecture” are just the pkg_* tools, such as pkg_add. For example, instead of having to write multiple lines of bash to install a package:
$ PKG_PATH="https://cdn.NetBSD.org/pub/pkgsrc/packages"
$ PKG_PATH="$PKG_PATH/OPSYS/ARCH/VERSIONS/All/"
$ export PKG_PATH
$ sudo pkg_add alacritty
using pkgin we only need to write one:
$ sudo pkgin install alacritty
Besides the install option, the tool also supports:
upgrade: Upgrades the installed pacakges and the dependencies that also need to be updatedfull-upgrade: Upgrades all packages on the systemremove: Removes a packageavail: List the available packagessearch: Search for a packageautoremove: Remove orphan dependenciesclean: Delete packages from the cache directory
There are even more options, however this is not a guide about pkgin but rather pkgsrc, so lets talk about compiling!
As previously mentioned, pkgsrc provides a way to compile everything from the source code. This generally consumes a lot more time and resources, however it can also improve performance. While packages that were pre-compiled may not be optimized, when compiling from source, we can optimize each and every software to our needs. Compiling from source also is considered as being more complex, however pkgsrc is very mature in managing the complexities for the user. The whole three months that I’ve used it so far, I never had an issue with pkgsrc itself but only with the platform around it, in my case macOS.
To start compiling, some prerequisites are needed; Apple’s command line developer tools and a copy of the pkgsrc source tree. If you’ve developed on macOS previously, the first is probably already installed. To test this, simply run xcode-select --install and if a new window opens, click on “install”, if not, you’re good to go. The latest source tree can either be downloaded from NetBSD’s website as a tarball or from their Github mirror as a git repository. Note that NetBSD (and thus pkgsrc) dont actually use git as their version control, but rather CVS. The github mirror uses a tool to automatically convert from CVS to git, so use this with care. With that being said; I’ve never had issues with the github mirror in the whole time since I started using pkgsrc.
To just test things out, I recommend to just download the tarball from the NetBSD’s project page:
$ cd $HOME
$ curl -O https://cdn.netbsd.org/pub/pkgsrc/current/pkgsrc.tar.gz
$ tar -zxf pkgsrc.tar.gz
$ rm pkgsrc.tar.gz
This will download and extract the tarball into the users $HOME directory. It is not that relevant where the pkgsrc repository is located in, however because we’re later on going to install the binaries only for this user, I generally put everything into $HOME.
Lastly we need to bootstrap some packages which are used by pkgsrc. They are needed for every package and are assumed on each system that pkgsrc is used to build packages. Luckily these can be bootstrapped pretty easily by running a script located in pkgsrc/bootstrap:
$ ./bootstrap \
--prefix $HOME/pkg \
--prefer-pkgsrc yes \
--make-jobs \
--unprivileged
The script will install, for example, bmake, BSD’s equivalent of the make command. This command has some differences to the more common GNU Make and pkgsrc’s package definitions are actually all just Makefiles. This also installs the pkg_* tools to manage the systems packages. With the flag --prefix we tell the boostrap script, where pkgsrc should be installed into. This can be any directory that can be written to. pkgsrc even allows for unprivileged installations - so if you do not have superuser privileges on a given computer, you still can build, manage and install software. Afterwards, we tell pkgsrc that all tools should build themself against the pkgsrc provided packages instead of the system ones. For example, macOS shipps perl, m4 and bison. New macOS versions however ship new versions of those tools and it is possible that this will break packages if they would use the binaries that are provided by Apple. The --prefer-pkgsrc flag tells pkgsrc that we do not want to use those tools but rather rely entirely on pkgsrc for everything. --make-jobs is used to speed up the build process and lastly the --unprivileged option tells pkgsrc that we do actually want to setup everything only for this user – in contrast to homebrew this doesn’t tamper with the permissons for a globally accessible directory. Everything that you install will only be available for your user.
Note that on macOS the /$HOME/pkg is not in the PATH variable by default, which means that all tools would only be accessible through their absolute path, e.g. by typing in /$HOME/pkg/bin/.... To fix this, run the following commands. They will add the needed directories to the PATH variable and then update the shell.
$ echo 'export PATH="/Users/<username>/pkg/bin:/Users/<username>/pkg/sbin:${PATH}"' >> $HOME/.zshrc
$ echo 'export MANPATH="/Users/<username>/pkg/man:${MANPATH}"' >> $HOME/.zshrc
$ source $HOME/.zshrc
After all of this, we now finally can compile our first package. To keep things simple, lets compile GNU Stow, a dotfiles/symlink management tool. pkgsrc is seperated into different directories based on the software’s category. For example, there is the audio category, which contains the build script for ardour, there is the www category, which contains the build script for firefox and for us, the sysutils category is of special interest, because it contains the build script for stow. So lets go in there with:
$ cd sysutils/stow
All that is left to do is to build the software. This can be done with bmake but behold, there is more behind this than one might think. pkgsrc builds upon the concept of “What if software was standardized and there was no need for different build systems. One should simply be able to go into the softwares repository and type make to build the software.” This can be done using pkgsrc. Commonly used options like install or clean are also available for every package and they do exactly what one might expect. So with that knowledge, lets build the package:
$ bmake install clean-depends clean
This will build the package, install it, then clean up the dependencies files and lastly clean up the packages files. To check if stow is installed, use stow --version, which in my case returns stow (GNU Stow) version 2.4.1.
However most users may not use stow and thus do not want this software installed. So lets see how to uninstall software in pkgsrc. First we can take use pkg_info to get further information about the stow package:
$ pkg_info stow
If this command doesn’t fail, it is confirmed, that this package is installed on the system. Now lets uninstall it using:
$ pkg_delete stow
This now should have successfully uninstalled stow from the system. To check this, run pkg_info stow again. This should now return something like pkg_info: can't find package 'stow'. This confirms that stow was successfully uninstalled from the system.
With the basics covered, lets explore the configuration of pkgsrc a bit more. The main configuration file for the framework happens in the mk.conf file, which is located in $HOME/pkg/etc. This file will already exist if pkgsrc is installed correctly, so there is no need to touch it. Lets take a look at the default mk.conf:
.ifdef BSD_PKG_MK
PKGSRC_COMPILER= clang
CC= clang
CXX= clang++
CPP= ${CC} -E
CLANGBASE= /usr
UNPRIVILEGED= yes
PKG_DBDIR= /Users/test/pkg/pkgdb
LOCALBASE= /Users/test/pkg
SYSCONFBASE= /Users/test/pkg/etc
VARBASE= /Users/test/pkg/var
PKG_TOOLS_BIN= /Users/test/pkg/sbin
PKGINFODIR= info
PKGMANDIR= man
PREFER_PKGSRC= yes
TOOLS_PLATFORM.sh?= /Users/test/pkg/bin/mksh
TOOLS_PLATFORM.awk?= /Users/test/pkg/bin/nawk
TOOLS_PLATFORM.sed?= /Users/test/pkg/bin/nbsed
.endif
The first few lines will set the correct c compiler which, in the case of macOS, is clang. clang comes with the previously installed command line developer tools, directly from Apple. Afterwards some variables are set for the correct prefixes of installed files. This might seem weird, because everything is just located under /User/test/pkg, however it can be really useful if, for example, the configuration (/etc) files live on an network filesystem and not on the actual system. This is the same with the other variables, so for example, databases (commonly located under /var) also can live under other locations than under the pkgsrc installation path.
Next, we prefer pkgsrc’s tools. The reason on why to do this is previously mentioned.
And lastly the installation path of the tools that pkgsrc uses are defined. This is done because some tools are different under different platforms, just like GNU Make and NetBSD’s Make. For example, macOS doesn’t use GNU sed and thus some configure scripts don’t work as expected if the default sed is used. With these variables set, installation scripts and alike from pkgsrc’s packages wont complain about some extension not being available.
This configuration script is pretty simple but works perfectly, however on most macOS systems some further configuration is wanted. Some additional settings that I’ve added to my configuraton include:
MAKE_JOBS= 10
This will tell make to use all the available cores on my system to build an application. By default it is set to 1 so setting this will improve compilation times by quite a bit. Note that your system may have more or less cores available. To find out your core count, use sysctl -n hw.physicalcpu.
I would also like to mention that pkgsrc can be configured to only install packages under certain licenses. This can be useful if a company doesn’t develop something that will be open source. It therefore only can use packages under some non-gpl license. This can be set using the ACCEPTABLE_LICENSES variable.
There are many many more possible options that can be adjusted and a list of all default variables can be found under pkgsrc/mk/defaults/mk.conf. Most of the variables have some sane defaults, however if wanted, pkgsrc can be customized pretty heavily.
To find out more about this awesome package manager, head over to pkgsrc.org, where you can find an extensive documentation for this software. Additionally if some help is needed either send an email to the official pkgsrc-users mailing list or send some message into the #pkgsrc IRC channel on the libera network.