Since I participate in the FreeBSD project there are from time to time some voices which say FreeBSD is dead, Linux is the way to go. Most of the time those voices are trolls, or people which do not really know what FreeBSD has to offer. Sometimes those voices wear blinders, they only see their own little world (were Linux just works fine) and do not see the big picture (like e.g. competition stimulates business, …) or even dare to look what FreeBSD has to offer.
Sometimes those voices raise a valid concern, and it is up to the FreeBSD project to filter out what would be beneficial. Recently there were some mails on the FreeBSD lists in the sense of “What about going into direction X?”. Some people just had the opinion that we should stay were we are. In my opinion this is similarly bad to blindly saying FreeBSD is dead and following the masses. It would mean stagnation. We should not hold people back in exploring new / different directions. Someone wants to write a kernel module in (a subset of) C++ or in Rust… well, go ahead, give it a try, we can put it into the Ports Collection and let people get experience with it.
This discussion on the mailinglists also triggered some kind of “where do we see us in the next years” / strategic thinking reflection. What I present here, is my very own opinion about things we in the FreeBSD project should look at, to stay relevant in the long term. To be able to put that into scope, I need to clarify what “relevant” means in this case.
FreeBSD is currently used by companies like Netflix, NetApp, Cisco, Juniper, and many others as a base for products or services. It is also used by end-users as a work-horse (e.g. mailservers, webservers, …). Staying relevant means in this context, to provide something which the user base is interested in to use and which makes it more easy / fast for the user base to deliver whatever they want or need to deliver than with another kind of system. And this in terms of time to market of a solution (time to deliver a service like a web-/mail-/whatever-server or product), and in terms of performance (which not only means speed, but also security and reliability and …) of the solution.
I have categorized the list of items I think are important into (new) code/features, docs, polishing and project infrastructure. Links in the following usually point to documentation/HOWTOs/experiences for/with FreeBSD, and not to the canonical entry points of the projects or technologies. In a few cases the links point to an explanation in the wikipedia or to the website of the topic in question.
Code/features
The virtualization train (OpenStack, OpenNebula, oVirt, CloudStack, Kubernetes, Docker, Podman, …) is running on full speed. The marked is as big/important, that solution providers even do joint ventures on crossing borders between each others, e.g. VMware is opening up to integrate their solution with solutions from Amazon/Azure/Google. The underlying infrastructure is getting more and more unimportant, as long as the services which shall be run perform as needed. Ease of use and time to market are the key-drivers (the last little piece of performance is mostly important for companies which go to the “edge” (both meanings intended in a non-exclusive-or way) like Netflix for their FreeBSD based CDN). FreeBSD is not really participating in this world. Yes, we had jails way before anyone else out there had something smilar, and some even do not have that right now. But if you are realistic, FreeBSD does not play a major role here. You can do nice things with jails and bhyve, but you have to do it “by hand” (ezjail, iocage and such are improvements on the ease of use side, but that is not enough as this is still limited on a host centric view). The world has moved on to administering a datacenter (to avoid the buzzwords “cloud” or “private-cloud”) with a single-click. In my opinion we would need to port several of the initialy mentioned cloud/container management solutions to FreeBSD and have them able to handle their work via jails and/or bhyve. If FreeBSD is not able to serve as a building block in this big picture, we will fall off the edge in this particular IT-area in the long run.
With all the ready-made containers available in the internet, we should improve our linuxolator. Our kernel-support for this is limited to a 2.6.32-ish ABI version (it is less than 2.6.32, more like 2.6.16, we are missing epoll and inotify support, among others, but this is the lowest version glibc in the CentOS 7 based linux_base port is able to run on… and glibc checks the version number). We need to catch-up to a more recent version if we want to be able to run those ready-made linux containers without issue (we can put a linux system into a jail and start that). If someone would like to work on that, a good start would be to run the Linux Test Project tests via the linuxulator and start fixing bugs. The last time I did that was in 2007 and about 16% of the test cases failed back then. It would be also quite nice if we could integrate those linuxulator tests into the FreeBSD CI. With improvements in the linuxulator and the above mentioned virtualization support, we should be able to run more/those linux-images … ehrm, sorry, docker/kubernetes/…-containers within the linuxulator.
Finish the work regarding kerberos in base. Cy Schubert is/was working on this. I do not know the status of this, but the “fixing 800 ports” part in his mail from May 2018 looks like some more helping hands would be beneficial. This would bring us to a better starting point for a more seamless integration (some ports need “the other” kerberos).
We have one port (as far as I was able to determine… the exact amount does not really matter) in terms of SDN – net/openvswitch – but the documentation of this … leaves room for improvement (kernel support / netmap support and functionality / FreeBSD specific HOWTO). As part of the virtualisation of everything we (yes: we – as part of the FreeBSD handbook, see docs category below for more on this) need to provide this info so that FreeBSD is able to participate in this area. We should also have a look at porting some more SDN software, e.g OpenContrail now tungstenfabric (there is an old contrail porting project), OpenStack Neutron, OpenDaylight, … so that users have a choice, respectively FreeBSD can be integrated into existing heterogeneous environments.
Sensors (temperature, voltage, fans, …), a topic with history. Short: in the Google Summer of Code 2007 code was produced, committed, and then removed again due to a dispute. My personal understanding (very simplified) is “remove everything because some of the data handled by this framework shall not be handled by this framework” (instead of e.g. “remove sensor X, this data shall not be handled in this way”), and “remove everything as this does not handle sensors of type X which are not used in servers but in enterprise class >99% non-IT-related sensors”. Nothing better has shown up since then. If I look at Window, VMware, Solaris and Linux, I can query sensors on my mainboard/chassis/disks/whatever (yes, I am mixing some apples with oranges here), plot them in monitoring systems, and get alarms. In FreeBSD we fail on this topic (actually multiple topics) which I consider to be something basic and mandatory. I do not suggest that we commit the Google Summer of Code 2007 code. I suggest to have a look at what makes sense to do here. Take the existing code and commit it, or improve on this code outside the tree and then commit it, or write something new. In the end it does not matter (for an user) which way it is handled, as long as we have something which users can use in the end. It surely makes sense to have an OS-provided framework of registering sensors in a central place (it would surely be nice if you could get the temp/fan values of your graphics card… ooops… sorry… AI/HPC accelerator together with other similar hardware data in your hardware).
To continue playing well (not only) in the high-availability area, we should also have a look at getting an implementation of MPTCP (Mutlipath TCP) into the tree. Apple (and others) is already using it since 2013 with good benefits (most probably not only for Siri users). There exists some code for FreeBSD, but it is far from usable and it does not look like there is progress since 2016. We say we have the power to serve, but with the cloudification of the recent years, all users expect that everything is always-on and never fails, and being able to provide the server side of this client-server related technology for those people which have such high demands is necessary to not fall behind (do not let us rest on our laurels).
SecureBoot needs also some helping hands. At some point operating systems which do not support it will not be considered by companies anymore.
Another item we should have a look at is to provide means to write kernel code in different languages. Not in the base system, but at least in ports. If someone wants to write a kernel module in C++ or Rust, why not? It offers possibilities to explore new areas. There are even reports of experiences with different languages. It does not fit your needs? Well, ignore it and continue writing kernel code in C, but let other people which want to use a screwdriver instead of a hammer do what they want, they will either learn that they should have used a hammer, or can report about benefits about the screwdriver.
Docs
I think we can improve our end-user docs to the next level. The base system is already well covered (we can surely find some features which we could document), but an user does not use FreeBSD to use FreeBSD. An user surely has a goal in mind which requires to setup some kind of service (mail server, web server, display server (desktop system), …). While one could argue that it is the 3rd party project which needs to document how to run their software on FreeBSD, I think we need to do our share here too. There are a lot of HOWTOs for Linux, and then you have to find some tips and tricks to make something work (better) on FreeBSD. What I have in mind here is that we should document how to make FreeBSD participate in a Windows Active Directory environment, or in an LDAP environment (as a client), improve the Samba part with FreeBSD specific parts (like how to make Samba use ZFS snapshots for Windows Shadow Copies), configuration management tools and so on. I do not talk about providing in-depth docs about the 3rd party software, but little HOWTOs with FreeBSD specific parts / tips and tricks, and a reference to the 3rd party docs. People come to us for real-world needs and if we provide them with a head-start of the most common items (e.g. also covering nginx or whatever and not only apache httpd) and then guide them to further docs will improve the value of our handbook even more for end-users (specially for newcomers, but also for experienced FreeBSD users which out of a sudden now need to do something which they never did before…).
We should also review our docs. The handbook lists e.g. procmail (just an example…). With procmail not being mainained anymore since a long time and known vulnerabilities we should replace the info there with info about maildrop (or any suitable replacement). Careful review may also find similar items which need some care.
One more item I have in mind in terms of docs for user is the restructuring of some parts. Now the world is more thinking in terms of XaaS (“something as a service”) we should also have a “cloud” section (going beyond of what we have in terms of virtualization already) in out handbook. We can put there items like the existing description of virtualisation items, but also should put there new items like glusterfs or object storage or the hopefully upcoming possibility of how to setup OpenStack/kubernetes/… on FreeBSD. This goes into the same direction as the first docs-item in terms of provide more documentation how to achieve goals of our users.
In my opinion we are also lacking on the developer-documentation side. Yes, we have man pages which describe the official API (in most cases). Where I see room for improvement is the source code documentation. Something like doxygen (or whatever the tool of the day is – which one does not really matter, any kind of extractable-from-source documentation is better than no extractable documentation) is already used in several places in our source (search for it via: egrep ‑R ‘\\(brief|file)’ /usr/src/) and we have already some infrastructure to extract and render (HTML / PDF) them. The more accessible / easy it is to start development in FreeBSD, the more attractive it will be (additional to the existing benefits) to people / companies to dive in. The best examples about documenting source code in our code I have found so far is the isci and ocs_fc device code.
Polishing
Polishing something in the topic of staying relevant? Yes! It is the details which matter. If people have 2 options with roughly the same features (nothing missing what you need, same price), which one do they take, the one which has everything consistent and well integrated, or the one with some quirks you can circumvent with a little bit of work on their side?
We have some nice features, but we are not using it to the extend possible. One of the items which come to my mind is DTrace. The area which I think needs polishing is to add more probes, and to have some kind of probe-convention about common topics. For example I/O related naming convention (maybe area specific, like storage I/O and network I/O) and covering all drivers to comply. We should also look into making it more accessible by providing more easy interfaces (no matter if text based (thanks to Devin Teske for dwatch, more of this magic please…), web based, or whatever) to make it really easy (= start a command or click around and you get the result for a specific set of probes/conditions/…). Some examples are statemaps, flamegraphs and most prominently the Oracle/Sun ZFS Storage Analytics to give you an idea what is possible with DTrace and and how to make it accessible to people without knowledge about the kernel internals and programming.
Some polishing in the ports collection would be to revisit the defaults options for ports with options. The target here should be to have consistent default settings (e.g. server software should not depend upon X11 by default (directly or indirectly), most people should not need to build the port with non-default options). One could argue that it is the responsability of the maintainer of the port, and to some extend it is, but we do not have guidelines which help here. So a little team of people to review all ports (and modify them if necessary) and come up with guidelines and examples would be great.
Additionally we should come up with meta-ports for specific use case, e.g. webserver (different flavours… apache/nginx/…), database (different flavours, with some useful tools like mytop or mysqltuner or similar) and then even reference them in the handbook (this goes along with my suggestion above to document real-world use cases instead of “only” the OS itself).
Recently ‑current has seen some low level performance improvements (via ifuncs, …). We should continue this and even extend it to revise default settings / values (non auto-tuned and even auto-tuned ones). I think it would be beneficial in the long run if we target more current hardware (without losing the ability to run on older hardware) and for those values which can not be auto-tuned provide some way of down-tuning (e.g. in a failsafe-boot setting in the loader or in documented settings for rc.conf or wherever those defaults can be changed).
Project infrastructure
We have a CI (continuous integration) system, but it is not very prominently placed. Just recently it gained some more attention from the developer side and we even got the first status report about it (nice! visibility helps making it a part of the community effort). There is a FreeBSD-wiki page about the status and the future ideas, but it was not updated since several months. There is also a page which talks in more details about using it for performance testing, which is something people have talked about since years but never became available (and is not today).
I think we need to improve here. The goals I think which are important are to get various testing, sanitizing and fuzzing technologies integrated into our CI. On the config repository I have not found any integration of e.g. the corresponding clang technologies (fuzzing, ASAN, UBSAN, MSAN (still experimental, so maybe not to target before other mature technologies)) or any other such technology.
We should also make our CI more public/visible (build status linked somewhere on www.freebsd.org, nag people more about issues found by it, have some docs how to add new tests (maybe from ports), so that more people can help in extend what we automatically test (e.g. how could I integrate the LTP (Linux Test Project) tests to test our linuxulator? This requires the download of a linux dist port, the LTP itself, and then to run the tests). There are a lot of nice ideas floating around, but I have the impression we are lacking some helping hands to get various items integrated.
Wrap-Up
Various items I talked above are not sexy. Those are typically not the things people do just for fun. Those are typically items people get paid for. It would be nice if some of the companies which benefit from FreeBSD would be so nice to lend a helping hand for one or another item. Maybe the FreeBSD Foundation has some contacts they could ask about this?
It could also be that for some of the items I mentioned here there is more ongoing that I know of. This means then that the corresponding work could be made more known on the mailinglists. When it is more known, maybe someone wants to provide a helping hand.
Send to Kindle
