XDG Base Directory Specification

The XDG Base Directory Specification defines an organized folder and file structure for applications to store associated user configuration, cache, data, state, and runtime information on UNIX-like systems. This allows for consistency across different programs and desktops. Consistency is key because, without the specification, we end up with messy and disorganized dotfiles.

The goal of this article is to explain what XDG Base Directory Specification is, why the specification is important, and how you can make use of it.

Table of Contents

History
Quick Start
Gems
- XDG
- Runcom
- Etcher
- Sod
- Gemsmith
Conclusion

History

Version 0.1.0 of the specification was first published on August 10, 2003 and has been in use for multiple decades. As of this writing, Version 0.8.0 is the latest.

Recently, there’s been some buzz around the Dot Config project. The enthusiasm is great but limited in scope because the Dot Config project only focuses on configuration information when the XDG Base Directory Specification deals with configuration, cache, data, state, and runtime information. Unfortunately, having a configuration only project detracts from the greater benefits — and overarching architecture — of the XDG Base Directory Specification.

Quick Start

The specification defaults, simply speaking, are:

# Cache
$HOME/.cache

# Configuration
$HOME/.config

# Data
$HOME/.local/share

# State
$HOME/.local/state

The above gives you a nice way to organize, store, and use the following information associated with your application. Here is a breakdown of the categories:

Cache: Where you store user specific cache for improved application performance. This can be an information that doesn’t change often but, once stored, helps keep your application performant.
Configuration: Where you store user specific configuration information so users of your application can customize behavior as best suites their workflow. This can be in the form of INI, TOML, YAML, and other formats.
Data: Where user specific data is stored for use by your application. This could be a SQLite database or any information that needs more permanent storage for your application.
State: Where user specific state data is stored for use by your application. This state-specific data allows your application to restore itself to a working state prior to crash, power failure, or any situation that might require your application to quickly boot itself back into a working state.
Runtime: Although, not shown above, this is for user specific runtime information that doesn’t have a distinct location and varies depending on implementation needs. Runtime information comes with special requirements and considerations so make sure to read the specification for details.

With the above, you have a clean — and well organized — folder structure for managing associated applications cache, config, data, and state information. To illustrate further, take a look at your own dotfiles. For example, here’s an example of my Dotfiles in the root of my $HOME directory which don’t adhere to the XDG Base Directory Specification:

Ugly, right? Well, the above occurs because so many of these applications don’t adhere to the XDG Base Directory Specification. To clean this up, these applications would only need to use a $HOME/.config/<application>/configuration.yml structure. The <application> sub-folder is important for the following reasons:

Keeps the XDG .config folder organized so each application is clearly identified. This also applies for .cache, .local, and any runtime information you might have associated with your application.
Allows each application to manage it’s own sub-folder (i.e. namespace) so the application can manage multiple files and/or additional sub-folders as desired.

By the way, the configuration doesn’t have to be YAML but could INI, TOML, or whatever format best suites the needs of the application. For example, here’s my $HOME/.config folder structure which shows applications which leverage the XDG Base Directory Specification:

Much nicer, right? The only oddity is Amazing Print which should use $HOME/.config/amazing_print/configuration.rb instead of $HOME/.config/aprc for all of the reasons mentioned earlier.

Now that you have a sense of what and why the XDG Base Directory Specification is important, let’s look at how you can use this specification to build better software applications. For the purposes of this article I’m going to stick with the Ruby programming language but any language is capable of adhering to the specification. To keep things simple, I’ll also focus on XDG configurations but you can apply the same methodology to cache, data, state, and runtime information too.

Gems

Over the years I’ve engineered several Ruby gems that allow you to quickly build sophisticated applications which adhere to the XDG Base Directory Specification. There is also an architectural hierarchy to these gems so you can start with the basics and work your way up to more complex scenarios depending on your needs.

The following is by no means exhaustive but is meant to be a high level overview. You’ll definitely want to read the documentation of each gem to learn more. That said, the foundation begins with the XDG gem.

XDG

The XDG gem is a pure Ruby implementation of the XDG Base Directory Specification and is a great place to get started. Also, if you’d like to read the specification in a different format, the XDG documentation might be a more enjoyable read. Here’s what the object API looks like:

xdg = XDG.new
xdg.cache_home   # Answers computed `$XDG_CACHE_HOME` value.
xdg.config_home  # Answers computed `$XDG_CONFIG_HOME` value.
xdg.config_dirs  # Answers computed `$XDG_CONFIG_DIRS` value.
xdg.data_home    # Answers computed `$XDG_DATA_HOME` value.
xdg.data_dirs    # Answers computed `$XDG_DATA_DIRS` value.
xdg.state_home   # Answers computed `$XDG_STATE_HOME` value.

With this gem, you can leverage all aspects of the XDG Base Directory Specification. Definitely dive into the documentation to learn more about the gem and the specification.

Moving up a layer in the stack, is the Runcom gem which adds additional superpowers beyond what the XDG gem provides.

Runcom

The Runcom gem builds upon the XDG gem by giving you dynamic local and global detection. Specifically, this means if you have a project specific configuration, Runcom will load the local project configuration first instead of the global XDG configuration (default). To illustrate further, here’s a concrete example:

# Global (XDG) Milestoner configuration.
$HOME/.config/milestoner/configuration.yml

# Local (Runcom) Milestoner configuration for the "demo" project.
$HOME/Engineering/demo/.config/milestoner/configuration.yml

In the above example, Runcom will detect and load the local Milestoner configuration for the demo project instead of the global configuration. This happens because we use the .config directory which adheres to the XDG Base Directory Specification but Runcom will first check for relative path to current directory before falling back to global configuration.

Use of the milestoner sub-directory also helps illustrate how configurations are organized per application. This also means additional configuration information could be stored in the milestoner folder if need be. In this case, we are only configuring the Milestoner gem for our demo project and no other files are necessary.

Even though the above example is configuration based, this applies for cache, data, and state too. Definitely check out the documentation to learn more.

So far we’ve talked about the XDG and Runcom gems which deal with the finding of associated application files but what if you want to manipulate these files further? This is where the Etcher gem comes into play.

Etcher

The Etcher gem takes what is provided by the XDG and Runcom gems and allows you to dynamically make adjustments when loading the associated application’s configuration, cache, data, state, and/or runtime information. This is done through a series of steps which allows you to load, transform, override, validate, and model the data being loaded. Here’s a simple example of a Bundler Inline script with a YAML loader:

#! /usr/bin/env ruby
# frozen_string_literal: true

# Save as `demo`, then `chmod 755 demo`, and run as `./demo`.

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "debug"
  gem "etcher"
  gem "refinements"
  gem "runcom"
end

using Refinements::Pathname

Pathname.make_temp_dir do |root|
  configuration = Runcom::Config.new root.join("configuration.yml").write("label: Test")
  registry = Etcher::Registry[loaders: [Etcher::Loaders::YAML.new(configuration.active)]]
  etcher = Etcher.new registry

  puts etcher.call
end

When you save and run the above script, you’ll find the following result yielded: Success({label: "Test"}). Notice the result is wrapped as a Success (monad) which means you’ll either get a Success or a Failure which gives you additional firepower for crafting fault tolerant loading of application information. There is a lot the Etcher gem can do so check out the documentation to learn more.

Sod

The Sod gem provides a Domain Specific Language (DSL) for building Command Line Interfaces (CLI) and definitely plays well with the XDG, Runcom, and Etcher gems we’ve discussed thus far. To begin, consider this Bundler Inline script:

#! /usr/bin/env ruby
# frozen_string_literal: true

# Save as `demo`, then `chmod 755 demo`, and run as `./demo`.

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "debug"
  gem "refinements"
  gem "runcom"
  gem "sod"
end

using Refinements::Pathname

Pathname.make_temp_dir do |root|
  context = Sod::Context[
    defaults_path: "defaults.yml",
    xdg_config: Runcom::Config.new(root.join("configuration.yml").write("label: Test"))
  ]

  cli = Sod.new banner: "Demo 0.0.0: A demonstration." do
    on(Sod::Prefabs::Commands::Config, context:)
    on Sod::Prefabs::Actions::Version, "Demo 0.0.0"
    on Sod::Prefabs::Actions::Help, self
  end

  cli.call
end

When you save and run the above as ./demo, you’ll get helpful information on how to use the newly crafted demo application:

Even better — much like what we were doing earlier with the Etcher gem — you can view the configuration by following the help text and running ./demo config --view:

The temporary directory, while ugly, is only being used for demonstration and automatic cleanup purposes so you can play with the script with minimal effort. Normally, you’d adhere to the XDG Base Directory Specification by using $HOME/.config/demo/configuration.yml instead of using the configuration.yml in the temporary directory shown above.

Gemsmith

So far, we’ve covered a lot of ground by starting with the XDG gem and working our way up to the Sod gem. Each step of the way we’ve explored using the XDG Base Directory Specification. As final step on this journey, you can use the Gemsmith to tie all of this together by scaffolding a fully functional CLI gem of your own with all of this power baked in for you via a single command in your terminal. For example, the following one-liner will build a CLI gem for you:

gemsmith build --name demo --cli
cd demo
gemsmith --install
demo config

The above will create a demo project in your current directory with CLI support using the XDG Base Directory Specification via the XDG, Runcom, Etcher, and Sod gems for you. Then you can proceed to install and play with the gem. To uninstall, run: gem uninstall demo.

How’s that for minimal effort! Definitely, check out the Gemsmith documentation to learn more, though.

Conclusion

During the course of this article, I hope you’ve enjoyed learning about the XDG Base Directory Specification and learned how you can take the specification and build more sophisticated applications via the following gem hierarchy:

While the above has been mostly CLI focused, I want to emphasize none of this has to be specific to CLIs. Graphical user interfaces, web applications, or any application that needs to manage user specific information associated with your application is perfectly suited for adhering to the XDG Base Directory Specification.

When you leverage the specification, you not only make life more organized for your users but we all benefit for promoting powerful standards that makes our engineering community better as a whole. 🙇🏻