Makefiles

Makefiles are an indispensable tool when it comes to standardizing things like build processes, and certain automation tasks. The Makefile and gnu make at this point is over 40 years old, and while its got a few hiccups and is a bit rough around the edges, its quite widely adopted and allows developers to work seamlessly across projects simply due to the presence of the humble makefile.

What I aim to do in this blog post is simple. I will try to introduce as many higher level features of make that will allow you to bring much needed sanity when writing makefiles.

Define directive

one of the main things a makefile does is execute commands under targets. When you have a makefile that calls the same commands with a different set of parameters, say when setting up a QA and a Dev environment, define is a very helpful feature to reduce duplication and make your targets DRY (don’t repeat yourself)

lets say you have the following few targets

setup/qa_postgres_db:
    ./spawn_and_seed_db --env qa --db_name qa_db

setup/dev_postgres_db:
    ./spawn_and_seed_db --env dev --db_name dev_db

you can see that the script spawn_and_seed_db is called twice, once for the qa and once for the dev environment. As an example this is fine and not too bad, but lets say you have a script with many parameters, and you want to call it with different parameters for each target, then you can use the define directive to define a block of commands that you can call with different parameters.

A simple example of a define block is:


time=`date +%s`

# define a block of commands
define setup_postgres_db
    ./spawn_and_seed_db --env "$(1)" --db_name "$(2) --start_time $(time)"
    echo "done"
endef

setup/qa_postgres_db:
    $(call setup_postgres_db, qa, qa_db)

setup/dev_postgres_db:
    $(call setup_postgres_db, dev, dev_db)

a couple of important points to notice here:

the variables are in the form of $(1), $(2) and so on
the define block can actually have multiple lines without the need to use \
the $(time) variable is picked from the make variable time defined above

this should simplify your makefiles a bit, since now you have the power of pseudo functions to call your commands with different parameters.

to read more on the define directive, check out the GNU make manual

Caution: readers maybe tempted to think that define directives work exactly like functions in programming languages, however this is simply not true. Due to the nature of scoping and parsing the makefile, it is best if readers stick to the simple format show above. Any thing else is bound to have bugs and other inconsistencies which might prove unhelpful

Helpful makefile setup

When looking around open source repos, an easy thing to notice is that they make use of makefiles a lot (pun intented :P). And generally speaking, most of these have a common makefile that contain a few commonly used targets.

Following one such open source repo, viz. the cloudposse build harness repo has quite a nice helper makefile.

Link to the file

Due to its apache 2.0 license, we can use this file for our personal use and modify it according to our needs.

Let me first put up the modified version of my file, then we can go through it in depth.

#
# Helpers - stuff that's shared between make files
#

EDITOR ?= vim

SHELL = /bin/bash

HELP_FILTER ?= .*
.DEFAULT_GOAL := help/all

green = $(shell echo -e '\x1b[32;01m$1\x1b[0m')
yellow = $(shell echo -e '\x1b[33;01m$1\x1b[0m')
red = $(shell echo -e '\x1b[33;31m$1\x1b[0m')

# Ensures that a variable is defined and non-empty
define assert-set
  @$(if $($(1)),,$(error $(1) not defined in $(@)))
endef

# Ensures that a variable is undefined
define assert-unset
  @$(if $($1),$(error $(1) should not be defined in $(@)),)
endef

test/assert-set:
	$(call assert-set,PATH)
	@echo assert-set PASS

test/assert-unset:
	$(call assert-unset,JKAHSDKJAHSDJKHASKD)
	@echo assert-unset PASS

## Help screen, set HELP_FILTER to filter out targets
help/filter:
	@printf "Available targets:\n\n"
	@$(SELF) -s help/generate | grep -E "\w($(HELP_FILTER))"

## Display help for all targets
help/all:
	@printf "Available targets:\n\n"
	@$(SELF) -s help/generate

# Generate help output from MAKEFILE_LIST
help/generate:
	@awk '/^[a-zA-Z_0-9%:\\\/-]+:/ { \
	  helpMessage = match(lastLine, /^## (.*)/); \
	  if (helpMessage) { \
	    helpCommand = $$1; \
	    helpMessage = substr(lastLine, RSTART + 3, RLENGTH); \
      gsub("\\\\", "", helpCommand); \
      gsub(":+$$", "", helpCommand); \
	    printf "  \x1b[32;01m%-35s\x1b[0m %s\n", helpCommand, helpMessage; \
	  } \
	} \
	{ lastLine = $$0 }' $(MAKEFILE_LIST) | sort -u
	@printf "\n"

lets go over it line by line

First few lines have a few variables

EDITOR - this sets the default editor when a command or cli calls up the default editor. In my case, I’ve set it to vim.

SHELL - by default, make runs all targets in the sh shell. This ensures that make will use the bash shell

HELP_FILTER - this is the filter for targets I use to show help for a specific group of targets. More on this later.

.DEFAULT_GOAL - this is a make file directive that sets the default goal to the help/all goal. This ensures that, if make is run without any target specified, the list of available targets will be shown

next up, there are a few color setups. These are copied over verbatim from the cloud posse build harness repo. In essence they will print text on the console in that particular color

Moving on, we have the define blocks that are used as a gatekeeping mechanism for required variables when calling targets

to see how they are used, see the tests test/assert-set and test/assert-unset

the test/assert-set target will call assert-set and see if the PATH variable is set. If it is not set, make will abort and next steps will not be run.

next up, we have two help/ targets. the help/all target prints out the help information of all the targets. While the help/filter will filter targets based on the value of the HELP_FILTER variable

a quick demonstration of this is to run this target on this file, and check the results

$ make help/filter HELP_FILTER=help
Available targets:

  help/all                            Display help for all targets
  help/filter                         Help screen, set HELP_FILTER to filter out targets

and finally, we have the help/generate target. Now this target is the crux of our help targets. It looks for comments on all targets which have start with a ##

a simple demonstration is the above output is listed based on the ## comment on the help targets in the makefile above

Including makefiles

Now that we have a sample makefile, lets bootstrap it such that it becomes plug and play no matter where we use it.

to do this, we simply include this file in our directory, along with our “main” Makefile in this fashion

SELF = $(MAKE)
TOPLEVEL = $(shell git rev-parse --show-toplevel)

include $(TOPLEVEL)/Makefile.helpers

## start the hugo dev server
hugo/dev_server:
	@hugo server \
	--buildDrafts \
	--disableFastRender \
	--watch --bind=0.0.0.0 \
	--baseURL=http://localhost:1313

Here, I’ve added 2 new variables:

SELF - this required for the help/ targets as shown in the helper makefile, since we use it to call the help/generate target
TOPLEVEL - this is a convinience method that only works if the makefile is placed in a git repository. I use it to have a top level path, so that it is easier to work with relative paths within the repository when referring to files or calling scripts

and then the include directive, which actually includes the helper makefile, this now gives us access to all the nice features we discussed above.

one important point to notice how I use the TOPLEVEL variable, this means that I can check in my helper makefile at the root of the repository, and then a nested makefile at any depth has access to this.

Intellij/VsCode setup

now, on their own makefiles are pretty simple, however, I run a few more tweaks that allow me to gain syntax highlighting and code completion for makefiles that arent simply named Makefiles.

when I have to break up a huge makefile, I generally prefer to have my makefiles broken by target groups, so lets say I have two projects A and B, and my targets are under two groups projectA/* and projectB/*

I generally make 2 makefiles like

Makefile.projectA
Makefile.projectB

and then import it in the main Makefile as

include Makefile.projectA
include Makefile.projectB

however, since the files are now not plain Makefile, IDEs dont provide syntax highlighting and code completion.

here’s how to set it up for intellij and vscode which in my opinion are the most used IDEs out there

note: I merely mention here a very opinionated point about making makefiles as Makefile.myotherfile format, you are free to choose whatever style fits you best, and set the associations according to that pattern

IntelliJ makefile setup

this applies to intelliJ 2021.3 at the time of writing, but in case of any future versions it should be more or less the same.

First up, please install the Makefile plugin. I assume that as a user of intellij you should have a fairly good idea how to do this, if not, there are tons of how to guides on installing plugins.

Here’s a quick refresher

hint: goto File -> Settings -> Plugins then search for markdown.

it should be the plugin supported and maintained by intellij, since that will add Makefile file associations and code completion to the IDE for Makefiles

next,

goto File -> Settings

there under the Editor settings look for File Types

here, look for GNU Makefile

and a new pattern as Makefile.*

this will enable intelliJ to recognize all such files with the makefile syntax and you can now resume your work with much more assistance than was previously possible.

VS Code

for a quick and dirty enabling, simply open your Makefile.projectA file and on the bottom row click on plain text. This should open up a simple dialog box where you can set it to makefile and vs code will recognize it as such.

for a more permanent solution, see this stack overflow question to setup your settings.json

https://stackoverflow.com/questions/29973619/how-to-make-vs-code-treat-a-file-extensions-as-a-certain-language

for the settings.json, please use this block to configure makefiles

"files.associations": {
    "Makefile.*": "makefile",
}

closing thoughts

this guide by no means is exaustive, infact it is the very opposite of exaustive. I encourage you to experiment further with makefiles and visit open source repositories and explore more different ways to work with make and makefiles. I have listed a few features and setups that I find quite useful, but naturally YMMV.

As a closing remark, I do hope you learnt something from this and enjoyed this as much as I did writing it. If you find some inaccuracies or factual / grammer / language / other errors, feel free to reach out to me so that I may include your corrections with the appropriate citation

Cheers!

Makefiles#

Define directive#

Helpful makefile setup#

Including makefiles#

Intellij/VsCode setup#

IntelliJ makefile setup#

VS Code#

closing thoughts#