[makefile] Create directories using make file

I'm a very new to makefiles and i want to create directories using makefile. My project directory is like this

+--Project  
   +--output  
   +--source  
     +Testfile.cpp  
   +Makefile  

I want to put all the objects and output into the respective output folder. I want to create folder structure which would be like this after compiling.

+--Project
   +--output
     +--debug (or release)
       +--objs
         +Testfile.o
       +Testfile (my executable file)
   +--source
     +Testfile.cpp
   +Makefile

I tried with several options, but could not succeed. Please help me to make directories using make file. I'm posting my Makefile for your consideration.

#---------------------------------------------------------------------
# Input dirs, names, files
#---------------------------------------------------------------------
OUTPUT_ROOT := output/

TITLE_NAME := TestProj 

ifdef DEBUG 
    TITLE_NAME += _DEBUG
else
ifdef RELEASE
    TITLE_NAME += _RELEASE
endif
endif


# Include all the source files here with the directory tree
SOURCES := \
        source/TestFile.cpp \

#---------------------------------------------------------------------
# configs
#---------------------------------------------------------------------
ifdef DEBUG
OUT_DIR     := $(OUTPUT_ROOT)debug
CC_FLAGS    := -c -Wall
else
ifdef RELEASE
OUT_DIR     := $(OUTPUT_ROOT)release
CC_FLAGS    := -c -Wall
else
$(error no build type defined)
endif
endif

# Put objects in the output directory.
OUT_O_DIR   := $(OUT_DIR)/objs

#---------------------------------------------------------------------
# settings
#---------------------------------------------------------------------
OBJS = $(SOURCES:.cpp=.o)
DIRS = $(subst /,/,$(sort $(dir $(OBJS))))
DIR_TARGET = $(OUT_DIR)

OUTPUT_TARGET = $(OUT_DIR)/$(TITLE_NAME)

CC_FLAGS +=   

LCF_FLAGS := 

LD_FLAGS := 

#---------------------------------------------------------------------
# executables
#---------------------------------------------------------------------
MD := mkdir
RM := rm
CC := g++

#---------------------------------------------------------------------
# rules
#---------------------------------------------------------------------
.PHONY: all clean title 

all: title 

clean:
    $(RM) -rf $(OUT_DIR)

$(DIR_TARGET):
    $(MD) -p $(DIRS)

.cpp.o: 
    @$(CC) -c $< -o $@

$(OBJS): $(OUT_O_DIR)/%.o: %.cpp
    @$(CC) -c $< -o $@

title: $(DIR_TARGET) $(OBJS)

Thanks in advance. Please guide me if i made any mistakes also.

In my opinion, directories should not be considered targets of your makefile, either in technical or in design sense. You should create files and if a file creation needs a new directory then quietly create the directory within the rule for the relevant file.

If you're targeting a usual or "patterned" file, just use make's internal variable $(@D), that means "the directory the current target resides in" (cmp. with $@ for the target). For example,

$(OUT_O_DIR)/%.o: %.cpp
        @mkdir -p $(@D)
        @$(CC) -c $< -o $@

title: $(OBJS)

Then, you're effectively doing the same: create directories for all $(OBJS), but you'll do it in a less complicated way.

The same policy (files are targets, directories never are) is used in various applications. For example, git revision control system doesn't store directories.

Note: If you're going to use it, it might be useful to introduce a convenience variable and utilize make's expansion rules.

dir_guard=@mkdir -p $(@D)

$(OUT_O_DIR)/%.o: %.cpp
        $(dir_guard)
        @$(CC) -c $< -o $@

$(OUT_O_DIR_DEBUG)/%.o: %.cpp
        $(dir_guard)
        @$(CC) -g -c $< -o $@

title: $(OBJS)

given that you're a newbie, I'd say don't try to do this yet. it's definitely possible, but will needlessly complicate your Makefile. stick to the simple ways until you're more comfortable with make.

that said, one way to build in a directory different from the source directory is VPATH; i prefer pattern rules

Or, KISS.

DIRS=build build/bins

... 

$(shell mkdir -p $(DIRS))

This will create all the directories after the Makefile is parsed.

I've just come up with a fairly reasonable solution that lets you define the files to build and have directories be automatically created. First, define a variable ALL_TARGET_FILES that holds the file name of every file that your makefile will be build. Then use the following code:

define depend_on_dir
$(1): | $(dir $(1))

ifndef $(dir $(1))_DIRECTORY_RULE_IS_DEFINED
$(dir $(1)):
    mkdir -p $$@

$(dir $(1))_DIRECTORY_RULE_IS_DEFINED := 1
endif
endef

$(foreach file,$(ALL_TARGET_FILES),$(eval $(call depend_on_dir,$(file))))

Here's how it works. I define a function depend_on_dir which takes a file name and generates a rule that makes the file depend on the directory that contains it and then defines a rule to create that directory if necessary. Then I use foreach to call this function on each file name and eval the result.

Note that you'll need a version of GNU make that supports eval, which I think is versions 3.81 and higher.

See https://www.oreilly.com/library/view/managing-projects-with/0596006101/ch12.html

REQUIRED_DIRS = ...
_MKDIRS := $(shell for d in $(REQUIRED_DIRS); \
             do                               \
               [[ -d $$d ]] || mkdir -p $$d;  \
             done)

$(objects) : $(sources)

As I use Ubuntu, I also needed add this at the top of my Makefile:

SHELL := /bin/bash # Use bash syntax

All solutions including the accepted one have some issues as stated in their respective comments. The accepted answer by @jonathan-leffler is already quite good but does not take into effect that prerequisites are not necessarily to be built in order (during make -j for example). However simply moving the directories prerequisite from all to program provokes rebuilds on every run AFAICT. The following solution does not have that problem and AFAICS works as intended.

MKDIR_P := mkdir -p
OUT_DIR := build

.PHONY: directories all clean

all: $(OUT_DIR)/program

directories: $(OUT_DIR)

$(OUT_DIR):
    ${MKDIR_P} $(OUT_DIR)

$(OUT_DIR)/program: | directories
    touch $(OUT_DIR)/program

clean:
    rm -rf $(OUT_DIR)

OS independence is critical for me, so mkdir -p is not an option. I created this series of functions that use eval to create directory targets with the prerequisite on the parent directory. This has the benefit that make -j 2 will work without issue since the dependencies are correctly determined.

# convenience function for getting parent directory, will eventually return ./
#     $(call get_parent_dir,somewhere/on/earth/) -> somewhere/on/
get_parent_dir=$(dir $(patsubst %/,%,$1))

# function to create directory targets.
# All directories have order-only-prerequisites on their parent directories
# https://www.gnu.org/software/make/manual/html_node/Prerequisite-Types.html#Prerequisite-Types
TARGET_DIRS:=
define make_dirs_recursively
TARGET_DIRS+=$1
$1: | $(if $(subst ./,,$(call get_parent_dir,$1)),$(call get_parent_dir,$1))
    mkdir $1
endef

# function to recursively get all directories 
#     $(call get_all_dirs,things/and/places/) -> things/ things/and/ things/and/places/
#     $(call get_all_dirs,things/and/places) -> things/ things/and/
get_all_dirs=$(if $(subst ./,,$(dir $1)),$(call get_all_dirs,$(call get_parent_dir,$1)) $1)

# function to turn all targets into directories
#     $(call get_all_target_dirs,obj/a.o obj/three/b.o) -> obj/ obj/three/
get_all_target_dirs=$(sort $(foreach target,$1,$(call get_all_dirs,$(dir $(target)))))

# create target dirs
create_dirs=$(foreach dirname,$(call get_all_target_dirs,$1),$(eval $(call make_dirs_recursively,$(dirname))))

TARGETS := w/h/a/t/e/v/e/r/things.dat w/h/a/t/things.dat

all: $(TARGETS)

# this must be placed after your .DEFAULT_GOAL, or you can manually state what it is
# https://www.gnu.org/software/make/manual/html_node/Special-Variables.html
$(call create_dirs,$(TARGETS))

# $(TARGET_DIRS) needs to be an order-only-prerequisite
w/h/a/t/e/v/e/r/things.dat: w/h/a/t/things.dat | $(TARGET_DIRS)
    echo whatever happens > $@

w/h/a/t/things.dat: | $(TARGET_DIRS)
    echo whatever happens > $@

For example, running the above will create:

$ make
mkdir w/
mkdir w/h/
mkdir w/h/a/
mkdir w/h/a/t/
mkdir w/h/a/t/e/
mkdir w/h/a/t/e/v/
mkdir w/h/a/t/e/v/e/
mkdir w/h/a/t/e/v/e/r/
echo whatever happens > w/h/a/t/things.dat
echo whatever happens > w/h/a/t/e/v/e/r/things.dat

make in, and off itself, handles directory targets just the same as file targets. So, it's easy to write rules like this:

outDir/someTarget: Makefile outDir
    touch outDir/someTarget

outDir:
    mkdir -p outDir

The only problem with that is, that the directories timestamp depends on what is done to the files inside. For the rules above, this leads to the following result:

$ make
mkdir -p outDir
touch outDir/someTarget
$ make
touch outDir/someTarget
$ make
touch outDir/someTarget
$ make
touch outDir/someTarget

This is most definitely not what you want. Whenever you touch the file, you also touch the directory. And since the file depends on the directory, the file consequently appears to be out of date, forcing it to be rebuilt.

However, you can easily break this loop by telling make to ignore the timestamp of the directory. This is done by declaring the directory as an order-only prerequsite:

# The pipe symbol tells make that the following prerequisites are order-only
#                           |
#                           v
outDir/someTarget: Makefile | outDir
    touch outDir/someTarget

outDir:
    mkdir -p outDir

This correctly yields:

$ make
mkdir -p outDir
touch outDir/someTarget
$ make
make: 'outDir/someTarget' is up to date.

TL;DR:

Write a rule to create the directory:

$(OUT_DIR):
    mkdir -p $(OUT_DIR)

And have the targets for the stuff inside depend on the directory order-only:

$(OUT_DIR)/someTarget: ... | $(OUT_DIR)