Example of using add_custom_command and add_custom_target together in CMake to handle custom build steps with minimal rebuilding: This example untars library headers for an INTERFACE library target

CMakeLists.txt

set(LIBFOO_TAR_HEADERS
  "${CMAKE_CURRENT_BINARY_DIR}/include/foo/foo.h"
  "${CMAKE_CURRENT_BINARY_DIR}/include/foo/foo_utils.h"
)

add_custom_command(OUTPUT ${LIBFOO_TAR_HEADERS}
  COMMAND ${CMAKE_COMMAND} -E tar xzf "${CMAKE_CURRENT_SOURCE_DIR}/libfoo/foo.tar"
  COMMAND ${CMAKE_COMMAND} -E touch ${LIBFOO_TAR_HEADERS}
  WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/include/foo"
  DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/libfoo/foo.tar"
  COMMENT "Unpacking foo.tar"
  VERBATIM
)

add_custom_target(libfoo_untar DEPENDS ${LIBFOO_TAR_HEADERS})

add_library(foo INTERFACE)
target_include_directories(foo INTERFACE "${CMAKE_CURRENT_BINARY_DIR}/include/foo")
target_link_libraries(foo INTERFACE ${FOO_LIBRARIES})

awesome

Nice example. I think add_dependencies(foo libfoo_untar) is needed? Or write add_custom_target(libfoo_untar ALL DEPENDS ${LIBFOO_TAR_HEADERS}).

This was helpful, thanks! And yes @fd3kyt, I had to use the add_dependencies for it to work. thanks!

In my project I'd like to just have the foo target and not the libfoo_untar target. Could I get that somehow?

Wow! It's so simple... where has this been all my life!

Hey, CMake, here's how to generate 'these files', they depend on 'those files'.
And I create this target whose goal in life will be to cause these files to be generated if they need to be.

In my project I'd like to just have the foo target and not the libfoo_untar target. Could I get that somehow?

That's what I wanted to do and what seemed the most straightforward way of doing things: "Hey CMake, here is a group of files and here is the group of files they depend on, and here is how to generate them. That's it that's all for those files and how and when to generate them.

After that I should be able to say that some target depends on that file and CMake should know what to do.

The documentation seems to say at first that you could do that ...

A target created in the same directory (CMakeLists.txt file) that specifies any output of the custom command as a source file is given a rule to generate the file using the command at build time.

but goes on to say not to do that if you have multiple targets that depend on one of those files.

Do not list the output in more than one independent target that may build in parallel or the two instances of the rule may conflict (instead use the add_custom_target() command to drive the command and make the other targets depend on that one).

From that my takeaway is that I will always create a target and not have other targets depend directly on the files.

Oh and as for "can I get that somehow?", the answer seems to be yes, but I was trying that at first and could not for the life of me get it to work. Just a simple script generates an output file from an input and I couldn't get making the target to cause the script to be called to generate the files. They were not in the same directory which is why I couldn't get it to work (I think) but either way, the way he's doing seems to be the 'right' way.

I've used your thing in many places with different files and different commands.

I applied it to one situations where the files are all generated the same way but updating one of the dependencies would cause the whole group of files to be regenerated. Given our situation, you may modify the generated file and see how that goes, then if pleased, you modify the input file.

So I ended up doing this:

A macro called add_something that creates a target.

One could create such an add_macro for each command that they use to generate things.

macro(add_some_kind_of_target target_name infile outfile)
    add_custom_command(
        OUTPUT ${outfile}
        COMMAND my_script.sh --in ${infile} --out ${outfile}
        DEPENDS ${infile} my_script.sh
    )
    add_custom_target(${target_name} DEPENDS ${outfile})
endmacro()

SET(INPUT_FILES
        "file1.in"
        "file2.in"
        "file3.in"
)

add_custom_target(group_target)

foreach(infilename ${INPUT_FILES})
    string(REGEX REPLACE ".in\$" ".out" outfilename ${infilename})
    set(infile_path ${CMAKE_CURRENT_SOURCE_DIR}/${infilename})
    set(outfile_path ${CMAKE_CURRENT_BINARY_DIR}/${outfilename})

    add_some_kind_of_target(${outfilename} ${infile_path} ${outfile_path})
    add_dependencies(group_target ${outfilename})

    INSTALL(FILES ${db_out_file} DESTINATION share/sqlite/database/BD)
endforeach()

The macro is more general, and in the foreach, I use it in a specific way. I chose to use the filename of the output file as the name of the target. I also specified INSTALLs for the output files. Also, one cool thing is the DEPENDS part in the macro also lists the generator script.

@PhilippeCarphin Hi, could you please tell me why we need add_custom_target here? I think only the command add_custom_command can do what we want to. Thank you so much!

@crownk1997

@PhilippeCarphin Hi, could you please tell me why we need add_custom_target here? I think only the command add_custom_command can do what we want to. Thank you so much!

It is to create a target whose goal in life is to just depend on the files generated by add_custom_command. You don't need the add_custom_target but you should use it.

The rule to generate the files (created with add_custom_command) is more simple, "here's how to generate these files if they are not there" and doesn't benefit from CMake's dependency resolution magic.

What the documentation says is that because these rules are dumber, if two targets depend on these files and you are building with -j ${n} where n > 1, then those targets may cause those files to be generated twice concurrently.

Having the custom target and having only dependencies on the target and not on the files gives you a protection against that kind of thing.

@crownk1997 To me `add_custom_command" is like "if the target depends on this file and it's not there, just run this command and don't ask questions" which is why you only want one target causing the "run this command and don't ask questions".

It's the custom target that asks the questions.

I was guided here looking for a solution to just such a make -j collision because add_custom_command runs twice - and the second one throws an error if you are trying to auto-generate docs using Doxygen followed by Sphinx. (both are custom_command + custom_target combos).

Initially, it seemed like there should be an alternative to add_custom_target that doesn't run more than once. ALL seems pretty forceful, since it doesn't have a dependency to trigger it. Now I see that it was a dependency in the Sphinx add_custom_command - asking for a file directly rather than Doxygen that caused the error. Changing the file to the "Doxygen" target created by add_custom_command fixed the problem.

How could this be changed if I didn't know the header files included in the tar? Basically if I want LIBFOO_TAR_HEADERS to be populated in some more automatic way. I struggled with file(GLOB..) but didn't work.

What I would do is untar the thing, run a find command in my shell, copy-it and paste like this

set(LIB_FOO_HEADERS
# Paste here
)

I prefer this over using GLOB. You can read up on the GLOB vs no-GLOB in debate for CMake.

For me it boils down to the fact that if I add a file, I have to go write it somewhere in a CMakeLists.txt which ensures that when I run make, cmake will be re-run. This is not the case with globs.

Sorry if you're disappointed with my "copy-paste some shell output in your CMakeLists.txt" answer but that is what I do 😄

First of all thanks for your answer. Your proposal is the best/single working solution that I have up to now. 😄
Yeah, GLOB can hide a lot of traps.
I was trying to do the find and copy-paste trick you mentioned a bit more automatically via a custom command or target, but the problem was that I couldn't assign it's output to a CMake variable. 🤕

There's a parameter to make the output of execute_process go into a variable (or to make the output of a custom command, I forget which) but it's not worth it.

Plus you would have to make sure that this automatic process is triggered when you add a new file anyway by remembering to run CMake when you add a file.

With the files written explicitely, you have to remember to add the file in a CMakeLIsts.txt somewhere but that will make sure the the necessary things get triggered.

I agree with you regarding the execute_process solution. I also like the idea of writing explicitly the files in my case now that you put it that way.

It is a good example.
if the process logic is too complicate, I have a idea about wrap some logic code to script.cmake, then the code will be more clean.
for example:

CMakeList.txt

set(InputFileList "config.h" "apis.h")
set(OutputFileName "${CMAKE_CURRENT_BINARY_DIR}/SingleHeadFile.h")

add_custom_command(OUTPUT ${OutputFileName}
COMMAND ${CMAKE_COMMAND} -P "${PROJECT_SOURCE_DIR}/cmake/GenerateSingleFile.cmake" "${InputFileList}" "${OutputFileName}"
WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}"
DEPENDS ${InputFileList}
COMMENT "Generate single-head-file from input-file-list."
VERBATIM
)
add_library(Foo INTERFACE ${OutputFileName})

GenerateSingleFile.cmake

-- your logic code here

Helps a lot. Thanks :)

So I ended up doing this: A macro called add_something that creates a target [ combining add_custom_command + add_custom_target ]

Every time I do NOT find something like this macro in plain CMake it makes me feel like I still don't understand CMake.

But no, it's apparently CMake that does not understand the most basic feature people need.

Best page found so far:
https://samthursfield.wordpress.com/2015/11/21/cmake-dependencies-between-targets-and-files-and-custom-commands/

Thank you for posting this. I am perplexed about one thing though. add_library and add_executable know that source files are present in ${CMAKE_CURRENT_SOURCE_DIR} so we don't need to explicitly mention it. But, add_custom_command needs the full path to a file in the current source directory. So,

add_custom_command(OUTPUT cross_compile_file.o
${CXX_CROSS_COMPILER} -c cross_compiled_file.cpp
MAIN_DEPENDENCY cross_compiled_file.cpp
VERBATIM)

results in no input files: cannot find cross_compiled_file.cpp error from the compiler. The behavior of cmake seems inconsistent here.
I guess the reasoning is that the custom_command could be anything so cmake isn't going to assume anything.