GNU Make and Linux file permissions

[aryan_]

Hi All,

I am recently having problems with the compilation of my C++ source files with make.

The modifications to the source files are recognized and, hence, the sources get re-compiled successfully in a certain machine while the modifications can not be recognized at all on another machine, and the old object files are kept being used to create the binaries.

git is used for version control. In our configuration, even though the machines where software development takes place can pull from each other, one server machine is used as a centric repository from which everybody pulls.

I am suspecting that, after pulling from git, somehow, Linux file permissions do not allow the developer to compile the source successfully.

Any ideas are appreciated.

Thanks.

[Rubberman]

This sort of problem often happens when the clocks of the various systems are out of sync. Have you configured all of your systems to sync their clock with ntp?

[aryan_]

This sort of problem often happens when the clocks of the various systems are out of sync. Have you configured all of your systems to sync their clock with ntp?

Can you please provide more details about this sync issue, and its impact on this compilation problem ?

Thanks.

[Rubberman]

If you don't keep your systems in time-sync then one can modify and build a module, yet awhile later someone on another system modifies some of the source, yet because the timebase on the later system is set earlier than the first one, it is possible that the timestamp on the modified file is earlier than the timestamp on the binary file. As a result, make thinks the build is up-to-date and it doesn't rebuild the file and dependencies. This is a common problem when you have multiple people working on the same set of souces, or even one person who is working on multiple machines (it does happen!), if you aren't running ntp with a common timebase. You can sync to a single server (or group of servers) on your local network, and those time servers can sync to a timebase on the internet, such as NIST.

[aryan_]

If you don't keep your systems in time-sync then one can modify and build a module, yet awhile later someone on another system modifies some of the source, yet because the timebase on the later system is set earlier than the first one, it is possible that the timestamp on the modified file is earlier than the timestamp on the binary file. As a result, make thinks the build is up-to-date and it doesn't rebuild the file and dependencies.

This for sure makes sense.

However, my case looks different.

Even after cloning a git repo from scratch and modifying some source files, "make" does not recognize the modifications, and hence does not re-build them. (git repo clone does not contain any binaries)

Do you think that the problem is with Makefiles themselves (dependencies, etc) ?

[Rubberman]

It is possible that the timestamps on the files you are git'ing are bogus. What happens if you touch all of the files?

[aryan_]

It is possible that the timestamps on the files you are git'ing are bogus. What happens if you touch all of the files?

If I touch all files, 'make' SEEMS to detect the timestamp changes for all touched files, and re-build the object files. However, I can not see the changes in program output.

Here are my Makefiles:

Makefile (I call 'make' for this file):

all:
	make -f Makefile_Ses
	make -f Makefile_Repo
	make -f Makefile_Code

clean:
	make -f Makefile_Ses clean
	make -f Makefile_Repo clean
	make -f Makefile_Code clean

Makefile_Ses:

LIB_DIR = ../common/lib
OBJ_DIR = ../common/src
SRC_DIR = ../common/src
INCLUDE_DIR = ../common/inc
INCLUDE_DIR_BOOST = ../../third_party

#build ses library
SRC_MODULES = Ses.cpp Tcp.cpp Data.cpp Str.cpp
OBJ_MODULES = $(patsubst %.cpp, %.o, ${SRC_MODULES})

SRC_FILES = $(addprefix $(SRC_DIR)/, $(SRC_MODULES))
OBJ_FILES = $(addprefix $(OBJ_DIR)/, $(OBJ_MODULES))
LIB_ST = $(LIB_DIR)/libSes.a

DEBUG = -g
CFLAGS = -Wall -c $(DEBUG) -I $(INCLUDE_DIR) -I $(INCLUDE_DIR_BOOST)

$(LIB_ST): $(OBJ_FILES)
	ar -cru $(LIB_ST) $(OBJ_FILES)

$(OBJ_DIR)/%.o:$(SRC_DIR)/%.cpp
	gcc ${CFLAGS} -o $@ $<

clean:
	rm -f $(OBJ_FILES)
	rm -f $(LIB_DIR)/*

Makefile_Repo:

BASE_DIR = $(HOME)
LIB_DIR = ../common/lib
OBJ_DIR = ./src
SRC_DIR = ./src
INCLUDE_DIR_REPO = ./inc

#build ses library
SRC_MODULES = Entry.cpp
OBJ_MODULES = $(patsubst %.cpp, %.o, ${SRC_MODULES})

SRC_FILES = $(addprefix $(SRC_DIR)/, $(SRC_MODULES))
OBJ_FILES = $(addprefix $(OBJ_DIR)/, $(OBJ_MODULES))
LIB_ST = $(LIB_DIR)/libRepo.a

DEBUG = -g
CFLAGS = -Wall -c $(DEBUG) -I $(INCLUDE_DIR_REPO)

$(LIB_ST): $(OBJ_FILES)
	ar -cru $(LIB_ST) $(OBJ_FILES)

$(OBJ_DIR)/%.o:$(SRC_DIR)/%.cpp
	gcc ${CFLAGS} -o $@ $<

clean:
	rm -f $(OBJ_FILES)
	rm -f $(LIB_DIR)/*

Makefile_Code:

LIB_DIR = ../common/lib
OBJ_DIR = ./src
SRC_DIR = ./src
INCLUDE_DIR_CODE = ./inc
INCLUDE_DIR_SES = ../common/inc
INCLUDE_DIR_BOOST = ../../third_party

#build ses library
SRC_MODULES = Ses.cpp RegisterPlugin.cpp HeaderPrs.cpp Prs.cpp StdPrs.cpp PrsBase.cpp Js.cpp
OBJ_MODULES = $(patsubst %.cpp, %.o, ${SRC_MODULES})

SRC_FILES = $(addprefix $(SRC_DIR)/, $(SRC_MODULES))
OBJ_FILES = $(addprefix $(OBJ_DIR)/, $(OBJ_MODULES))

LIBS = -lRepo
LIBS += -lNsConf
LIBS += -lSes
LIBS += -lclassreg
LIBS += -lstringtokenizer
LIBS += -lz

LIB_ST = libh.so

DEBUG = -g
CFLAGS = -Wall -fPIC -c $(DEBUG) -I $(INCLUDE_DIR_CODE) -I $(INCLUDE_DIR_SES) -I $(INCLUDE_DIR_BOOST)
LFLAGS = -shared -Wl

$(LIB_ST): $(OBJ_FILES)
	@echo "Creating $(LIB_ST)"
	gcc $(LFLAGS),-soname,$(LIB_ST).1 -o $(LIB_ST).1.0 $(OBJ_FILES) -L$(LIB_DIR) -L../../common/lib -L../../conf/configLib/lib $(LIBS)

$(OBJ_DIR)/%.o:$(SRC_DIR)/%.cpp
	gcc ${CFLAGS} -o $@ $<

clean:
	rm -f $(OBJ_FILES)

For example, when I modify PrsBase.cpp, which is built in Makefile_Code, I CAN observe from the output of 'make' that the modifications to PrsBase.cpp are detected, and PrsBase.cpp is re-built. However, when I execute the application which loads libh.so plugin, which is built in Makefile_Code, I can not see the expected output from PrsBase.cpp. Somehow, the changes do not go into libh.so in the line "gcc $(LFLAGS),-soname,$(LIB_ST).1 -o $(LIB_ST).1.0 $(OBJ_FILES) -L$(LIB_DIR) -L../../common/lib -L../../conf/configLib/lib $(LIBS)" in Makefile_Code.

[Rubberman]

If there is an older version of the library in your LD_LIBRARY_PATH environment variable (if not defined, default is /usr/lib or /usr/lib64), then you need to change your LD_LIBRARY_PATH so that the directory where the new .so is found is first in the path specification. Also, it's possible that you are executing the wrong application file, depending upon your PATH environment. Finally, if you need to really make sure that the executable is getting the correct library, then build a static library (libname.a) and force link that with the -static linker option to force it to link with the static library instead.

[aryan_]

If there is an older version of the library in your LD_LIBRARY_PATH environment variable (if not defined, default is /usr/lib or /usr/lib64), then you need to change your LD_LIBRARY_PATH so that the directory where the new .so is found is first in the path specification. Also, it's possible that you are executing the wrong application file, depending upon your PATH environment. Finally, if you need to really make sure that the executable is getting the correct library, then build a static library (libname.a) and force link that with the -static linker option to force it to link with the static library instead.

I finally resolved the problem.

First of all, the problem had nothing to do with Makefiles.

The machine where these source files are compiled using 'make' had two separate users on it both of whom were developing software for the same project.

In our case, all binaries are invoked via a shell script with root priviliges, which, right at the beginning, checks /etc/ld.so.conf file to see if $HOME/project_folder/plugin_libs line exists. If not, it adds it into /etc/ld.so.conf, and run 'ldconfig'.

Since first user had already added $HOME/project_folder/plugin_libs line to ld.so.conf when the second user checks the existence of $HOME/project_folder/plugin_libs line with a different $HOME value, another $HOME/project_folder/plugin_libs line with a different $HOME value
is added to ld.so.conf.

Since the first appearing line in ld.so.conf has presedence over the the line coming after it, the plugins from the other user's project folder kept being loaded - the modified ones were ignored.