Skip to content

FIX: propagation dependencies and rebuild out of date libs #2751

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 6, 2017

Conversation

gangliao
Copy link
Contributor

@gangliao gangliao commented Jul 6, 2017

Pain points:

When I implemented buddy allocator, finally I want to merge all small pieces libraries to paddle_memory for testing memory recycling effects.

if(${WITH_GPU})
  nv_library(system_allocator SRCS system_allocator.cc DEPS gflags cpu_info gpu_info)
else(${WITH_GPU})
  cc_library(system_allocator SRCS system_allocator.cc DEPS gflags cpu_info)
endif(${WITH_GPU})

cc_test(system_allocator_test SRCS system_allocator_test.cc DEPS system_allocator)

cc_library(meta_data SRCS meta_data.cc)

cc_library(meta_cache SRCS meta_cache.cc)

cc_library(memory_block SRCS memory_block.cc)

cc_library(buddy_allocator SRCS buddy_allocator.cc DEPS glog)

cc_library(memory SRCS memory.cc)

cc_library(paddle_memory
    DEPS
    memory meta_data
    meta_cache memory_block
    buddy_allocator system_allocator)

cc_test(memory_test SRCS memory_test.cc DEPS place paddle_memory)

I found the original generic implementation doesn't allow propagation dependencies and also cannot track out of date records.

The new features for merging static libraries.

  1. propagation dependencies
  2. rebuild the out of date static library

image

@gangliao gangliao changed the title FIX: propagation dependencies and out of date rebuild FIX: propagation dependencies and rebuild out of date libs Jul 6, 2017
set(outlibfile "$<TARGET_FILE:${TARGET_NAME}>")

foreach(lib ${libs})
add_custom_command(TARGET ${TARGET_NAME} POST_BUILD
COMMAND ${CMAKE_AR} ru ${outlibfile} @"../${lib}.objlist"
WORKING_DIRECTORY ${lib}.objdir)
COMMAND ${CMAKE_AR} cr ${outlibfile} *.o
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not using modifier: u?

r[ab][f][u]  - replace existing or insert new file(s) into the archive
[u]          - only replace files that are newer than current archive contents
[c]          - do not warn if the library had to be created

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this flag, it can add new symbols into the library.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it.

Copy link
Contributor

@typhoonzero typhoonzero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@gangliao gangliao merged commit 73e29d7 into PaddlePaddle:develop Jul 6, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants