-
Notifications
You must be signed in to change notification settings - Fork 140
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Refactor digital_fingerprinting stages and add morpheus-split conda r…
…ecipe (core, dfp, llm) (#1897) - [x] Move dfp stages to python/morpheus_dfp - [x] Create morpheus_dfp python package - [x] Rename morpheus-core conda recipe to morpheus-libs, a split package conda recipe that generates morpheus-core, morpheus-dfp and morpheus-llm - [x] Update github workflow to upload the morpheus-dfp conda package to anaconda Other pending tasks, including test and doc refactoring, is tracked via python/morpheus_dfp/todo.md Dev install now requires - ``` pip install -e python/morpheus pip install -e python/morpheus_dfp pip install -e python/morpheus_llm ``` Authors: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) Approvers: - Christopher Harris (https://github.com/cwharris) - David Gardner (https://github.com/dagardner-nv) URL: #1897
- Loading branch information
1 parent
1a5c7a7
commit 502cbf3
Showing
104 changed files
with
1,553 additions
and
497 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2022-2024, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| # It is assumed that this script is executed from the root of the repo directory by conda-build | ||
| # (https://conda-forge.org/docs/maintainer/knowledge_base.html#using-cmake) | ||
|
|
||
| # This will store all of the cmake args. Make sure to prepend args to allow | ||
| # incoming values to overwrite them | ||
|
|
||
| source $RECIPE_DIR/cmake_common.sh | ||
|
|
||
| CMAKE_ARGS=${CMAKE_ARGS:-""} | ||
|
|
||
| export CCACHE_BASEDIR=$(realpath ${SRC_DIR}/..) | ||
| export USE_SCCACHE=${USE_SCCACHE:-""} | ||
|
|
||
| if [[ -n "${MORPHEUS_CACHE_DIR}" ]]; then | ||
| # Set the cache variable, then set the Staging prefix to allow for host searching | ||
| CMAKE_ARGS="-DMORPHEUS_CACHE_DIR=${MORPHEUS_CACHE_DIR} ${CMAKE_ARGS}" | ||
|
|
||
| # Double check that the cache dir has been created | ||
| mkdir -p ${MORPHEUS_CACHE_DIR} | ||
| fi | ||
|
|
||
| # Enable core. Core is enabled by default and this is to just highlight that it is on | ||
| CMAKE_ARGS="-DMORPHEUS_BUILD_MORPHEUS_CORE=ON ${CMAKE_ARGS}" | ||
|
|
||
| # Disable dfp, llm and doca | ||
| CMAKE_ARGS="-DMORPHEUS_SUPPORT_DOCA=OFF ${CMAKE_ARGS}" | ||
| CMAKE_ARGS="-DMORPHEUS_BUILD_MORPHEUS_DFP=OFF ${CMAKE_ARGS}" | ||
| CMAKE_ARGS="-DMORPHEUS_BUILD_MORPHEUS_LLM=OFF ${CMAKE_ARGS}" | ||
|
|
||
| if [[ "${USE_SCCACHE}" == "1" ]]; then | ||
| CMAKE_ARGS="-DCCACHE_PROGRAM_PATH=$(which sccache) ${CMAKE_ARGS}" | ||
| fi | ||
|
|
||
| echo "CC : ${CC}" | ||
| echo "CXX : ${CXX}" | ||
| echo "CUDAHOSTCXX : ${CUDAHOSTCXX}" | ||
| echo "CUDA : ${CUDA}" | ||
| echo "CMAKE_ARGS : ${CMAKE_ARGS}" | ||
|
|
||
| echo "========Begin Env========" | ||
| env | ||
| echo "========End Env========" | ||
|
|
||
| BUILD_DIR="build-conda-core" | ||
|
|
||
| # remove the old build directory | ||
| if [[ -d "./${BUILD_DIR}" ]]; then | ||
| echo "Deleting old build dir at ./${BUILD_DIR}" | ||
| rm -rf "./${BUILD_DIR}/" | ||
| fi | ||
|
|
||
| # Run configure | ||
| cmake -B ${BUILD_DIR} \ | ||
| ${CMAKE_ARGS} \ | ||
| --log-level=verbose \ | ||
| . | ||
|
|
||
| # Build the components | ||
| cmake --build ${BUILD_DIR} -j${PARALLEL_LEVEL:-$(nproc)} --target install | ||
|
|
||
| # Install just the morpheus core python wheel components | ||
| ${PYTHON} -m pip install -vv ${BUILD_DIR}/python/morpheus/dist/*.whl |
Oops, something went wrong.