Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use aligned_alloc or _aligned_malloc #884

Merged
merged 1 commit into from
Apr 13, 2023
Merged

Use aligned_alloc or _aligned_malloc #884

merged 1 commit into from
Apr 13, 2023

Conversation

prusnak
Copy link
Collaborator

@prusnak prusnak commented Apr 10, 2023

Fixes #880

@prusnak prusnak mentioned this pull request Apr 10, 2023
Closed
4 tasks
@sw
Copy link
Contributor

sw commented Apr 11, 2023

aligned_malloc requires the size to be a multiple of the alignment, which is probably fine for us. posix_memalign does not have that requirement.

ggerganov
ggerganov requested changes Apr 13, 2023
View reviewed changes
Copy link
Owner

@ggerganov ggerganov left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be better to add a GGML_ALIGNED_MALLOC macro somewhere at the start to avoid cumbersome #if in the code

@prusnak
Copy link
Collaborator Author

prusnak commented Apr 13, 2023

Reworked the PR to use GGML_ALIGNED_MALLOC and GGML_ALIGNED_FREE macros.

@prusnak
Introduce GGML_ALIGNED_MALLOC/GGML_ALIGNED_FREE macros
bac3966 
which allows us to use aligned_alloc or _aligned_malloc functions
@ggerganov ggerganov merged commit 6c24870 into master Apr 13, 2023
@ggerganov ggerganov deleted the aligned-alloc branch April 13, 2023 14:08
Deadsg pushed a commit to Deadsg/llama.cpp that referenced this pull request Dec 19, 2023
@abetlen
Add seed parameter support for completion and chat_completion requests.
86aeb9f 
Closes ggerganov#884
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ggerganov ggerganov ggerganov approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Use aligned_alloc() if aligned allocations are required
3 participants
@prusnak @sw @ggerganov