Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update readme for openmp on mac #267

Merged
merged 3 commits into from
Aug 16, 2023
Merged

Conversation

krrishnarraj
Copy link
Contributor

No description provided.

@karpathy
Copy link
Owner

I'm just really unable to get any substantial gains here, even following the new readme. Am I alone?

@krrishnarraj
Copy link
Contributor Author

Then gain was more than 2x on my macbook with openmp enabled.

Chip: M1 pro with 10 cpu cores
Memory: 32GB
OS: 13.5
/usr/bin/gcc --version: Apple clang version 14.0.3
/opt/homebrew/opt/llvm/bin/clang --version: Homebrew clang version 16.0.6

Launched the binary as: ./run llama2_7b.bin

Run tok/s
make runfast 1.79
make runfast CC=/opt/homebrew/opt/llvm/bin/clang 1.79
make runomp CC=/opt/homebrew/opt/llvm/bin/clang 4.01
above binary with OMP_NUM_THREADS=4 3.96

Not setting OMP_NUM_THREADS gave the best performance.

For further debugging, let us know your setup

@karpathy karpathy merged commit df6557a into karpathy:master Aug 16, 2023
@karpathy
Copy link
Owner

ok let's go with this for now ty

vinhtran2611 pushed a commit to vinhtran2611/llama2.c that referenced this pull request Jan 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants