Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Threaded C++ site update on FreeBSD #376

Closed
jteresco opened this issue Dec 12, 2020 · 2 comments · Fixed by #407
Closed

Threaded C++ site update on FreeBSD #376

jteresco opened this issue Dec 12, 2020 · 2 comments · Fixed by #407

Comments

@jteresco
Copy link
Contributor

When threading is enabled, compiling with g++ on noreaster, the following run-time error occurs:

ld-elf.so.1: /usr/local/lib/gcc5/libstdc++.so.6: version GLIBCXX_3.4.22 required by /tm-tmpfs/travelmapping/DataProcessing/siteupdate/cplusplus/siteupdate not found
@yakra
Copy link
Contributor

yakra commented Jan 22, 2021

I've just had a handful of successful runs (compiled with clang++) on noreaster, testing out as many as 8 threads.
HighwayData @ 27a625459516a8f5c53f73a24ce51416ab991677
UserData @ 3ec63996d33e499027a3d4c8f47abc0d3ce7659f

Some times for a few select tasks:

task 1 thread 4 threads 8 threads
Searching for near-miss points 2.9 0.7 0.4
Processing traveler list files 13 3.5 2
Augmenting travelers for detected concurrent segments 5.4 1.4 0.7
Computing stats per traveler 6.7 1.8 1.6
Creating per-traveler stats logs and augmenting data structure 28.3 8.5 6.7
total time @ ErrorList abort 195.8 125.7 123.4

Things seemed to slow down a bit on the bits that require more disk access, such as Reading waypoints for all routes and Writing near-miss point merged wpt files. @jteresco, I can see why you got that SSD. :)

Subgraph generation took about 38s even when I wrote the files to /dev/null. What this tells me is that httpd, mysqld, student programs, and whatever else keep the system busy enough to not leave as much L3 cache & memory bandwidth for siteupdate.

There's a catch, though.
A devilishly weird, obscure-but-reproducible little bug.
More to come at #375...

@yakra
Copy link
Contributor

yakra commented Jan 31, 2021

#407 closes this. The threaded version will compile & run on noreaster.
Other remaining issues:

@yakra yakra added the BSD label Jun 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants