C++ site update to production #375
Comments
|
A first question on this: what C++ compiler and version do you use when you build it for Linux? g++ 9.3.0 compiles and links, but I get a run time error on noreaster: ld-elf.so.1: /usr/local/lib/gcc5/libstdc++.so.6: version GLIBCXX_3.4.22 required by /tm-tmpfs/travelmapping/DataProcessing/siteupdate/cplusplus/a.out not found clang 10.0.1 doesn't compile it successfully, being unhappy with the use of subscript operators. |
Other than #281 (just cosmetic, an easy fix for a clean readable siteupdate.log), yes! I believe I have mutexes around everything else I need to, and even some things I don't. The last crashes & glitches were fixed in #333. Since then I've run siteupdate many thousands of times during speed tests with no other known issues except #327, which "is super obscure, and not too big a deal in the real world". It shouldn't hold the C++ version back from going live, and may become moot anyway once multi-threaded HighwayGraph setup enters the picture.
No need for super-fast storage subsystems. NMP_merged files take about 0.5s on lab2. Unsure what the exact bottleneck is here, but everything else is CPU/memory bound. Graph files are written to disk at less MBps than nmp_merged.
Yup. Same message I got. What versions do I use?
I'm new to clang *checks wikipedia article*. |
|
All right, good. We can think about moving in this direction. As far as clang, it gives pages of errors, many of which look like this: In file included from siteupdate.cpp:72:
./classes/DatacheckEntry.cpp:69:14: error: type 'std::array' (aka 'array, allocator >, 6>') does not provide a subscript operator
if (fpentry[0] != route->root) return 0;
~~~~~~~^~
I don't think it really matters to me if we compile with clang, g++, or something else, so I wouldn't consider that a high priority. Here's a new Issue about getting this compiled and running with pthreads support: #376 |
|
My working tree on noreaster has (for the moment) some changes to 4b9c719 to enable compilation with clang++.
Even though this memory is allocated on the heap, I'm seeing consistent values for all Even if I comment out
This will take a pretty deep dive, one that I don't have the attention span for right yet. |
|
The bug above is reproducible with Starting to get the impression it's related to a specific point (or range rather) in memory.
Checking out an older HighwayData commit to cause all the waypoint objects (and thus that particular colocation list pointer) to be stored in different places seems to do the trick. The pointer stays valid and siteupdate runs to completion. So what's the deal? Is there bad RAM? Why do we lose exactly 0x900000000, with two ones turning to zeros?
|
What memory locations?
Doing a
siteupdate.log with some debug text: What are we looking at here?
When does it happen?
In the siteupdate.log excerpt above, note how when detecting colocations while reading in WPTs for usala, both diff --git a/siteupdate/cplusplus/classes/WaypointQuadtree/WaypointQuadtree.cpp b/siteupdate/cplusplus/classes/WaypointQuadtree/WaypointQuadtree.cpp
index 334524e..296c17d 100644
--- a/siteupdate/cplusplus/classes/WaypointQuadtree/WaypointQuadtree.cpp
+++ b/siteupdate/cplusplus/classes/WaypointQuadtree/WaypointQuadtree.cpp
@@ -58,6 +58,19 @@ void WaypointQuadtree::insert(Waypoint *w, bool init)
}
other_w->colocated->push_front(w);
w->colocated = other_w->colocated;
+
+ // <<< DEBUG
+ if (w->str() == "la.la1088 LA36 (30.433911,-89.91344)")
+ { std::cout << "DEBUG: this=la.la1088 LA36 (30.433911,-89.91344)" << std::endl;
+ std::cout << "DEBUG: this->colocated=" << (void*)w->colocated << std::endl;
+ std::cout << "DEBUG: other->colocated=" << (void*)other_w->colocated << std::endl;
+ }
+ if (other_w->str() == "la.la1088 LA36 (30.433911,-89.91344)")
+ { std::cout << "DEBUG: other=la.la1088 LA36 (30.433911,-89.91344)" << std::endl;
+ std::cout << "DEBUG: this->colocated=" << (void*)w->colocated << std::endl;
+ std::cout << "DEBUG: other->colocated=" << (void*)other_w->colocated << std::endl;
+
+ }// DEBUG >>>
}
}
if (!w->colocated || w == w->colocated->back())What happens after this?
siteupdate.cpp #ifdef threading_enabled
// stuff that never happens in siteupdateST because of the #ifdef
#else
for (HighwaySystem* h : highway_systems)
{ std::cout << h->systemname << std::flush;
bool usa_flag = h->country->first == "USA";
for (Route* r : h->route_list)
r->read_wpt(&all_waypoints, &el, args.highwaydatapath+"/hwy_data", usa_flag, &all_wpt_files);
std::cout << "!" << std::endl;
}
#endif
// DEBUG //
list<Waypoint*> schmist = all_waypoints.point_list();
schmist.sort();
ofstream schmile("schmile");
for (Waypoint *w : schmist)
{ schmile << &w->colocated << '\t';
schmile << w->colocated << '\t';
if (!w->colocated) schmile << '\t';
schmile << w->str() << std::endl;
}
cout << et.et() << "Sorting waypoints in Quadtree." << endl;Top: existing code in
What do we see here? [yakra@noreaster ~/TravelMapping/DataProcessing/siteupdate/cplusplus]$ egrep 'la.la1088 LA36|la.la0036 LA1088' schmile
0x8fe9029e8 0x9000aee40 la.la0036 LA1088 (30.433911,-89.91344)
0x900080048 0xaee40 la.la1088 LA36 (30.433911,-89.91344)It'd seem that nothing else happens after WaypointQuadtree insertion before this sanity check, and that this one pointer's value magically changes. I'd almost have no rational explanation other than possibly bad RAM, but not all is necessarily as it seems. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Home come the pointer is fine if located < 0x900000000, but gets mangled if located > 0x900000000? Narrowing down, with
|
...Or is it?
|
* C arrays for updates entries * colocated moved with Waypoint (there appear to be sizeof/offset issues in clang, as noted in TravelMapping#375)
* C arrays for updates entries * colocated moved within Waypoint (there appear to be sizeof/offset issues in clang, as noted in TravelMapping#375)
|
A full pass thru siteupdate with valgrind will be worthwhile. |
|
"[your] memory management" -> "C++'s memory management" |
|
Heh. My memory mgmt too, considering what I've done (and not done) knowing the language's shortcomings. foo* bar = new foo;
// deleted on termination of programlittered throughout the source. I explicitly Looks like Valgrind has something to say about all this. Another minor surprise I didn't look into was something about a branch relying on uninitialized data. Edge construction or compression IIRC. A bit of a surprise; I'd think I'd notice some effects... |
It happens exactly once, for US19TrkPit. format needs to be initialized before the destructor is called.
We get 3 errors when the destructor calls |
Nope. We neglect to delete it all the time, even with the fix in place, leaking 1 byte per thread per edge. The destructor calls Best to delete Better yet?
|
|
We're there! |
I'd like to work toward using the C++ version of the site update program. @yakra are you confident in the threaded version's readiness for production site updates?
I think that moving to the C++ program its inherent efficiency advantages over Python and the ability to do some meaningful multithreading, in conjunction with the addition of SSDs and/or using memory filesystem could really speed the whole process significantly.
The text was updated successfully, but these errors were encountered: