Crash in to_json_string #614
Comments
|
I think it's something to do with the way the object lifetime is managed, it's not clear how you stop it auto deleting, which seems to happen when you call the json function. |
|
It'd be really handy if the c++ libs could be used independently of the python bindings. But looking at the code, I suspect this isn't possible. |
|
any timeline_ = otio::create_safely_typed_any(new otio::Timeline(name_)); |
|
It is absolutely the intention that the C++ code should be completely usable without python. Some documentation on how to use the C++ code with respect to lifetime management was recently rescued from another branch and added to master, but I will try to take a look at the code and see if I see any gotchas that might be causing your crash. I suspect that you would want to use a Retainer around your raw pointer, to be more safe. You’d have to find the documentation to see examples of this.
Even if doing so stops your crash, this may or may not indicate a bug. I’d have to see more context of what you are doing.
…Sent from my iPad
On Nov 7, 2019, at 1:29 AM, Al Crate ***@***.***> wrote:
It'd be really handy if the c++ libs could be used independently of the python bindings. But looking at the code, I suspect this isn't possible.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
It looks like the default refcount is set to zero when you construct an object, as soon as you run a command that increments the count, it hits zero afterwards and then the object gets deleted. |
|
Hmm still having problems with using any of this, it just randomly dies. I must be messing up the object lifetime preservation somehow. |
|
Here's a trivial example. |
|
Yes, you’re mismanaging the lifetime ownership of this, which is understandable given that you’ve got zero documentation on how to use this.
—————
I apologize, but it seems the C++ documentation I wrote is *still* not available anywhere outside my own private branch, and it really needs to get out there so people have something to read.
For now, please see
https://opentimelineio-deb.readthedocs.io/en/latest/cxx-proposal/cxx.html#memory-management <https://opentimelineio-deb.readthedocs.io/en/latest/cxx-proposal/cxx.html#memory-management>
Also, please look specifically at the “Examples” section.
-------------------------------------------------------------
If you get back a raw pointer from an API call, you either need to just return it to someone else to deal with, without doing anything, or if you want to make use of it in any significant way, you need to “retain” it yourself, by doing
otio::Timeline *tmln_ = new otio::Timeline("test”);
otio::SerializableObject::Retainer<otio::Timeline> tptr(tmln);
tptr->to_json_string(…);
I admit it is not obvious that calling “to_json_string()” is a significant use of the object, but it is.
Internally, to_json_string() needed to retain the object for its own uses, and then released it when it was done.
That left your object destructed, so the second call failed, because you didn’t retain it before calling to_json_string().
Pretty much the only time you can use a raw pointer safely in C++ is if you just return it to someone else.
I know this is not exactly a comprehensive guide, but if you read the second on memory management, hopefully it’ll be clearer. Additionally, the proposal overall in the C++ section is fairly close to what we ended up with, so hopefully it’ll suffice.
Stephen: Where are we in getting even the coarse C++ stuff I wrote up available to people? It really needs to get exposed so as to answer basic questions like this.
… On Nov 7, 2019, at 8:05 AM, Al Crate ***@***.***> wrote:
Here's a trivial example.
otio::Timeline *tmln_ = new otio::Timeline("test”);
otio::ErrorStatus err;
std::cout << tmln_->to_json_string(&err) << std::endl;
std::cout << tmln_->to_json_string(&err) << std::endl;
Produces
{
"OTIO_SCHEMA": "Timeline.1",
"metadata": {},
"name": "asd",
"global_start_time": null,
"tracks": {
"OTIO_SCHEMA": "Stack.1",
"metadata": {},
"name": "tracks",
"source_range": null,
"effects": [],
"markers": [],
"children": []
}
}
terminate called after throwing an instance of 'std::system_error'
what(): Invalid argument
Aborted
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub <#614>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADAWJJILYOXAETGIISTNJULQSQ4F3ANCNFSM4JKDPJVQ>.
|
|
C++ docs are up on master now, sorry about that: https://opentimelineio.readthedocs.io/en/latest/cxx/cxx.html |
|
I think I meant:
tptr.value->to_json_string(…);
You access the underlying pointer in a Retainer<> via the .value member, not via auto-magic behavior.
… On Nov 7, 2019, at 8:57 AM, David Baraff ***@***.***> wrote:
Yes, you’re mismanaging the lifetime ownership of this, which is understandable given that you’ve got zero documentation on how to use this.
—————
I apologize, but it seems the C++ documentation I wrote is *still* not available anywhere outside my own private branch, and it really needs to get out there so people have something to read.
For now, please see
https://opentimelineio-deb.readthedocs.io/en/latest/cxx-proposal/cxx.html#memory-management <https://opentimelineio-deb.readthedocs.io/en/latest/cxx-proposal/cxx.html#memory-management>
Also, please look specifically at the “Examples” section.
-------------------------------------------------------------
If you get back a raw pointer from an API call, you either need to just return it to someone else to deal with, without doing anything, or if you want to make use of it in any significant way, you need to “retain” it yourself, by doing
otio::Timeline *tmln_ = new otio::Timeline("test”);
otio::SerializableObject::Retainer<otio::Timeline> tptr(tmln);
tptr->to_json_string(…);
I admit it is not obvious that calling “to_json_string()” is a significant use of the object, but it is.
Internally, to_json_string() needed to retain the object for its own uses, and then released it when it was done.
That left your object destructed, so the second call failed, because you didn’t retain it before calling to_json_string().
Pretty much the only time you can use a raw pointer safely in C++ is if you just return it to someone else.
I know this is not exactly a comprehensive guide, but if you read the second on memory management, hopefully it’ll be clearer. Additionally, the proposal overall in the C++ section is fairly close to what we ended up with, so hopefully it’ll suffice.
Stephen: Where are we in getting even the coarse C++ stuff I wrote up available to people? It really needs to get exposed so as to answer basic questions like this.
> On Nov 7, 2019, at 8:05 AM, Al Crate ***@***.*** ***@***.***>> wrote:
>
> Here's a trivial example.
> otio::Timeline *tmln_ = new otio::Timeline("test”);
> otio::ErrorStatus err;
> std::cout << tmln_->to_json_string(&err) << std::endl;
> std::cout << tmln_->to_json_string(&err) << std::endl;
> Produces
> {
> "OTIO_SCHEMA": "Timeline.1",
> "metadata": {},
> "name": "asd",
> "global_start_time": null,
> "tracks": {
> "OTIO_SCHEMA": "Stack.1",
> "metadata": {},
> "name": "tracks",
> "source_range": null,
> "effects": [],
> "markers": [],
> "children": []
> }
> }
> terminate called after throwing an instance of 'std::system_error'
> what(): Invalid argument
> Aborted
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub <#614>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADAWJJILYOXAETGIISTNJULQSQ4F3ANCNFSM4JKDPJVQ>.
>
|
|
That seems to work, but deserializing seems broken. (or I'm doing it wrong). otio::SerializableObject *obj = otio::SerializableObject::from_json_file(path, &err); 0 0x0000000065676e00 in ?? () |
|
Look, if the C++ code was that broken, nothing would work. We are just wrapping c++ with python. Could you post your entire (hopefully short) example that doesn’t work for you?
…Sent from my iPad
On Nov 8, 2019, at 1:04 AM, Al Crate ***@***.***> wrote:
That seems to work, but deserializing seems broken. (or I'm doing it wrong).
otio::SerializableObject *obj = otio::SerializableObject::from_json_file(path, &err);
if(obj){
timeline_ = obj;
retainer_ = otio::SerializableObject::Retainerotio::Timeline(timeline_);
}
As soon as I try and use the duration function on the timeline object it crashes.
timeline_->duration(&err)
0 0x0000000065676e00 in ?? ()
#1 0x00000000005dde97 in opentimelineio::v1_0::Timeline::duration (this=0x7ffeb8003540, error_status=0x7ffec5de9640)
at /builds/opentimelineio/0.12.0/bd667702e4/include/opentimelineio/timeline.h:46
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
That's pretty much it, I'm trying to deserialize a otio json file, but it crashes as soon as I try to use it. There are no examples in the documentation for how to do this, I did say that I'm probably doing it wrong. I'm creating a retainer before running the duration command, so it shouldn't be destroyed at this point. |
|
it looks like you created the retainer inside a scope so it was gone before you called duration. But I can’t tell unless I see what you wrote.
…Sent from my iPad
Sent from my iPad
> On Nov 8, 2019, at 7:43 AM, Al Crate ***@***.***> wrote:
That's pretty much it, I'm trying to deserialize a otio json file, but it crashes as soon as I try to use it.
There are no examples in the documentation for how to do this, I did say that I'm probably doing it wrong.
I'm creating a retainer before running the duration command, so it shouldn't be destroyed at this point.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
The declarations are outside, All I need to know is how I'm meant to deserialize a otio timeline, as my try above crashes, so I guess I'm not calling it correctly or using the wrong function. |
|
I don’t remember if any_cast fails violently when the type inside is wrong, but the any won’t be holding a Timeline*, so the cast was wrong. It will be holding an otio::SerializableObject::Retainer<>. I’ll send you an example when I have a chance to write one in a few hours.
… On Nov 11, 2019, at 12:39 AM, Al Crate ***@***.***> wrote:
The declarations are outside, All I need to know is how I'm meant to deserialize a otio timeline, as my try above crashes, so I guess I'm not calling it correctly or using the wrong function.
otio::Timeline *timeline_= nullptr;
otio::SerializableObject::Retainer<otio::Timeline> retainer_;
otio::any tmp;
if(deserialize_json_from_file(path, &tmp, &err)) {
timeline_ = otio::any_cast<otio::Timeline *>(tmp);
retainer_ = otio::SerializableObject::Retainer<otio::Timeline>(timeline_);
}
// crashes.
timeline_->duration(&err);
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub <#614>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADAWJJKV5ITG2F6J4FJ42ZTQTEK2PANCNFSM4JKDPJVQ>.
|
|
1. On my system, when I run your code, I get this (when I do the any_cast):
terminate called after throwing an instance of 'linb::bad_any_cast'
what(): bad any cast
I can’t tell you why you don’t get a similar termination in your environment, which obviously would have been less confusing than what you get.
2. Here’s what you could write to make this work:
int main() {
ErrorStatus err;
if (SerializableObject* so = SerializableObject::from_json_file("/home/deb/foo.otio", &err)) {
if (Timeline* tl = dynamic_cast<Timeline*>(so)) {
auto duration = tl->duration(&err);
printf("Got duration: %g %g\n", duration.rate(), duration.value());
}
}
}
If you want to make it safer, add a retainer:
int main() {
ErrorStatus err;
SerializableObject::Retainer<Timeline> retainer;
if (SerializableObject* so = SerializableObject::from_json_file("/home/deb/foo.otio", &err)) {
if (Timeline* tl = dynamic_cast<Timeline*>(so)) {
retainer = tl;
}
}
if (retainer.value) {
auto duration = retainer.value->duration();
}
}
3. Note: when deserializing, there is no magic that will ever give you back the derived type (in C++). You will always need to cast it.
Whenever you see an any, if it holds any kind of OTIO object, it will be holding a Retainer<>. Not a SerializableObject*, not a Timeline*, not a Retainer<Timeline>. Just a plain basic Retainer<>.
Hope that helps.
4. Clearly, someone should take the time to write up some more targeted documents about using otio directly in C++.
… On Nov 11, 2019, at 12:39 AM, Al Crate ***@***.***> wrote:
The declarations are outside, All I need to know is how I'm meant to deserialize a otio timeline, as my try above crashes, so I guess I'm not calling it correctly or using the wrong function.
otio::Timeline *timeline_= nullptr;
otio::SerializableObject::Retainer<otio::Timeline> retainer_;
otio::any tmp;
if(deserialize_json_from_file(path, &tmp, &err)) {
timeline_ = otio::any_cast<otio::Timeline *>(tmp);
retainer_ = otio::SerializableObject::Retainer<otio::Timeline>(timeline_);
}
// crashes.
timeline_->duration(&err);
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub <#614>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADAWJJKV5ITG2F6J4FJ42ZTQTEK2PANCNFSM4JKDPJVQ>.
|
|
Thanks for the help, I managed to get it working! It might make more sense if the python wrapped c++ objects where derived from the base classes with special reference counting or something. cheers! |
I've also had random SEGFAULT's when calling to_json_string more than once.
I'm playing with using the base opentimelineio code in a c++ app, but there's little documentation, so I'm not sure if I'm doing this correctly.
cheers!
opentimelineio::OPENTIMELINEIO_VERSION::Timeline *timeline_ = new opentimelineio::OPENTIMELINEIO_VERSION::Timeline("test");
opentimelineio::OPENTIMELINEIO_VERSION::ErrorStatus err;
aout(this) << timeline_->to_json_string(&err, 0) << std::endl;
aout(this) << timeline_->to_json_string(&err, 0) << std::endl;
#0 0x00007ffff466d1f7 in raise () from /lib64/libc.so.6
#1 0x00007ffff466e8e8 in abort () from /lib64/libc.so.6
#2 0x00007ffff4fa55c2 in __gnu_cxx::__verbose_terminate_handler () at ../../../../gcc/libstdc++-v3/libsupc++/vterminate.cc:95
#3 0x00007ffff4fa34a6 in __cxxabiv1::__terminate(void ()()) () at ../../../../gcc/libstdc++-v3/libsupc++/eh_terminate.cc:47
#4 0x00007ffff4fa34e1 in std::terminate () at ../../../../gcc/libstdc++-v3/libsupc++/eh_terminate.cc:57
#5 0x00007ffff4fa3723 in __cxxabiv1::__cxa_throw (obj=obj@entry=0xfe7000, tinfo=tinfo@entry=0x7ffff529f8a0 , dest=dest@entry=0x7ffff4fd36e0 std::system_error::~system_error())
at ../../../../gcc/libstdc++-v3/libsupc++/eh_throw.cc:95
#6 0x00007ffff4fd38e7 in std::__throw_system_error (__i=22) at /user_data/.tmp/bobscratch-5q6dzG/build/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/char_traits.h:320
#7 0x00007ffff6d0b0a4 in opentimelineio::v1_0::SerializableObject::_managed_retain() () from /builds/opentimelineio/0.12.0/bd667702e4/lib/libopentimelineio.so
#8 0x00007ffff6d0c1f0 in opentimelineio::v1_0::SerializableObject::to_json_string[abi:cxx11](opentimelineio::v1_0::ErrorStatus, int) const () from /builds/opentimelineio/0.12.0/bd667702e4/lib/libopentimelineio.so
The text was updated successfully, but these errors were encountered: