Skip to content

Add try/catch stmt, @strict directive & runtime error handling to E2 #2174

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Aug 14, 2021
Merged

Add try/catch stmt, @strict directive & runtime error handling to E2 #2174

merged 21 commits into from
Aug 14, 2021

Conversation

thevurv
Copy link

@thevurv thevurv commented May 28, 2021

Adds a try / catch statement to E2 that allows users to catch any errors that occur inside of the block, and if so, run the catch block, setting the variable name provided to a new local variable that contains the error string.

Edit: Could also add finally to fit with golo but I think that's overkill and wouldn't be used

Fixes #1141 (But it's closed so I don't know whether this would be accepted at all)

Uses

I have a few uses for this even if e2 is supposed to be "simple and error free"

  1. External cores throwing errors
  2. Nonexistent string calls
  3. For better custom error handling if people want to be verbose about using something incorrectly in an E2 Library or something.

As for nonexistent string calls, it can also be solved by adding a "defined" function for #ifdef at runtime but I think that just looks weird. Can also make a PR for that.

Syntax

try {
	error("my error")
} catch(E) {
	assert(E == "my error", "Test failed!")
	print("Success")
}
#--> Success

Updated w/ @strict:

@strict

noentity():applyForce(vec()) # --> Runtime error about the entity being invalid

Adds a try / catch statement to E2 that allows users to catch any errors that occur inside of the block, and if so, run the catch block, setting the variable name provided to a new local variable that contains the error string.
@thegrb93
Copy link
Contributor

Can you also make 'not enough memory' uncatchable? Pretty impressive work.

@thegrb93
Copy link
Contributor

I don't see any issues. Will merge if there's no objections

@Divran
Copy link
Contributor

Divran commented Jun 2, 2021

My opinion is that this is entirely unnecessary in a language that's designed to never have runtime errors.

It'd be far simpler for both us and the end user to just nail down all runtime errors we currently have and make them fail silently, too.
The only ones we have afaik is running a nonexistant string function, and running a nonexistant udf. There aren't many others.
In both of those cases, simply adding a function that checks if the given function (with parameters) exists and returns 1 or 0 would solve all issues.

Using it with error to cause manual jumps to the catch case is just an overkill goto statement and isn't a feature we want.

That said, try catch isn't a bad feature and I'm not against it by itself. The problem is E2 isn't designed for it, so this would only add essentially more bloat to an already bloated language.

@Divran
Copy link
Contributor

Divran commented Jun 2, 2021

If a @strict mode were to be added, I'd support try catch.

@thegrb93
Copy link
Contributor

thegrb93 commented Jun 3, 2021

My only worry is certain cases where catching an error would leave E2 in a bad state. I can't think of any but that doesn't mean one doesn't exist.

@bigdogmat
Copy link
Member

bigdogmat commented Jun 3, 2021

return, break, continue, and exit will have to be whitelisted. Might be a few others I can't remember atm

Edit: Not sure how this one should be whitelisted considering the stack trace

error("Attempted to reset the E2 twice in the same tick!", 2)

Edit2: This'll also catch any actual Lua errors in extension code, which I'm not sure how I feel about being catchable. All of the above would fixed if this only caught special objects that the E2 error function produced.

Edit3: Personally I feel like for the small amount of uses there's not really a point to try-catch in E2. If we migrated functions that silently failed to instead throwing then I'd have more support behind this.

@thevurv
Copy link
Author

thevurv commented Jun 3, 2021

  1. E2 invalidates if you type return/break/continue inside try afaik (will check again later), didn't think about exit though. And I'll look into "Attempted to reset the E2 twice in the same tick!" that would've been bad if this got through..

  2. Catching lua errors should be fine, if we make it only catch e2 errors it wouldn't be able to handle as much but I could see the concern I guess

  3. Will probably end up taking up divran's @strict idea because it'd make this useful and I don't think we want to break a ton of e2s that depend on basically undefined behavior..

Would replace instances of return <default_value> with return throw("Strict Error Message", <optional default_value in case you don't want to use the type one>)

Could do it through the preprocessor or by replacing the function, both would just use the default return value in wire_expression_types/wire_expression_types2

@Divran
Copy link
Contributor

Divran commented Jun 3, 2021

error("Attempted to reset the E2 twice in the same tick!", 2)

This error should not be caught by any try catch because it's there to prevent you from lagging the server by spamming the reset function. Unless a better/different way to prevent that is implemented.

Edit2: This'll also catch any actual Lua errors in extension code, which I'm not sure how I feel about being catchable. All of the above would fixed if this only caught special objects that the E2 error function produced.

there shouldn't be any lua errors in extensions in wiremod, that's how e2 was designed. there are currently a few of them like I mentioned above, but the majority of them can be swapped out for non-failing ones.
third party extensions however can definitely cause lua errors.

@bigdogmat
Copy link
Member

bigdogmat commented Jun 3, 2021

This error should not be caught by any try catch because it's there to prevent you from lagging the server by spamming the reset function. Unless a better/different way to prevent that is implemented.

That's my point, but currently it can't be whitelisted considering it has a stack trace.

there shouldn't be any lua errors in extensions in wiremod, that's how e2 was designed. there are currently a few of them like I mentioned above, but the majority of them can be swapped out for non-failing ones.
third party extensions however can definitely cause lua errors.

I'm not talking about known/intentional errors, I mean runtime errors that are the side effect of errors in extension code. E.g. an extension attempts to do something with an invalid entity. Or an addon overwrites a function that wiremod uses and it errors. These types of errors would be caught by the catch, when they should completely halt the E2 so that they can fixed and known.

@thevurv thevurv marked this pull request as draft June 4, 2021 22:06
@thevurv
Copy link
Author

thevurv commented Jun 4, 2021

I'll leave this on draft as I try and implement @strict and try and deal with specific E2 error throwing etc..

thevurv added 2 commits June 11, 2021 13:13
* Made ``exit`` skip the try catch block as a whole.
* Added ``@strict`` directive
* Refactored the directives a bit to add ``@strict``
* Added a ``throw`` function to registerFunction that either throws an error or returns the default value of the function inside using the type defined. You can override this with the second argument.
* Edited the preprocessor for this ^
* Edited the entire core/entity file to preview how this would look.
* Some copytype calls still existed and were replaced with ``E2Lib.fixDefault``
@thevurv
Copy link
Author

thevurv commented Jun 11, 2021

Ok, first draft of @strict is done. The internals for throw are really gross but I really don't know how else to go about it. It works clean inside of registerFunction / e2functions though. It will automatically return the default value for the function's return type, or the second arg to throw which would override it. I edited the entire entity core to use throw so it can be tested.

@TomyLobo
Copy link
Contributor

On your specific implementation:
Right now, this is implemented with a blacklist. This is a no-go.
For security reasons, this needs to be a whitelist, otherwise we need anticipate each and every shenanigan someone with too much time on their hand will think of in the future.
You can have a special prefix for errors thrown by throw, so you can blanket-whitelist those.

On the concept in general:
Tbh, this feels like adding throw/catch to C or something. It just doesn't fit with the way E2 works.
I also don't really see Exceptions as the non-plus-ultra way to go anymore.
They introduce non-local returns, which make the control flow harder to anticipate and follow.
In Lua, you have the extra problem that you lack a "finally" feature, which makes cleanup so much harder.
It also lakes type-safe exceptions, which makes it harder to distinguish exceptions by cause.
Lua generally doesn't give us much of anything in terms of exception handling, so I'm worried this is going to be a big mess.

Like I said, you need a whitelist for security reasons.
With that, I don't really see how you can make this very useful, since that kills 2 of your 3 use cases (if I interpret the 2nd point correctly).

I'm sorry people here led you to believe that your idea is going somewhere.
If CaptainPRICE (the person who created the issue you linked) has an idea, it's generally understood that it's a "just-so" feature and he just creates those cause he wants to have an impact or something.
He's more of a bad-idea guy.

@thevurv
Copy link
Author

thevurv commented Jun 12, 2021

It wasn't really price's idea that made me do this. I already implemented this in my addon through making a function that basically pcalled string calls and that worked fine, but was really nasty because of how E2 doesn't have functions as variables. I just brought that issue up since it helped push my case that people want this.

I think the discussion on @strict has made this useful in trying to get E2 to be more verbose on what's happening with less undefined behavior. I'll also make it whitelisted to throw soon enough since this would be the main use case now.

thevurv added 4 commits June 12, 2021 10:02
Throwing is much more explicit now. You call self:throw(msg, var) and var is returned if the chip isn't strict. Got rid of the throw hack to try and get the default type. The only problem here now is trying to get some debug info about where the error happened. Right now it's not very useful and isn't able to point to the function calling self:throw for some reason.
Hardcoded the starting point for this line.  (#"trigger" + 2)
Native lua/glua error() will always error try and catch blocks for security reasons.

The only errors that will be caught are created by E2Lib.catchableError. (self.throw and e2's error)

Replaced context:throw with context.throw to be more memory efficient i guess.
Added ``@strict`` compat to a lot more libraries. I think adding more would make this PR too big, it's already pretty huge.
Additionally made some code a bit more clean and optimized & used E2Lib.newE2Table more. (Notably with bone fwd up right)
@thegrb93
Copy link
Contributor

Making functions that used to not throw now throw is definitely going to get rejected. E2 doesn't follow that paradigm.

@thegrb93
Copy link
Contributor

thegrb93 commented Jun 14, 2021

Unless the throw is only going to happen with @strict? I guess that makes more sense.

@thevurv
Copy link
Author

thevurv commented Jun 14, 2021

@thevurv
Copy link
Author

thevurv commented Jun 26, 2021

I'm not sure what to do with this anymore, I did a lot of changes to add throw compat to functions, but there's no way to debug this as far as I know.. so you just get a lot of errors that don't point to any lua or e2 function at all.. I could throw in a debug.getinfo but it doesn't even point to the original e2function source This now has runtime e2 source traces

@thegrb93
Copy link
Contributor

thegrb93 commented Jun 26, 2021

Usually the thing thing to do in that case is to revert files and code chunks until its working again. Then you know what's breaking it

@thevurv thevurv changed the title Add try/catch statement to Expression2 Add try/catch stmt, @strict directive & runtime error handling to E2 Jul 11, 2021
@thegrb93
Copy link
Contributor

Unfortunately I have no idea 😅

@thegrb93
Copy link
Contributor

thegrb93 commented Jul 15, 2021

Could you make a suite of unit tests added to the existing tests? Might be the best course of action

* Add a ``strict.txt`` unit test to data/expression2/tests.
* Also add a small try/catch test to ``parsing.txt``

* Fix assert() e2function not throwing a catchable error. It now provides a runtime src position like the rest of the errors

* Fix exit() not exiting the chip. The ``Skip`` table was just making exit() exit the try / catch block rather than passing it to the error handler to exit the whole chip.
@thevurv
Copy link
Author

thevurv commented Jul 15, 2021

Done, adding these actually brought my attention to a good amount of stuff, 👍.

Should probably have more tests in the future for general stuff instead of just that parsing and regression tests though. And could make them run with a concommand by invoking the compiler and just assigning the owner to the caller.. but that will be for another PR

@thegrb93
Copy link
Contributor

Could you add a test that catches an error when trying to call a non-existent function? ("customFunction")(). Is it able to catch that?

@thevurv
Copy link
Author

thevurv commented Jul 15, 2021

It already exists in tests/strict.txt

@thevurv
Copy link
Author

thevurv commented Jul 16, 2021

Looks like this might have broken #include, getting some weird error on my local server. Would be nice for another unit test using that in case I didn't notice and this got merged and broke everything..

@thegrb93
Copy link
Contributor

Yeah sounds like one of the cases I was afraid of. Catching an error and resuming with e2 being in a bad state.

@thevurv
Copy link
Author

thevurv commented Jul 17, 2021

It was actually me just calling a function I added (CallInstruction) incorrectly and getting a cryptic nil error with no stack trace.

Works fine now, but I then realized that #include is not preprocessed and is an actual statement.. that would work inside of try / catch..

try {
	#include ...
} catch(Err) {
	print(Err)
}

and that somehow breaks the stack / the instruction to find the Err variable.. (why is it a statement :P)

@thevurv thevurv marked this pull request as draft July 24, 2021 22:58
* Include was messed up because whoever implemented it didn't think about needing the scope after erroring. It didn't load the old scope back which messed up trying to use try/catch.

* Fixed runtime traces because before I tried to make it only take from the seq instruction which would always make the error come from where the first instruction is.

* Renamed E2Lib.catchableError to raiseException to be more clear. Also has a 4th param for whether it's catchable, that is default true.

* Added E2Lib.unpackException(struct) which is a boilerplate bridge function for when you want backwards compatibility with error messages & error structs in pcalls.
@thevurv
Copy link
Author

thevurv commented Aug 4, 2021

Fixed the issue with #include which was just an oversight when it was implemented.

Think it's ready but I need to resolve all of the conflicts with do-while now..

thevurv added 2 commits August 4, 2021 15:36
commit 02d5dfd
Author: stepa2 <sssstepa452@yandex.ru>
Date:   Thu Jul 29 20:48:37 2021 +0300

    Implemented do-while statement for E2 (issue #2191) (#2193)

    * Allow calling entity:setPos and entity:setAng on entities without collisions
    Also allow calling entity:propManipulate, but position and rotation is only updated

    * Update prop.lua

    Removed tailing whitespace, fixed luacheck warning

    * Implemented do-while statement for E2 (#2191)

    * Removed debug messages I fogot to remove, added parsing tests, disabled bugged optimization

    * Linting pass

commit 73b4dbf
Author: thegrb93 <grbrown93@sbcglobal.net>
Date:   Sun Jul 25 00:52:11 2021 -0400

    Fix explodeRE (#2204)

commit 6a5d4a2
Author: Derpius <49565664+Derpius@users.noreply.github.com>
Date:   Sat Jul 24 23:58:13 2021 +0100

    Add 4th option to view request bypass (#2198)

    * Add 4th option to view request bypass

    * Update comment

commit adca748
Author: thegrb93 <grbrown93@sbcglobal.net>
Date:   Sat Jul 24 18:57:53 2021 -0400

    Fix light sprite size (#2201)

    * Fix light sprite size

    * Revert sizing code. Also simplified because you can't even see the smaller sprites

    * Made spritesize an option/input

    * Add back tube light sprites

    * Add missing clamp

commit c4d333a
Author: Divran <arviddivran@gmail.com>
Date:   Sat Jul 24 23:19:26 2021 +0200

    fixed crash exploit

commit 966b1e4
Author: AbigailBuccaneer <AbigailBuccaneer@users.noreply.github.com>
Date:   Sat Jul 17 15:09:26 2021 +0100

    Restore expression2 ghost functionality

    Commit ec99005 accidentally prevent the
    E2 tool from showing ghosts.

commit 01c36ae
Merge: f85c234 1a1ddd3
Author: Divran <arviddivran@gmail.com>
Date:   Tue Jul 13 22:54:54 2021 +0200

    Merge pull request #2197 from sammyt291/patch-1

    Light sprite colors and size

commit 1a1ddd3
Author: sammyt291 <samuel_canning@hotmail.co.uk>
Date:   Sun Jul 11 20:07:42 2021 +0100

    Light sprite colors and size

    Changed the smaller 3 sprites on the tubular light to match the color of the main sprite on the light
    Allowed the sprite overlay to be sized according to the size slider already present, clamped to 128 to avoid changing existing contraptions with large-size glows enabled.

commit f85c234
Author: stepa2 <sssstepa452@yandex.ru>
Date:   Sat Jul 3 03:35:04 2021 +0300

    Allow calling entity:setPos and entity:setAng on entities without collisions (#2188)

    * Allow calling entity:setPos and entity:setAng on entities without collisions
    Also allow calling entity:propManipulate, but position and rotation is only updated

    * Update prop.lua

    Removed tailing whitespace, fixed luacheck warning

commit 860604b
Author: CoreyLee Hassell <Anticept@users.noreply.github.com>
Date:   Fri Jul 2 20:34:47 2021 -0400

    Revert "Delete json funcs until Facepunch/garrysmod-issues#4976 is fixed (#2177)" (#2185)

    This reverts commit c6ea454.

commit eb3841f
Author: thegrb93 <grbrown93@sbcglobal.net>
Date:   Fri Jun 25 19:50:34 2021 -0400

    Improve regex and fix pcall not returning (#2186)

    * Improve regex and fix pcall not returning

    * Don't use %b

commit 1ef0f63
Author: thegrb93 <grbrown93@sbcglobal.net>
Date:   Thu Jun 24 02:25:28 2021 -0400

    Add level setting, also play regardless of distance/location (#2183)

commit ec99005
Author: 100PXSquared <100pxsquared@gmail.com>
Date:   Wed Jun 23 08:01:26 2021 +0100

    E2 View Requests System (#2157)

    * Add E2 view requests system

    * Prevent prop protection stopping view requests

    * Perform CanTool check if chip owner is invalid

    * Refactor checks in TOOL:Think

    * Add SteamID whitelist convar to view requests

    * Fix request answer netmsg validation

    * Add EoF newline

    * Refactor E2 view requests

    * Replace bypass list with bypass mode convar

    * Implement allow always

    * Replace checks in remote updater code request

    * Fix copy paste induced bug

commit f888eb1
Author: thegrb93 <grbrown93@sbcglobal.net>
Date:   Wed Jun 23 02:39:23 2021 -0400

    Fix turret behavior if parented (#2184)

commit f8335f9
Author: thegrb93 <grbrown93@sbcglobal.net>
Date:   Fri Jun 18 11:35:20 2021 -0400

    Limit regex cpu consumption (#2181)

    * Limit regex cpu consumption. Fixes: #2063 Related (Facepunch/garrysmod-requests#1878)

    * Better error message

commit d0a84b7
Author: Divran <arviddivran@gmail.com>
Date:   Fri Jun 18 00:44:56 2021 +0200

    sound duration is very expensive

commit f54ff0a
Author: Divran <arviddivran@gmail.com>
Date:   Fri Jun 18 00:13:01 2021 +0200

    fixed auto indenting

    auto un-indenting after typing } was broken (triggered even if the same
    line contained a {), and did not add itself to the undo table, so could
    not be undone.
    auto indenting after typing { was also broken if the same line contained
    a }

    adjusted digi screen bandwidth
    slowed down clientside digi screen rendering to prevent fps lag

    fixed hologram queues after last update broke it

commit 4855366
Author: Divran <arviddivran@gmail.com>
Date:   Wed Jun 16 18:58:31 2021 +0200

    digi screen max global bandwidth

commit 7b5ac40
Author: Divran <arviddivran@gmail.com>
Date:   Wed Jun 16 02:18:54 2021 +0200

    improved digi screen performance

    added cpu time based usage checks serverside
    and coroutine processing clientside
    hopefully this is (at least v1 of) the final strip of duct tape for this
    notorious entity

commit d8cdc76
Author: Divran <arviddivran@gmail.com>
Date:   Tue Jun 15 22:37:39 2021 +0200

    fixed multiple crash and lag exploits

    reset() no longer resets e2 quotas
    emergency ram shutdown now also clears gtable memory
    hologram scale/clip update rate limited to 0.1s and opcost increased to
    30
    and a few others

commit c6ea454
Author: thegrb93 <grbrown93@sbcglobal.net>
Date:   Sat Jun 12 19:13:26 2021 -0400

    Delete json funcs until Facepunch/garrysmod-issues#4976 is fixed (#2177)
@thevurv
Copy link
Author

thevurv commented Aug 4, 2021

Fixed conflicts, github still says there are for some reason, don't know if it's because I made it a squash merge... Guess I'll need to revert that, thanks gh :/

@thevurv thevurv marked this pull request as ready for review August 5, 2021 02:27
@thegrb93
Copy link
Contributor

thegrb93 commented Aug 5, 2021

Squishing isn't a good idea until the review is over anyway (and the merge button does this automatically) because it requires a force push which then requires reviewers to review the whole thing again.

* Fixed expressions being missing thanks to missing return keyword
* Removed useless local variable in var operator
@thegrb93
Copy link
Contributor

thegrb93 commented Aug 8, 2021

@Vurv78 hows the progress here?

@thevurv
Copy link
Author

thevurv commented Aug 8, 2021

It's done as far as I know this time. Fixed the #include issue (which was more include's problem than try/catch) and runtime traces. Probably want to test it out a lot though since I changed a lot of stuff

Not a bug or anything but something I randomly came across..
@thegrb93
Copy link
Contributor

I didn't see any bad code. This is a high risk change though. We can merge and watch for issues reported, and we can revert if it isn't fixed.

@thegrb93 thegrb93 merged commit 5fc05d5 into wiremod:master Aug 14, 2021
@thevurv thevurv deleted the try_catch branch August 25, 2021 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

E2: Exception/Error handling (try-catch statement)
5 participants